Hardening a Fresh Server: From First SSH to Firewall

You've received the server, the IP and root password are in hand, and the first thing many people do is log in with ssh root@IP and immediately change the SSH port. Half an hour later, the terminal window disconnects and they can no longer get in. If you're standing at this exact point right now, change the order of tasks. Server hardening isn't a checklist to run top to bottom; it has an order that, if not followed, will throw you out of the server yourself.

Before any change, build a way back

Any change that could cut off access must have a return path. You have two options: either keep the KVM/VNC console open from the server management panel, or create a second user with sudo access beforehand that works independently of the current user.

adduser deploy
usermod -aG sudo deploy
mkdir -p /home/deploy/.ssh
cp /root/.ssh/authorized_keys /home/deploy/.ssh/
chown -R deploy:deploy /home/deploy/.ssh
chmod 700 /home/deploy/.ssh
chmod 600 /home/deploy/.ssh/authorized_keys

Now log in from a second terminal with ssh deploy@IP and keep it open. Make all subsequent changes from that terminal. If you get locked out somewhere, that open terminal will save you. If things go completely wrong and you have no other option, reinstalling the server and working with rescue mode is the last resort; but reaching that point means an extra hour of work.

Update first, everything else after

A freshly delivered server usually has an image that was released weeks or months ago. Before installing any package, update the repositories:

apt update && apt upgrade -y
apt install -y unattended-upgrades
dpkg-reconfigure --priority=low unattended-upgrades

On Debian 12 and Ubuntu 22.04 these commands run without issues. If the kernel was updated, needrestart or reboot is required; but postpone the reboot until after configuring the firewall, so that if the server doesn't come back up, you know where the problem is.

Here's where people make a mistake: many run apt upgrade on a server that doesn't have a firewall yet, and after the reboot they find their services won't come up because the new version overwrote a config. Before upgrading major packages, check apt list --upgradable, and if MySQL or Nginx is in the list, back up the config first.

Close SSH in a way that doesn't lock yourself out

Order matters. First put the public key on the server, then disable the password. If you do it the other way around, you'll lose access between the two steps.

1. Place the public key in the new user's ~/.ssh/authorized_keys.
2. Log in from a separate terminal with the key and make sure it works.
3. Create the file /etc/ssh/sshd_config.d/99-hardening.conf.
4. Test the config validity with sshd -t.
5. Apply with systemctl reload sshd, not restart.

PermitRootLogin no
PasswordAuthentication no
PubkeyAuthentication yes
MaxAuthTries 3
LoginGraceTime 20
AllowUsers deploy

About changing the SSH port: I keep the default 22. Changing the port doesn't add real security, it only reduces log noise, and in exchange, if you misconfigure the firewall or your ISP blocks the port, you waste your time. If you insist, open the new port in the firewall first, then reload sshd, and keep the old port open for a few days.

Firewall: only what's necessary

On Ubuntu and Debian, ufw is the simplest way. But before enabling it, add the SSH port:

ufw default deny incoming
ufw default allow outgoing
ufw allow 22/tcp
ufw allow 80/tcp
ufw allow 443/tcp
ufw enable
ufw status verbose

If you have Docker on the server, an important note: Docker writes its own iptables rules directly and bypasses ufw. That means a port that's closed in ufw may still be reachable from outside. Use docker ps to see which ports are published, and in compose only bind the ports that are truly necessary, for example 127.0.0.1:3306:3306 instead of 3306:3306. For proper setup, see setting up Docker on a VPS.

If you have a MySQL service, definitely close port 3306 from outside. Even if you have a strong password, scanners keep trying. MySQL security and restricting access covers the steps for removing the anonymous user and closing bind-address.

Fail2ban and monitoring: when the attack starts

After closing SSH, look at the /var/log/auth.log log. If the server has been on the internet for a few hours, you'll see hundreds of failed attempts. Fail2ban blocks these automatically:

apt install -y fail2ban
systemctl enable --now fail2ban
fail2ban-client status sshd

The output of fail2ban-client status sshd shows the current number of blocked entries. If the number is zero and the server has been on the internet for a few days, it means the jail isn't properly enabled or the log is in a different path.

Take monitoring seriously. A server without monitoring won't tell you when it fills up or load spikes, until the moment the site goes down. Tools like htop, iotop, and netdata are enough. If load spikes and you can't figure out why, diagnosing the cause of high server load walks through it step by step.

SSL, backup, and what remains after hardening

Server hardening isn't complete without an SSL certificate. If you have a web service, Certbot is the fastest way:

apt install -y certbot python3-certbot-nginx
certbot --nginx -d example.com -d www.example.com

After installation, run systemctl list-timers | grep certbot to make sure the renewal timer is active. Let's Encrypt certificates are valid for ninety days, and if automatic renewal doesn't work, your site won't come up one morning with a security error. Installing a free certificate with Certbot also covers common renewal errors.

Don't forget backups. Hardening prevents many things, but it doesn't prevent accidentally deleting a file. A simple script with rsync or scp that sends a nightly backup to another destination is worth more than any firewall. Transferring files with scp and rsync shows the correct pattern.

What can be postponed

Complete hardening is a day's work. If you're in a hurry, follow this order: update, sudo user, SSH key, disable password, firewall. SELinux or AppArmor, advanced monitoring, and kernel-level hardening can wait until next week. But do these five on the first day.

If you've just gotten a server from a provider and don't want to do all this manually, a managed Linux host with initial secure settings is a lighter starting point; but if you want full control, the same raw server with the checklist above will do the job.

Frequently asked questions

Why can't I log in after changing the SSH port?

You probably didn't open the new port in the firewall, or you restarted the sshd service and the config had an error. Log in via the KVM console, run sshd -t to see the config error, and make sure ufw allow NEWPORT/tcp was run before the reload. As long as you keep a second terminal open, this problem is solved in a few minutes.

Is disabling root login enough?

No. If PasswordAuthentication yes remains, brute-force attacks on other users continue. Close both together and only leave key-based login open. Also set AllowUsers so that only the users you want are allowed to log in.

Should I use ufw and iptables together?

No. ufw is a management layer on top of iptables, and if you write raw iptables rules at the same time, your rules may be wiped after a reboot or reload. Pick one. If you have Docker, know that Docker writes its own rules separately and you must restrict ports at the container level itself.

After hardening, how do I know if someone tried to log in?

Use fail2ban-client status sshd to see the number of blocked entries, and run journalctl -u ssh --since "24 hours ago" | grep Failed. If the blocked count is high but none of them are your own IPs, the system is working correctly. If your own IP got blocked, unban it with fail2ban-client set sshd unbanip IP.