Linux Server Hardening Checklist; In Order of Impact

You've received the server, you enter the IP, and you log in with root. So far nothing bad has happened. The bad thing starts when you open journalctl -u ssh and see that in the past 24 hours, 4,300 failed login attempts from 190 different IPs have been recorded. You'll see this number on any server whose port 22 is open to the internet; not because you're a specific target, but because scanners crawl the entire IPv4 space. Server hardening means deciding right here what to close first, because the order of the work determines how many hours you'll spend and how many times you'll lose the server.

The Correct Order of Server Hardening: From Access to Monitoring

Most checklists you find have 30 items and none of them say which item comes first. I settled on the following order after locking myself out of my own server several times. The logic is simple: first secure access, then reduce the attack surface, then understand what's happening.

1. Create a non-root user with sudo and disable direct root login.
2. Replace password login with public key authentication.
3. Bring up the firewall and leave only the necessary ports open.
4. Enable automatic security updates.
5. Put fail2ban or its equivalent on SSH.
6. Set up log monitoring and alerting.

If you shuffle this order, you'll usually lock yourself out at step three. I'll explain this in detail below.

Non-Root User and Disabling root Login

First, create a user and add them to the sudo group:

adduser deploy
usermod -aG sudo deploy
rsync --archive --chown=deploy:deploy ~/.ssh /home/deploy

Take that rsync line seriously. If you don't copy the public key to the new user's home directory and then set PermitRootLogin no, you're effectively locked out until someone has access to the server console panel from outside. I've seen this mistake made by someone on a cloud server without KVM, and they had to open a ticket and wait.

Then in /etc/ssh/sshd_config, configure these lines:

PermitRootLogin no
PasswordAuthentication no
PubkeyAuthentication yes
MaxAuthTries 3
AllowUsers deploy

Before systemctl restart sshd, leave another terminal open and log in with the new user. If that terminal works, close the current session. If it doesn't work, don't close the current session. This one habit has saved you more than any backup. More details about keys and fail2ban are in SSH Security; From Changing the Port to fail2ban and Public Keys.

Firewall: Which Ports Are Actually Necessary

On Ubuntu 22.04 and above, ufw is enough. On servers with heavy traffic or NAT and forwarding, direct nftables is better because it has no extra abstraction layer and you can see the rules more precisely.

ufw default deny incoming
ufw default allow outgoing
ufw allow 22/tcp
ufw allow 80,443/tcp
ufw enable

A point that trips up many people: if you've changed the SSH port, don't remove the rule for 22 until you're sure the new port works. If you get the order of ufw enable and changing the port wrong, the result is a completely unreachable server.

A firewall doesn't stop volumetric attacks. If your server is targeted by a flood of traffic, ufw rules are useless because packets need to be filtered somewhere before reaching the server. For that layer, you need DDoS protection that works at the network level, not on the machine itself.

Automatic Security Updates

On Debian and Ubuntu:

apt install unattended-upgrades
dpkg-reconfigure --priority=low unattended-upgrades

In /etc/apt/apt.conf.d/50unattended-upgrades, make sure the security repository is enabled and set Automatic-Reboot to false unless you have a specific restart window. The kernel isn't patched without a restart, and this is where people make a mistake: they think that because apt upgrade finished without errors, the server is patched. Then uname -r shows the same old version and the vulnerability is still there.

fail2ban and Monitoring

With MaxAuthTries 3, every IP is rejected after three attempts, but it can still try infinitely many times. fail2ban blocks this at the firewall level:

apt install fail2ban
fail2ban-client status sshd

The correct output looks something like this: Currently banned: 47. If the banned count stays at zero, it means the jail is configured on the wrong log; on systemd systems you should set backend = systemd, not the path /var/log/auth.log which may not exist.

Monitoring without alerting is useless. A daily logwatch or a simple script that counts successful logins is enough. If you want to know what should be logged and how to keep the log intact, read What Are Audit Logs and How Do We Keep Them Intact?.

Things That Have Less Impact in Server Hardening

Changing the SSH port from 22 to, say, 22222. This stops lazy scanners and makes the log cleaner, but it doesn't stop any determined scanner. If you have limited time, first do the public key and PermitRootLogin no, then change the port. Doing it the other way around is a waste of time.

Installing an antivirus on a web server also usually has little effect. The real attack surface on a Linux server is the running services and their versions, not the files on disk. Use ss -tulpn to see what's listening and close any service you don't recognize. This takes five minutes and has more effect than several security tools.

A Limitation You Should Know About

None of this work prevents vulnerabilities in the application itself. If you have WordPress and a plugin version isn't patched, hardened SSH doesn't help at all because the attacker enters through the web, not through the shell. For that path, WordPress Security; From wp-config to Plugins That Are Themselves Holes is the right starting point. Server hardening and application security are two separate layers, and neither replaces the other.

When You Lose the Server, What Shows You Where You Went Wrong

Three common signs:

• Connection refused on the SSH port: the firewall or the sshd service isn't up. From the server console, run systemctl status sshd.
• Permission denied (publickey) with a key that previously worked: the permissions on ~/.ssh are 777 or 775. It should be 700 for the directory and 600 for authorized_keys. sshd rejects this by default and doesn't give a clear message either.
• Successful login but sudo doesn't work: the user isn't in the sudo group, or you need to log in again for the group membership to take effect.

If your server is behind a corporate network and employees connect from outside, SSH hardening isn't enough; you also need to control the access path. The Comprehensive Guide to Enterprise VPN for Secure Employee Access covers this same layer.

To check whether your DNS records and network path are correct, use the DNS and network lookup tools before any major change. If the site isn't accessible from outside after a firewall change, you first need to know whether the problem is DNS or packet filtering.

If you'd prefer not to maintain this layer yourself, ServerNet Security Services covers these same tasks as a service; but even in that case, you should know what's configured on your server, because the day something goes wrong, that's the first question.

Frequently Asked Questions

Does changing the SSH port really increase security?

To some extent, but not as much as people imagine. Changing the port reduces log volume and the number of automated attempts, but it doesn't stop scanners that crawl all ports. The real effect comes from the public key and disabling root login. If you can only do one thing, choose PasswordAuthentication no.

Why can't I connect to the server after enabling the firewall?

Because you didn't add the SSH port rule before ufw enable, or you changed the port and removed the old rule. The solution is from the server console, not from SSH. Always keep a session open before enabling the firewall and test in a second session.

Can automatic updates break the server?

Yes, and the main reason is automatic restart. If Automatic-Reboot is set to true, the server comes up at an inconvenient time and dependent services may not start properly. Set it to false and do the restart in your own maintenance window.

What's the minimum I should do today?

Create a non-root user with a public key, set PermitRootLogin no and PasswordAuthentication no, and bring up ufw with only the necessary ports. These four tasks take less than half an hour and eliminate most automated login attempts. The rest of the checklist only makes sense after these.