Logging In with SSH Keys and Completely Disabling Passwords on the Server

You've opened the auth.log and you see login attempts on port 22 from three different continents. Or worse: an Accepted password for root sitting in the middle of a flood of Failed password lines. If you're still logging in with a password, the problem isn't "if" — it's "when." SSH key login reduces this attack surface to nearly zero, provided you do it correctly and open an escape route for yourself before disabling passwords.

Before Anything Else: Keep a Backup Session Open

Take this seriously. Open a second SSH terminal, log in, and don't close it until you're done. If you break sshd_config and your current session drops, you'll have no way back except through your hosting provider's KVM console. I've seen server admins lock themselves out after systemctl restart sshd and wait until morning for a ticket response.

Before making any changes, back up the config file:

cp /etc/ssh/sshd_config /etc/ssh/sshd_config.bak-$(date +%F)

Generating an SSH Key: Which Algorithm and Why

Run this on your own system, not on the server:

ssh-keygen -t ed25519 -a 100 -C "ali@laptop-1403"

The -a 100 option increases the number of KDF rounds to protect the private key's passphrase. If you have an older system that doesn't support ed25519, use -t rsa -b 4096. Don't generate RSA 2048 anymore; it hasn't been broken in practice yet, but its security margin has shrunk enough that it's not worth the risk.

I insist on a passphrase for the private key. A key without a passphrase means anyone who grabs the ~/.ssh/id_ed25519 file has exactly the same access you do. With ssh-agent, you enter it once a day and work comfortably for the rest of the day.

Copying the Public Key to the Server

The simplest way:

ssh-copy-id -i ~/.ssh/id_ed25519.pub ali@server-ip

If the server is on a non-default port, add -p 2222. If you don't have ssh-copy-id, do it manually:

cat ~/.ssh/id_ed25519.pub | ssh ali@server-ip "mkdir -p ~/.ssh && chmod 700 ~/.ssh && cat >> ~/.ssh/authorized_keys && chmod 600 ~/.ssh/authorized_keys"

Permissions are critical here. If ~/.ssh has 755 permissions or authorized_keys is group-writable, sshd will silently reject the key. Silently. Meaning you won't see anything in the log except Authentication refused: bad ownership or modes, which many people overlook.

Configuring sshd_config for SSH Key-Only Login

Now go to the main file. Set these lines in /etc/ssh/sshd_config:

PubkeyAuthentication yes
PasswordAuthentication no
PermitRootLogin prohibit-password
ChallengeResponseAuthentication no
UsePAM yes
AllowUsers ali deploy
MaxAuthTries 3
LoginGraceTime 20

A point many people get wrong: they confuse PermitRootLogin no with prohibit-password. The first completely blocks root login; the second only blocks root's password and allows the key. If you have a root key and want to use it, set prohibit-password. If not, set no and work from a regular user with sudo.

Don't underestimate AllowUsers. Without it, any system user with a valid shell can attempt login. With it, only the two names you specified are allowed.

Test the Configuration Before Restarting

sshd -t

If the output is empty, the configuration is valid. If it returns an error, fix it right there. Then:

systemctl reload sshd

Use reload, not restart. reload doesn't drop active sessions. Now, in that second terminal you kept open, open a new session. If you get in with the key, close the first terminal and move on to the next task.

This Is Where They Mess Up: Disabling Passwords Before Testing the Key

The recurring pattern is this: the admin sets PasswordAuthentication no, runs reload, their current session doesn't drop, they feel relieved and walk away. The next morning when they try to connect, they get Permission denied (publickey). The cause is usually one of three things: the key is in the wrong user's authorized_keys, file permissions are broken, or AllowUsers left out the username. In all three cases, the error message is identical and unhelpful.

How to diagnose: on the server, run journalctl -u ssh -f and connect from another system. The line userauth_pubkey: key type ssh-ed25519 not in PubkeyAcceptedAlgorithms or Authentication refused: bad ownership or modes tells you exactly what to fix.

Escape Route: If You Lock Yourself Out

The first thing you do is log in through your hosting provider's emergency console. If you don't have that either, use another user who still has access. If neither exists, you'll need to use rescue mode or emergency boot on the server and mount the disk.

Once you're back in, follow this order:

1. Temporarily set PasswordAuthentication yes and run reload.
2. Log in with the password and fix the key issue at its root.
3. Disable the password again and this time test from another system.

A practical tip: before disabling passwords, always create a second user with a separate key. If your primary key is lost or you forget the passphrase, that second user will save you. This takes five minutes and eliminates a sleepless night.

Next Layers: fail2ban and Port Restriction

Disabling passwords neutralizes brute-force attacks but still fills up logs and consumes bandwidth. Check out SSH security; from changing the port to fail2ban and public keys and set up a simple jail for sshd:

[sshd]
enabled = true
port = 22
maxretry = 3
bantime = 3600

Changing the port alone isn't security — it just reduces noise. Someone targeting your server will find the port. But if your logs are flooded with login attempts and you want to separate signal from noise, it's worth it.

If your server is behind heavy malicious traffic, make sure the network layer is covered before any configuration. DDoS protection from ServerNet covers this layer for cloud services and lets you apply SSH settings without downtime.

Where Not to Keep the Key

The private key doesn't stay on the server. Not in /root/.ssh, not in the project folder, not in a Git repository. If you connect from server to server for deployment, keep the key on the origin and use ssh-agent forwarding. If you have a team, read Password management in teams; why sharing in chat is forbidden? — the same logic applies to keys.

To know who logged in, when, and from where, keep your logs intact. What are audit logs and how do we keep them intact? opens up exactly this topic and is critical for answering the question "who did this?"

Final Checklist Before Disabling Passwords

If every row is green, set PasswordAuthentication no, run sshd -t, reload, and test from another system. If the test succeeds, close the backup terminal and move on to fail2ban.

Frequently Asked Questions

Why do I still get Permission denied (publickey) after disabling passwords?

In ninety percent of cases, the problem is file permissions, not the key itself. If ~/.ssh doesn't have 700 permissions or authorized_keys is writable by others, sshd silently rejects the key. Use journalctl -u ssh -f while attempting login to see the exact message.

The second scenario is that the key is in the wrong user's authorized_keys. If you connect with ssh ali@server, the key must be in /home/ali/.ssh/authorized_keys, not in /root/.ssh.

Does changing the SSH port alone improve security?

No. Changing the port only reduces the volume of automated scans and keeps logs readable. Someone targeting your server will find the new port with a simple scan. Real security comes from public keys, disabling passwords, and restricting allowed users.

If you changed the port, remember to set the same port in the firewall and in fail2ban, otherwise you'll lose access after a reboot.

Where should I keep the private key so it stays secure?

On your own system, in ~/.ssh with 600 permissions, and always with a passphrase. Take an encrypted backup of the private key and keep it separate from your laptop. The private key never stays on the destination server and never goes into a Git repository.

If the key is compromised, just removing it from authorized_keys isn't enough; you must generate a new key and remove the old one from every server it had access to.

How do I know if someone logged into the server with my key?

In /var/log/auth.log or the output of journalctl -u ssh, look for the line Accepted publickey for. Each line includes the username, key fingerprint, and source IP. If you see an unknown IP, remove the key from authorized_keys immediately.

For these logs to remain verifiable later, they must be kept intact and stored separately from the server. If you lose the server, the logs on that same server are lost too.