TL;DR: four lines in sshd_config and a reload. Every Ubuntu server should have at least this.

# /etc/ssh/sshd_config
PermitRootLogin no
PasswordAuthentication no
PubkeyAuthentication yes
MaxAuthTries 3

sudo systemctl reload ssh

Do key-based login first, then flip PasswordAuthentication no. Otherwise you'll lock yourself out. Keep a second SSH session open while you test. Ask me how I know.

The checklist

  1. Create a non-root user, add SSH key
  2. Disable root login
  3. Disable password auth (key-only)
  4. Tighten auth attempts and idle timeout
  5. Restrict who can log in (AllowUsers)
  6. Change the port (optional, mostly cosmetic)
  7. Install fail2ban
  8. 2FA (optional, for jump hosts and anything internet-facing that matters)

Work top to bottom. Test after every step.

1. Key-based auth

On your workstation:

ssh-keygen -t ed25519 -C "lukas@laptop"
ssh-copy-id lukas@server

ed25519 keys are small, fast, and modern. Don't generate new RSA keys in 2026. If ssh-copy-id isn't installed, append the public key manually to ~/.ssh/authorized_keys on the server — 600 authorized_keys, 700 ~/.ssh, owned by the user.

Test the login:

ssh -i ~/.ssh/id_ed25519 lukas@server

If that works without asking for a password, your key is in place. Now you can disable passwords safely.

If you don't have a non-root user yet, start here: Linux user management: adduser, sudo, SSH keys.

2. Disable root login

No SSH user should be root. Ever. Not even for "just a minute".

sudo sed -i 's/^#*PermitRootLogin.*/PermitRootLogin no/' /etc/ssh/sshd_config
sudo sshd -t
sudo systemctl reload ssh

sshd -t validates the config before you reload. If you skip that step and there's a typo, reload fails and your next login fails too. Always run sshd -t.

3. Disable password auth

Only after your key works.

sudo sed -i 's/^#*PasswordAuthentication.*/PasswordAuthentication no/' /etc/ssh/sshd_config
sudo sed -i 's/^#*KbdInteractiveAuthentication.*/KbdInteractiveAuthentication no/' /etc/ssh/sshd_config
sudo sshd -t && sudo systemctl reload ssh

On Ubuntu 22.04+ there's often an override in /etc/ssh/sshd_config.d/50-cloud-init.conf that re-enables password auth. Check it:

sudo grep -rE '^PasswordAuthentication' /etc/ssh/

If the last matching line says yes, passwords are still on no matter what you did in the main file. Fix or delete the override.

Verify effective config:

sudo sshd -T | grep -iE 'permitrootlogin|passwordauthentication'

sshd -T prints what the daemon actually applies, including every override. This is the only output you should trust.

4. Auth attempts and idle timeout

# /etc/ssh/sshd_config
MaxAuthTries 3
LoginGraceTime 30
ClientAliveInterval 300
ClientAliveCountMax 2

MaxAuthTries 3 disconnects after three bad attempts per connection. LoginGraceTime 30 kills stalled logins that never complete. ClientAliveInterval + CountMax drops dead sessions after 10 minutes of silence so they don't sit there forever tying up pty slots.

5. AllowUsers

Whitelist who can SSH. Single line:

AllowUsers lukas deploy

Everyone else is locked out regardless of keys. This is cheap and effective — if someone creates a user (or worse, a distro package does) you don't suddenly have a new SSH-reachable account.

For groups, AllowGroups ssh-users works the same way and scales better.

6. Port change (if you want)

This is the security-theater step. Moving off port 22 cuts scan-log noise by ~95% but doesn't stop a targeted attacker — nmap finds any port in seconds. I still do it, because quieter logs are easier to read.

# /etc/ssh/sshd_config
Port 2222

Don't forget UFW:

sudo ufw allow 2222/tcp
sudo ufw delete allow 22/tcp   # only after you confirm the new port works

UFW basics if you need them: UFW firewall rules on Ubuntu.

Reload SSH, open a second session on the new port, and only then close the old one.

7. fail2ban

SSH brute-force attempts are constant. fail2ban reads auth logs and bans IPs that fail too many times. Five minutes of setup, forever of quiet logs.

I wrote the full walkthrough here: fail2ban setup for SSH and nginx. Do it now.

8. 2FA (optional)

Google Authenticator PAM module gives you TOTP on SSH login. Worth it for jump hosts, public-internet bastions, anything where a stolen key is game over.

sudo apt install libpam-google-authenticator
google-authenticator

Follow the prompts, scan the QR with Authy / Google Authenticator / whatever. Then:

# /etc/pam.d/sshd (append)
auth required pam_google_authenticator.so nullok

# /etc/ssh/sshd_config
ChallengeResponseAuthentication yes
KbdInteractiveAuthentication yes
AuthenticationMethods publickey,keyboard-interactive

AuthenticationMethods makes SSH require both the key and the TOTP. nullok in PAM means users who haven't set up TOTP can still log in with just the key — flip it off once everyone's enrolled.

Verify

From another machine:

ssh -v lukas@server

Read the output. You should see Authenticated to server using "publickey". If you see Offering RSA public key then Password authentication, something's off.

On the server:

sudo sshd -T | grep -iE 'permitrootlogin|passwordauthentication|allowusers|maxauthtries|port'

This is the authoritative view of what the daemon is running with.

Don't lock yourself out

Rule I follow every time I touch sshd_config:

  1. Keep the current SSH session open
  2. Make changes
  3. sudo sshd -t — config syntax check
  4. sudo systemctl reload ssh (not restart: reload keeps existing sessions)
  5. Open a new SSH session from another terminal
  6. Only close the original session once the new one works

If you lock yourself out, your only recovery is out-of-band access: hoster's serial console, IPMI, rescue system, physical keyboard. Don't skip step 1.

Gotchas

Config overrides in sshd_config.d

On Ubuntu, anything under /etc/ssh/sshd_config.d/*.conf is included. 50-cloud-init.conf in particular loves to enable PasswordAuthentication yes. Edit that file, don't just edit sshd_config.

Wrong permissions on ~/.ssh

SSH silently refuses keys if permissions are too loose. ~/.ssh must be 700, authorized_keys must be 600, and both must be owned by the user. Check with stat or ls -la ~/.ssh.

Reload vs restart

systemctl restart ssh kills existing sessions on some Ubuntu versions. reload doesn't. Always reload unless you're doing something that actually requires a full restart.

SELinux / AppArmor on weird port

On some distros a non-standard SSH port gets blocked by MAC until you tell the policy it's allowed. Ubuntu's default AppArmor profile is permissive enough not to care — if you're on RHEL-ish systems, run semanage port -a -t ssh_port_t -p tcp 2222.

Baseline for every new server

I do SSH hardening as part of the first-hour setup, right after the initial checklist. It takes five minutes and it's the highest-leverage security work you can do on a Linux box. Don't leave it for "later". Later is how boxes end up in the mining-pool stats.

Related posts