How to Harden an SSH Server Against Unauthorized Access

Every server I have ever inherited had one thing in common: SSH was the first thing someone tried to break into. It does not matter if the box is a tiny VPS running a side project or a production node behind a load balancer — the moment it has a public IP, automated scanners start knocking on port 22 within minutes. I have watched auth logs fill up with thousands of failed login attempts against

root

and

admin

in a single afternoon on a freshly provisioned host. Hardening SSH is not optional busywork, it is the baseline for keeping a server yours.

This runbook walks through the process I use on every new Linux server, from a bare cloud image to a locked-down SSH configuration that I would actually trust in production. I will use sw-infrarunbook-01 as the example hostname and infrarunbook-admin as the operator account throughout.

Prerequisites

Before touching

sshd_config

, make sure you have a few things in place. Getting these wrong is how people lock themselves out of a remote server with no console access, which is a genuinely bad afternoon.

• Root or sudo access to the server, ideally through a provider console (out-of-band access) in case SSH breaks mid-change.
• A non-root user account already created and able to sudo — in my examples this is

  infrarunbook-admin

  .
• A local SSH key pair generated on your workstation, or the ability to generate one.
• Knowledge of your own current public IP or IP range, if you plan to restrict access by source address.
• A second open terminal session to the server before you restart the SSH daemon. This is the single most important habit in this entire guide.

I cannot stress that last point enough. Keep an active session open while you test changes in a new one. If something goes wrong, your existing session is still alive and you can revert.

Step-by-step setup

1. Create a dedicated admin user

Never do daily work as root over SSH. Create a named account and give it sudo rights instead.

adduser infrarunbook-admin
usermod -aG sudo infrarunbook-admin

2. Generate and install an SSH key pair

On your workstation, generate an Ed25519 key. It is smaller and faster than RSA, and modern OpenSSH versions handle it natively.

ssh-keygen -t ed25519 -C "infrarunbook-admin@sw-infrarunbook-01"

Copy the public key to the server for the admin user:

ssh-copy-id -i ~/.ssh/id_ed25519.pub infrarunbook-admin@10.20.30.15

If

ssh-copy-id

is not available, append the public key manually into

/home/infrarunbook-admin/.ssh/authorized_keys

and set permissions to

700

on the

.ssh

directory and

600

on the file. Wrong permissions here are a classic reason key auth silently fails.

3. Test key-based login before changing anything else

Open a fresh terminal and confirm you can log in with the key, without a password prompt:

ssh -i ~/.ssh/id_ed25519 infrarunbook-admin@10.20.30.15

Do not proceed until this works cleanly. I have seen people disable password auth before verifying key login, and the result is always the same: a support ticket asking for a console reset.

4. Edit sshd_config

Open

/etc/ssh/sshd_config

and make the following changes one at a time, testing as you go where possible.

Disable root login entirely. There is no legitimate reason to SSH directly as root on a server with a working sudo user.

PermitRootLogin no

Disable password authentication. Once your key login is confirmed, this is the single biggest reduction in attack surface you can make. It kills brute force attempts outright, because there is no password to guess.

PasswordAuthentication no
KbdInteractiveAuthentication no

Restrict which users can even attempt to connect.

AllowUsers infrarunbook-admin

Consider moving off port 22. This will not stop a targeted attacker, but it does cut down the noise from mass internet scanners by an enormous margin. I have seen auth log volume drop by over 95% on a box just from this one change. Pick something above 1024 and out of common conflict ranges.

Port 2222

Limit authentication attempts and login grace time to reduce the window an attacker has per connection:

MaxAuthTries 3
LoginGraceTime 20
MaxSessions 4

Disable empty passwords and X11 forwarding unless you specifically need the latter:

PermitEmptyPasswords no
X11Forwarding no

5. Restart, do not reload, and test from the second session

sshd -t
systemctl restart sshd

Run

sshd -t

first — it validates the config syntax and will tell you immediately if you have a typo that would otherwise kill the daemon on restart. From your second, still-open session, open a third connection attempt using the new port and key:

ssh -i ~/.ssh/id_ed25519 -p 2222 infrarunbook-admin@10.20.30.15

Only close your original root/console session once this succeeds.

6. Add fail2ban for defense in depth

Even with password auth disabled, fail2ban is worth running. It catches connection floods, protects against future misconfigurations, and gives you visibility into who is probing the server.

apt install fail2ban -y
cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local

Edit

/etc/fail2ban/jail.local

and set an SSH-specific stanza matching your new port:

[sshd]
enabled = true
port = 2222
filter = sshd
logpath = /var/log/auth.log
maxretry = 3
bantime = 3600
findtime = 600

systemctl enable fail2ban
systemctl restart fail2ban

7. Scope access at the firewall too

SSH hardening should not live in

sshd_config

alone. If you know the source ranges that legitimately need access — an office network, a VPN concentrator, a bastion host — restrict the port at the firewall level as well.

ufw allow from 10.20.30.0/24 to any port 2222 proto tcp
ufw deny 2222/tcp
ufw enable

This gives you two independent layers. If one gets misconfigured, the other still holds.

Full configuration example

Here is a complete

/etc/ssh/sshd_config

reflecting everything above, as I would leave it on sw-infrarunbook-01 after a hardening pass:

Port 2222
Protocol 2
AddressFamily inet

HostKey /etc/ssh/ssh_host_ed25519_key
HostKey /etc/ssh/ssh_host_rsa_key

PermitRootLogin no
PasswordAuthentication no
KbdInteractiveAuthentication no
PermitEmptyPasswords no
ChallengeResponseAuthentication no
UsePAM yes

PubkeyAuthentication yes
AuthorizedKeysFile .ssh/authorized_keys

AllowUsers infrarunbook-admin
MaxAuthTries 3
LoginGraceTime 20
MaxSessions 4
ClientAliveInterval 300
ClientAliveCountMax 2

X11Forwarding no
AllowTcpForwarding no
PermitTunnel no
PrintMotd no

LogLevel VERBOSE
SyslogFacility AUTH

ClientAliveInterval

combined with

ClientAliveCountMax

drops dead or hung sessions after ten minutes of no response, which matters more than people expect on servers with flaky client-side networks — stale sessions pile up and make it harder to see who is actually connected.

LogLevel VERBOSE

logs the key fingerprint used for every login, which has saved me more than once when auditing which key was used for a specific access event.

Verification steps

Do not consider the job done until you have actually verified each control, not just written it into the config file.

• Confirm root login is refused:

  ssh -p 2222 root@10.20.30.15

  should be rejected outright.
• Confirm password auth is dead: temporarily remove your key from the agent (

  ssh-add -D

  ) and attempt a connection — it should fail immediately rather than prompting for a password.
• Check that only the allowed user can connect: try a login as a different local user and confirm it is denied even with a valid key.
• Verify fail2ban is watching the right port and log file:

  fail2ban-client status sshd

  should show the jail active with the correct filter.
• Tail the auth log during a deliberate failed attempt to confirm fail2ban actually bans after the configured threshold:

  tail -f /var/log/auth.log

  alongside

  fail2ban-client status sshd

  .
• Run a port scan against the old port 22 from an external host to confirm nothing is still listening there:

  nmap -p 22,2222 10.20.30.15

  .
• Review

  sshd -T

  output, which dumps the fully resolved effective configuration — useful for catching a directive that got silently overridden by a later block in the file.

Common mistakes

I have made or seen almost every one of these at some point, usually under time pressure.

Disabling password auth before confirming key login works. This is the number one way people lock themselves out of a remote box entirely. Always test the new key in a second session before you touch

PasswordAuthentication

.

Forgetting the firewall rule when changing ports. Moving SSH to 2222 in

sshd_config

does nothing if the firewall only allows inbound traffic on 22. You will restart

sshd

, lose your connection, and have no way back in except through the console.

Wrong permissions on the .ssh directory. OpenSSH is strict about this — if

~/.ssh

is not

700

or

authorized_keys

is not

600

, key auth fails silently with a generic permission denied message that gives you no hint why.

Restarting sshd without validating syntax first. A single typo in

sshd_config

can prevent the daemon from starting at all. Always run

sshd -t

before

systemctl restart sshd

, not after something breaks.

Assuming a non-standard port is a substitute for real hardening. Port obscurity reduces noise from automated scanners, it does not stop a targeted attacker who is actually looking at your infrastructure. Treat it as one layer among several, never the only one.

Leaving AllowUsers or AllowGroups out entirely. Without it, any local account with a valid key or password (if still enabled anywhere) can attempt SSH login, including service accounts that were never meant to be reachable remotely.

Not rotating or auditing keys over time. I have inherited servers with authorized_keys files going back years, full of keys from contractors who left the company long ago. Treat SSH keys like credentials, because that is exactly what they are — review and prune them periodically.

The goal of this whole process is not to make SSH impossible to use, it is to make it boring. A boring, predictable, well-logged SSH surface is exactly what you want on anything facing the internet.