Implementing Two-Factor Authentication for Server SSH Access

I have lost count of how many post-incident reviews I have sat in where the root cause boiled down to a single leaked SSH key. A contractor's laptop got compromised, a key sat unencrypted in a CI pipeline, or someone committed a private key to a repo by accident. SSH keys are good. They are not enough on their own, especially for anything that touches production. Adding a second factor on top of key auth means that a stolen key by itself is worthless to an attacker. This guide walks through setting up real two-factor authentication for SSH using TOTP (time-based one-time passwords) via PAM, on a server we will call sw-infrarunbook-01, running Ubuntu/Debian-style package management. The same approach works on RHEL-based systems with minor package name changes.

Before we get into it, a word of caution from experience: SSH lockouts from a botched 2FA rollout are one of the most common ways I have seen people get themselves stuck outside a remote box with no console access. Read the whole article before touching sshd_config on anything you cannot physically walk over to.

Prerequisites

You will need root or sudo access on the target server, a working SSH key pair already in place (we are layering 2FA on top of key auth, not replacing it), and a smartphone or desktop app capable of generating TOTP codes — Google Authenticator, Authy, or FreeOTP all work fine since the underlying protocol is an open standard, not something Google owns exclusively despite the package name.

You also need out-of-band access to the server. This could be a cloud provider's web console, IPMI/iDRAC, or physical access. If your only path to the server is SSH and you make a mistake in sshd_config, you can lock yourself out entirely. I always keep a second SSH session open to the server while testing changes, and I never close that session until I have verified the new config works from a completely fresh connection.

Finally, confirm which user accounts need 2FA. In my experience it is a mistake to roll this out for every single account on day one. Start with the admin accounts that have sudo rights, prove it works, then expand.

Step-by-step setup

First, install the PAM module that generates and verifies TOTP codes:

sudo apt update
sudo apt install libpam-google-authenticator

On RHEL/CentOS/Rocky systems, the package is usually called google-authenticator and may require the EPEL repository enabled first.

Next, log in as the account you want to protect — in our case, infrarunbook-admin — and run the setup wizard:

ssh infrarunbook-admin@sw-infrarunbook-01
google-authenticator

It will ask a series of questions. Answer yes to time-based tokens. It then prints a QR code and a secret key in the terminal. Scan the QR code with your authenticator app right away, before doing anything else, because the secret only displays once (well, it is saved to a file, but do not rely on scrolling back through terminal history for something this sensitive).

The remaining prompts matter more than people give them credit for:

• Update the .google_authenticator file — say yes, this is where the secret and scratch codes live.
• Disallow multiple uses of the same token — say yes, this prevents replay attacks within the same 30-second window.
• Increase the time window slightly if your server clock drifts — I usually leave this at the default unless I know NTP is unreliable on that host.
• Enable rate limiting — say yes, this stops brute-force attempts against the OTP itself.

Write down the emergency scratch codes it gives you. Store them somewhere other than the server itself — a password manager, not a text file in the home directory. If the server is ever compromised and the attacker finds scratch codes sitting next to the very account they protect, the whole exercise was pointless.

Now edit the PAM configuration for SSH. Open /etc/pam.d/sshd and add the following line, typically near the top, after any @include common-auth line or in place of it depending on your distro's defaults:

auth required pam_google_authenticator.so

If you want SSH key login to be exempt from the OTP prompt for automated processes (deployment scripts, for example) while still requiring it for interactive human logins, you will need nullok handling and careful separation of accounts. I generally avoid mixing automation accounts and human accounts under the same 2FA policy — keep service accounts on key-only auth restricted by IP, and human accounts on key-plus-OTP. Trying to make one policy fit both usually ends up weakening the automation account's security instead of strengthening the human one.

Now edit /etc/ssh/sshd_config. This is the step where most mistakes happen, so go slowly.

ChallengeResponseAuthentication yes
UsePAM yes
AuthenticationMethods publickey,keyboard-interactive

That last line is the one that actually enforces the two-factor requirement. It tells sshd that a successful login requires both a valid public key and a successful keyboard-interactive (PAM/OTP) exchange. Without this line, PAM will still prompt for the code, but a bare password or a bare key alone might still succeed depending on your other settings, which defeats the entire point.

On newer OpenSSH versions, ChallengeResponseAuthentication has been renamed to KbdInteractiveAuthentication. Check your version with sshd -V and set the one that applies. Setting both does no harm on most modern distros since the older directive is usually kept as an alias.

Restart sshd, but do not close your existing session:

sudo systemctl restart sshd

Full configuration example

Here is a complete, working /etc/ssh/sshd_config snippet reflecting a 2FA-enforced setup on sw-infrarunbook-01, alongside the matching PAM entry.

# /etc/ssh/sshd_config
Port 22
ListenAddress 192.168.10.15
PermitRootLogin no
PasswordAuthentication no
KbdInteractiveAuthentication yes
ChallengeResponseAuthentication yes
UsePAM yes
AuthenticationMethods publickey,keyboard-interactive
AllowUsers infrarunbook-admin
MaxAuthTries 4
LoginGraceTime 30

# /etc/pam.d/sshd
# Standard Unix authentication removed in favor of key + OTP
#@include common-auth
auth required pam_google_authenticator.so nullok
account required pam_unix.so
session required pam_unix.so
session optional pam_systemd.so

Note the nullok flag in the PAM line above. During rollout, this lets accounts without a configured .google_authenticator file log in with just their key, so you are not locking out every account the moment you restart sshd. Once every relevant account has run through the setup wizard, remove nullok so that 2FA becomes mandatory rather than optional.

And the per-user secret file, generated automatically by the wizard, lives at:

/home/infrarunbook-admin/.google_authenticator

Permissions on that file should be 600, owned by the user. The setup script handles this correctly by default; do not go changing it manually.

Verification steps

Open a brand new terminal — do not reuse the session you have kept open as a safety net — and attempt to connect:

ssh infrarunbook-admin@sw-infrarunbook-01

You should see your normal key-based authentication happen silently in the background, followed by a prompt like:

Verification code:

Enter the six-digit code from your authenticator app. If it succeeds, you are in. If it fails, do not panic and do not touch the server yet — go back to your still-open safety session first.

Check the auth log for what actually happened:

sudo tail -n 50 /var/log/auth.log

On RHEL-based systems this is typically /var/log/secure instead. Look for lines referencing pam_google_authenticator — they will tell you directly whether the code was rejected due to clock drift, an incorrect secret, or a rate limit trigger.

Also worth testing: confirm that key-only auth, without the OTP step, is actually rejected now. Temporarily try connecting with a tool or flag that skips keyboard-interactive (some SSH clients or automation frameworks default this way) and confirm the connection is refused. If it is not refused, your AuthenticationMethods line is not being applied, usually because UsePAM is set to no somewhere, or because a Match block further down the config is overriding it for that connection.

Finally, verify server clock accuracy, since TOTP is time-based and drift beyond about 30-90 seconds (depending on your window setting) will cause valid codes to fail intermittently in a way that looks like a bug but is actually just NTP:

timedatectl status

Common mistakes

The single most common mistake I see is closing the only open SSH session before verifying the new config in a fresh one. If sshd_config has a typo or the PAM stack is misconfigured, sshd may still be running fine for the session you already have open, while refusing every new connection attempt. That safety session is your only way back in without console access. Keep it open until verification is fully done.

Second mistake: forgetting that AuthenticationMethods can be overridden by a Match block later in the file. If you have a Match Address or Match User block further down sshd_config, sshd applies whichever directives appear last for a matching connection. I have seen a perfectly good top-level 2FA policy get silently bypassed because of a leftover Match block for an old jump host IP range that nobody remembered was there.

Third: rolling out pam_google_authenticator.so without nullok on the first pass, across every account at once. Any account that has not yet run the setup wizard gets locked out immediately on the next login attempt, because PAM has no secret file to check against. Roll out with nullok, confirm every account is configured, then tighten it.

Fourth mistake, and a subtle one: assuming PermitRootLogin no means root is protected from OTP concerns. It just means root cannot log in over SSH at all, which is good practice anyway, but do not assume it substitutes for actually securing the accounts that do have sudo rights.

Fifth: not accounting for scratch codes properly. If someone loses their phone, and their scratch codes are sitting in a plaintext file on the same server they protect, you have not gained the security you think you have. Scratch codes belong in a password manager or a physically separate secure location, never alongside the server they unlock.

Last one, and it is more of an operational gap than a technical mistake: not documenting the recovery procedure. When (not if) someone gets a new phone or loses access to their authenticator app, you need a documented, auditable process for regenerating their secret that does not involve someone on the ops team just disabling 2FA for that account "temporarily" and forgetting to re-enable it. I have seen that exact scenario stay in place for over a year before someone else noticed during a security audit.

Two-factor authentication on SSH is not complicated to set up technically. The actual difficulty is discipline: keeping a safety session open, testing before enforcing, and treating scratch codes and secrets with the same care as the keys they are meant to complement. Get those habits right and the PAM configuration itself takes about fifteen minutes per server.