5 Effective Tips to Harden SSH Server on Ubuntu

In this post I’m going to share 5 tips you can use to secure SSH on a public-facing Ubuntu server. SSH stands for Secure SHell. It’s the primary way for administrators to login to remote Unix/Linux servers and also one of the primary attack vector by bad guys. It’s important that we secure it as much as possible.

Checking Your SSH Server Log

Before applying my tips, you can view your SSH log by issuing the following command. The -u flag specifies that we only want to see logs belonging to the ssh service unit.

sudo journalctl -u ssh

Press J to scroll down, K to scroll up. Press F to scroll down one full screen, B to scroll up one full screen. Press Q to quit. If you want to go straight to the end of the log, run

sudo journalctl -eu ssh

You will see that bad guys are constantly trying to gain access to your SSH server, as indicated by messages like below.

Apr 23 18:58:33 sshd[14505]: Failed password for root from 42.7.26.60 port 60148 ssh2
Apr 23 18:58:35 sshd[14505]: Failed password for root from 42.7.26.60 port 60148 ssh2
Apr 23 18:58:38 sshd[14507]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=58.218.198.170 user=root
Apr 23 18:58:38 sshd[14505]: Failed password for root from 42.7.26.60 port 60148 ssh2
Apr 23 18:58:40 sshd[14507]: Failed password for root from 58.218.198.170 port 63586 ssh2
Apr 23 18:58:41 sshd[14505]: Failed password for root from 42.7.26.60 port 60148 ssh2
Apr 23 18:58:42 sshd[14507]: Failed password for root from 58.218.198.170 port 63586 ssh2
Apr 23 18:58:44 sshd[14505]: Failed password for root from 42.7.26.60 port 60148 ssh2
Apr 23 18:58:45 sshd[14507]: Failed password for root from 58.218.198.170 port 63586 ssh2

Tip 1: Disable Root SSH Login

Bad guys need to know the username in order to login to your server. Every Linux system has a root user account so it’s a bad idea to allow root user to login via SSH. We can create another user that has the ability to SSH login and disable root SSH login. Thus, the attacker has to figure out the username before trying to brute force the password. You can create a user on Ubuntu by running the following command. Replace username with your preferred username. Avoid common usernames like admin.

sudo adduser username

You will be asked to set password for the new user. Next, add this user to the sudo group so this user can manage the server via sudo.

sudo adduser username sudo

Verify that this new user can login via SSH and is able to use sudo. Then edit the SSH daemon configuration file.

sudo nano /etc/ssh/sshd_config

Find the following line:

#PermitRootLogin yes

Remove the # symbol and change yes to no to disable root SSH login.

PermitRootLogin no

Save and close this file. Then restart SSH service for the changes to take effect.

sudo systemctl restart ssh

Most of the time, I don’t use the root account, so I like setting a long, complicated password for root that even I can’t remember. Because I don’t want to keep the default password that’s generated by my hosting provider. What if your account on the hosting provider is hacked, and the bad actor can use the default root password to access your server via VNC console?

I use the following command to generate a 32 character long random password.

openssl rand -base64 32

To change the root password, run

sudo passwd root

If I need to use the root account, I simply switch to root with the following command. I don’t have to provide the root password. I just need to type in the sudo password of my user account.

sudo su -

If an attacker tries to log in as a user with SSH disabled (such as root), you will see the following line in your SSH log.

Failed password for invalid user root

Here invalid means the user doesn’t exist on the system or has no SSH login permission. If a user who has SSH login permission typed a wrong password, then you will see a line like:

Failed password for username

Tip 2: Use Public Key Authentication and Disable Password Authentication

Passwords are a weak point in security and people make bad passwords. A better authentication method is public key authentication: you upload your public key to the server and only your private key can be used to login. Some people call it password-less login.

To enable public key authentication and disable password authentication, please see the following tutorial:

• 2 simple steps to set up password-less SSH login on Ubuntu

Tip 3: Enable Two-Factor Authentication (2FA)

Normally, you only need to enter a password or use SSH key to log in to your Ubuntu server remotely. Two factor authentication (2FA) requires you to provide two pieces of information in order to login, which can greatly increase the security of your OpenSSH server. These days many websites and services (Facebook, Google, Twitter, etc) allows user to set up 2FA to secure their accounts and it’s a good idea to also enable 2FA on your SSH server.

To enable Two-factor authentication for OpenSSH server on Ubuntu, check the following tutorial.

• How to Set Up SSH Two Factor Authentication on Ubuntu 18.04, 18.10 Server

Tip 4: Use Fail2ban to Block Repeat Offenders

Fail2ban is a set of server and client programs to limit brute force authentication attempts. Install it from default Ubuntu repository.

sudo apt install fail2ban

After it’s installed, it will be automatically started, as can be seen with:

sudo systemctl status fail2ban

The fail2ban-server program included in fail2ban monitors log files and issues ban/unban command. By default, it would ban a client’s IP address for 10 minutes if the client failed password 5 times. The ban is done by adding iptables firewall rules. You can check iptables rules by running the following command.

sudo iptables -L

In the screenshot below, the remote client 54.ip-37-187-225.eu is banned on my server. It will be refused to connect to port 22 for 10 minutes. (It sill can connect to other opened ports on the server.)

The fail2ban rules file is /etc/fail2ban/jail.conf. You don’t have to edit it as the default settings should suffice to prevent SSH brute force attack. If you want to modify the default settings, you should add your modifications in /etc/fail2ban/jail.local file.

For example, you can add the following lines in jail.local file.

[sshd]
enabled = true
maxretry = 3
bantime = 24h
ignoreip = 127.0.0.1/8 ::1/128 12.34.56.78

Where:

• maxretry: number of failures that have to occur in the last 10 minutes to ban the IP.
• bantime: effective ban duration. Note that the fail2ban version on Ubuntu 16.04 does not support the h (hour) time unit, so you need to use the default s (second) time unit.
• ignoreip: list of IPs not to ban

To set default values for all jails, put the parameter in [DEFAULT] instead of [sshd]. For example, you can put the ignoreip addresses to [DEFAULT].

[DEFAULT]
ignoreip = 127.0.0.1/8 ::1/128 12.34.56.78

You can now remove the ignoreip parameter from the [sshd] jail, so if you add more IP addresses to the whitelist, you can add them in [DEFAULT].  If there are 2 same parameter, then the one in [sshd] will override the other one in [DEFAULT].

Save and close the file. Then reload fail2ban for the changes to take effect.

sudo systemctl reload fail2ban

You can check the fail2ban log in /var/log/fail2ban.log file. For more info on the fail2ban jail configuration, see the man page.

man jail.conf

Tip 5: IP Address Whitelisting

Using fail2ban might not be enough to harden your OpenSSH server. What if fail2ban itself can be exploited to allow remote code execution? You can eliminate unnecessary attacks to your server with firewall rules, so bad actors can’t even connect your SSH port.

If only yourself need to log in to the server, why allow other IP address to log in? You can limit SSH login to your IP address and deny all other IP addresses. My home IP address is dynamic but I have my own VPN server running in a data center so that I have a static IP address when using VPN. You can even set up the VPN server on the same host, so you can log into the OpenSSH server from the IP address of that server.

The OpenSSH server can use TCP Wrappers to define whitelist and blacklist. To allow your IP address to login via SSH, edit the /etc/hosts.allow file.

sudo nano /etc/hosts.allow

Add allowed IP address like below. If the IP addresses are not in the same range, then you need to add a line for each host individually.

sshd:192.168.0.0/24
sshd:127.0.0.1
sshd:12.34.56.78

Save and close the file. Then edit the /etc/hosts.deny file. You can disallow all other hosts by adding the following line.

sshd:ALL

Save and close the file. You don’t need to reload any service. The changes take effect when the files are saved.

Note that it’s important you edit the /etc/hosts.allow file first, or you could be locked out of your server.

After some time, you can check the SSH logs.

sudo journalctl -eu ssh

You will see the messages like below, indicating TCP Wrappers has denied these IP addresses from connecting to the SSH daemon.

Dec 26 07:57:42 sshd[5962]: refused connect from 37.187.92.192 (37.187.92.192)
Dec 26 07:57:55 sshd[5979]: refused connect from 178.128.71.114 (178.128.71.114)
Dec 26 07:58:00 sshd[5985]: refused connect from 188.120.235.20 (188.120.235.20)
Dec 26 07:58:05 sshd[5991]: refused connect from 181.49.150.45 (181.49.150.45)

Using iptables Firewall to Restrict IP Address

Some Linux distributions like Arch Linux doesn’t include TCP wrappers by default. In that case, you can use the iptables firewall to create a whitelist of IP addresses that can connect port the SSH port. Firewall whitelisting makes it look as if the SSH port on your server is closed, which is what I prefer.

To prevent the current SSH connections drops out, we need to allow established sessions with the following iptables command.

sudo iptables -A INPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT

Then allow your own IP address to connect to the SSH port with the below command. Replace your-ip-address with your real IP address such as 74.125.128.103.

sudo iptables -A INPUT -p tcp --dport 22 -s your-ip-address -j ACCEPT

Where:

• -A INPUT is used to append the above rule to the INPUT chain which deals with incoming traffic.
• -p tcp specifies the protocol is TCP since SSH daemon listens on TCP port.
• --dport 22 specifies the destination port is 22 which is the default SSH port. If you changed your SSH port, then you also need to adjust the port here.
• -s your-ip-address specifies the source IP address.
• -j ACCEPT means jump to the ACCEPT target which will allow this SSH connection.

You can add multiple firewall rules to allow multiple IP addresses. After that’s done, run the following command to reject all other IP addresses to connect to your SSH server. We don’t specify the source IP address which means all other IP addresses will be disallowed.

sudo iptables -A INPUT -p tcp --dport 22 -j REJECT

Check iptables firewall rules with the below command.

sudo iptables -L

If later you want to add a new IP address to the whitelist, then you need to insert a new rule.

sudo iptables -I INPUT -p tcp --dport 22 -s your-ip-address -j ACCEPT

The -I option is used to insert the above firewall rule to the INPUT chain. By default, it will insert the above rule as the first rule in the INPUT chain. Note that if we append a new rule (-A) to the bottom of INPUT chain, then this rule has no effect because it comes after the “reject all other IP” rule.

If you use an iptables frontend like UFW, please see the following tutorial on how to create a whitelist.

• Getting Started with UFW Firewall on Debian, Ubuntu, Linux Mint Server

Checking Last Login IP Address

PrintLastLog is used to tell you when the last login happened and from what IP address. You can see it after you SSH into your server. It looks something like this:

Last login: Thu Jun  2 04:10:08 2016 from 12.34.56.78

If you don’t recognize the IP address, then you know something is not right. To enable PrintLastLog, edit the SSH daemon configuration file.

sudo nano /etc/ssh/sshd_config

And set PrintLastLog to yes.

PrintLastLog yes

Save the file and restart the SSH service.

What if You Are Locked Out?

Some folks think that you may accidentally be locked out by applying all the above tips.

However, if you can physically access the server, then you don’t need SSH to log in. Even if your server is hosted in the cloud, you can always use VNC console to log into your server. For example, I run my mail server on ScalaHosting. ScalaHosting provides a web-based VNC console in the account control panel. This VNC connection is not affected by SSH.

DigitalOcean provides a secure web-based VNC console too.

So does Vultr.

You can always log in via VNC when SSH is unavailable.

Next Step

I hope this tutorial helped you harden OpenSSH server on Ubuntu. This is one article in the security tutorial series. You may also want to check out other articles to harden your server.

• Set Up Automatic Security Update (Unattended Upgrades) on Ubuntu
• Canonical Livepatch Service: Patch Linux Kernel on Ubuntu without Reboot
• How to Use UFW Firewall on Debian, Ubuntu, Linux Mint

As always, if you found this post useful, then subscribe to our free newsletter to get more tips and tricks. Take care 🙂