How to Secure Email Server Against Hacking with VPN (Debian/Ubuntu)

In this tutorial, I’m going to share with you my tips and tricks to secure email servers against hacking with a self-hosted VPN server. Many spammers are trying to hack into other people’s email servers. If successful, they would use the hacked email server to send large volumes of spam or steal valuable data. Why do we use a self-hosted VPN server? Because it allows you to enable whitelisting, so only trusted users connected to the VPN server can access your mail server.

Prerequisites

It’s assumed that you have an email server up and running. If not, follow one of the tutorials below to set up your own mail server.

• How to Easily Set up a Full-Fledged Mail Server on Ubuntu 20.04 with iRedMail
• How to Easily Set Up a Mail Server on Debian 10 Buster with iRedMail

It’s also assumed that you have set up a VPN server. If not, please follow one of the tutorials below. The mail server and VPN server can run on separate hosts, or on the same host.

• Set Up Your Own WireGuard VPN Server on Ubuntu
• Set Up Your Own WireGuard VPN Server on Debian

Hint: It’s recommended to run VPN server and mail server on separate hosts to reduce operational complexities. If the mail server and VPN server run on the same host, then there are additional steps required, namely setting up a response policy zone on the VPN server to override the DNS A record of your mail server hostname.

• Set Up Response Policy Zone (RPZ) in BIND Resolver on Debian/Ubuntu

Let’s say the DNS A record for mail.yourdomain.com resolves to 12.34.56.78, then you need to create a record in the response policy zone to resolve it to the VPN server’s private IP address 10.10.10.1.

In the following texts, I use 12.34.56.78 as the public IP address of the VPN server. If the VPN server and mail server run on the same host, then you need to replace 12.34.56.78 with the VPN private IP range 10.10.10.0/24.

Step 1: Add VPN Server IP Address to the Firewall Whitelist

Once you have a mail server and VPN server up and running, you should add the VPN server’s IP address to the whitelist of the mail server firewall. If you use UFW firewall (Debian/Ubuntu), run the following command on the mail server. Replace 12.34.56.78 with the IP address of the VPN server.

sudo ufw insert 1 allow in from 12.34.56.78

You can also whitelist your other servers’ IP addresses. For example, some folks might have another web server that needs to send emails through the email server. Then also add it in the whitelist.

sudo ufw insert 1 allow in from IP-address-of-the-other-web-server

Step 2: Close Submission Port, IMAP Port, and POP3 Port

• Submission port: 587 and 465
• IMAP port: 143 and 993
• POP3 port: 110 and 995

Port 587 and 465 are used by mail clients like Mozilla Thunderbird and Microsoft Outlook to submit outgoing emails. Hackers can lunch brute force attack on port 587 and 465.

The following is an example found in my mail log (/var/log/mail.log on Debian/Ubuntu, /var/log/maillog on CentOS/RHEL). The bad actor was trying to login, but failed SASL authentication every time.

postfix/smtps/smtpd[18071]: Anonymous TLS connection established from unknown[92.118.38.56]: TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)
postfix/smtps/smtpd[18071]: warning: unknown[92.118.38.56]: SASL LOGIN authentication failed: UGFzc3dvcmQ6
postfix/smtps/smtpd[18071]: lost connection after AUTH from unknown[92.118.38.56]
postfix/smtps/smtpd[18071]: disconnect from unknown[92.118.38.56] ehlo=1 auth=0/1 rset=1 commands=2/3
postfix/smtps/smtpd[18071]: connect from unknown[92.118.38.56]
postfix/smtps/smtpd[18071]: Anonymous TLS connection established from unknown[92.118.38.56]: TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)
postfix/smtps/smtpd[18071]: warning: unknown[92.118.38.56]: SASL LOGIN authentication failed: UGFzc3dvcmQ6
postfix/smtps/smtpd[18071]: lost connection after AUTH from unknown[92.118.38.56]
postfix/smtps/smtpd[18071]: disconnect from unknown[92.118.38.56] ehlo=1 auth=0/1 rset=1 commands=2/3
postfix/smtps/smtpd[18071]: connect from unknown[92.118.38.56]
postfix/smtps/smtpd[18071]: Anonymous TLS connection established from unknown[92.118.38.56]: TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)
postfix/smtps/smtpd[18071]: warning: unknown[92.118.38.56]: SASL LOGIN authentication failed: UGFzc3dvcmQ6
postfix/smtps/smtpd[18071]: lost connection after AUTH from unknown[92.118.38.56]
postfix/smtps/smtpd[18071]: disconnect from unknown[92.118.38.56] ehlo=1 auth=0/1 rset=1 commands=2/3

I don’t want to see this kind of activity in my mail log, so I simply close port 587, 465, 143, 993, 110 and 995 in the firewall. And because my VPN server’s IP address is whitelisted, so only users who are connected to my VPN server can access those ports.

To close these ports in UFW firewall, first list your firewall rules.

sudo ufw status numbered

Sample output

As you can see, there are both IPv4 and IPv6 rules. To close port 587, I need to delete rule #16 and #6. (You should first delete the rule with a higher index number.)

sudo ufw delete 16
sudo ufw delete 6

Then you should run sudo ufw status numbered command again to get a new listing of firewall rules. Notice that the index number changed for some rules.

This time I want to close port 465, so I need to delete rule #15 and #6.

sudo ufw delete 15
sudo ufw delete 6

Then use the same method to close port 143, 993, 110, and 995. Of course, you need to keep port 25 open to receive emails from other SMTP servers.

Step 3: Protecting Admin Panel and Webmail

We can close TCP port 80 and 443 to protect admin panel and webmail from hacking. However, that will forbid public access to all virtual hosts. Some folks might have virtual hosts in Apache/Nginx that need to open to the Internet. Instead of closing port 80 and 443 in the firewall, we can use the builtin access control feature in Apache/Nginx.

Nginx

Edit the virtual host file for webmail, such as

sudo nano /etc/nginx/conf.d/mail.your-domain.com.conf

Add the following lines to the SSL server block that listens on port 443. (You can ignore the server block that listens on port 80.) This will allow only the IP address 12.34.56.78 to access the webmail, and deny all other IP addresses.

allow 12.34.56.78;
deny all;

If you have multiple VPN servers, you can add multiple IP addresses like so:

allow 12.34.56.78;
allow 12.34.56.79;
deny all;

Save and close the file. Then test Nginx configurations.

sudo nginx -t

If the test is successful, reload Nginx for the changes to take effect.

sudo systemctl reload nginx

Users not in the whitelist will see a 403 forbidden error.

Apache

Edit the virtual host file for webmail, such as

sudo nano /etc/apache2/sites-enabled/mail.your-domain.com-le-ssl.conf

Add the following lines between the <VirtualHost>...</VirtualHost> tags. This will allow only the IP address 12.34.56.78 to access the webmail, and deny all other IP addresses.

    <LocationMatch "^/">
       Require ip 12.34.56.78
    </LocationMatch>

If you have multiple VPN servers, you can add multiple IP addresses like so:

    <LocationMatch "^/">
       Require ip 12.34.56.78 12.34.56.79
    </LocationMatch>

Save and close the file. Then test Apache configurations.

sudo apache2ctl -t

If the syntax is Ok, reload Apache for the changes to take effect.

sudo systemctl reload apache2

Users not in the whitelist will see a 403 forbidden error.

Disable DNS Over HTTPS in Your Web Browser

If the VPN server and mail server run on the same host, then you need to disable DNS over HTTPS in your web browser.

• When your computer is connected to the VPN server, your DNS traffic is already encrypted, so you don’t need the DNS over HTTPS feature in the web browser.
• When DNS over HTTPS is enabled in the web browser, you are using a third-party DNS resolver, so your own DNS resolver won’t be used and you will be blocked from accessing the admin panel and webmail.

In Firefox it’s in the network settings. In Chrome, it’s in the Privacy & Security settings (secure DNS). Once it’s disabled, close your web browser, and wait a minute. Then you will be able to access the admin panel and webmail. If it still won’t work, you should clear your browser cache.

Note: You can set up your own DNS over HTTPS resolver, but it’s not necessary when you are connected to your own VPN server.

Certbot TLS Certificate Renewal

If you enable whitelisting in Apache/Nginx virtual host, then you will also block Let’s Encrypt servers to access your web server, which is required for renewing Let’s Encrypt TLS certificate with HTTP-01 challenge. To solve this problem, we can disable whitelisting before certificate renewal and enable it again after the renewal.

Create a shell script in the /root/ directory.

sudo nano /root/certbot-renewal.sh

If you use Nginx, add the following lines to this file.

#! /bin/bash

# disable whitelisting
sed -i 's/deny all;/#deny all;/g' /etc/nginx/conf.d/mail.your-domain.com.conf
systemctl reload nginx

# renew TLS certificate
certbot renew --quiet

# enable whitelisting
sed -i 's/#deny all;/deny all;/g' /etc/nginx/conf.d/mail.your-domain.com.conf
systemctl reload nginx postfix dovecot

If you use Apache, add the following lines to this file.

#! /bin/bash

# disable whitelisting
sed -i 's/Require ip/#Require ip/g' /etc/apache2/sites-enabled/mail.your-domain.com-le-ssl.conf
systemctl reload apache2

# renew TLS certificate
certbot renew --quiet

# enable whitelisting
sed -i 's/#Require ip/Require ip/g' /etc/apache2/sites-enabled/mail.your-domain.com-le-ssl.conf
systemctl reload apache2 postfix dovecot

Save and close the file. Then add execute permission to this file.

sudo chmod +x /root/certbot-renewal.sh

Then edit the root user’s crontab file.

sudo crontab -e

Add the following line at the end of the file, so the shell script will run once a day.

@daily bash /root/certbot-renewal.sh

Save and close the file.

Close SSH Port?

Since your VPN server’s IP address is whitelisted, you can also close SSH port in the firewall. However, doing so carries a risk. If your VPN server stops working, then you would lock yourself out. To protect SSH service from hacking, I recommend setting up public key authentication or two-factor authentication.

• 2 Simple Steps to Set up Passwordless SSH Login on Ubuntu
• Set Up SSH Two-Factor Authentication (2FA) on Ubuntu Server

Wrapping Up

I hope this tutorial helped you secure your email server against hacking. You may also want to check out other security tutorials.

• How to Use UFW Firewall on Debian, Ubuntu, Linux Mint
• Set Up Automatic Security Update (Unattended-Upgrade) on Ubuntu
• Canonical Livepatch Service: Patch Linux Kernel on Ubuntu without Reboot

As always, if you found this post useful, then subscribe to our free newsletter to get new tutorials 🙂