Set Up OpenConnect VPN Server (ocserv) on Debian 12 Bookworm

This tutorial is going to show you how to run your own VPN server by installing OpenConnect VPN server on Debian 12 Bookworm. OpenConnect VPN server, aka ocserv, is an open-source implementation of the Cisco AnyConnnect VPN protocol, which is widely used in businesses and universities. AnyConnect is an SSL-based VPN protocol that allows individual users to connect to a remote network.

Why Set Up Your Own VPN Server?

• Maybe you are a VPN service provider or a system administrator, which behooves you to set up your own VPN server.
• You don’t trust the no-logging policy of VPN service providers, so you go the self-host route.
• You can use VPN to implement network security policy. For example, if you run your own email server, you can require users to log in only from the IP address of the VPN server by creating an IP address whitelist in the firewall. Thus, your email server is hardened to prevent hacking activities.
• Perhaps you are just curious to know how VPN server works.

Features of OpenConnect VPN server

• Lightweight and fast. In my test, I can watch YouTube 4K videos with OpenConnect VPN. YouTube is blocked in my country (China).
• Runs on Linux and most BSD servers.
• Compatible with Cisco AnyConnect client
• There is OpenConnect client software for Linux, macOS, Windows, and OpenWRT. For Android and iOS, you can use the Cisco AnyConnect Client.
• Supports password authentication and certificate authentication
• Supports RADIUS accounting.
• Easy to set up

I particularly like the fact that compared to other VPN technologies, it is very easy and convenient for the end-user to use OpenConnect VPN. Whenever I install a Linux distro on my computer and want to quickly unblock websites or hide my IP address, I install OpenConnect client and connect to the server with just two lines of commands:

sudo apt install openconnect

sudo openconnect -b vpn.mydomain.com

There is also OpenConnect VPN client for Fedora, RHEL, CentOS, Arch Linux, and OpenSUSE. You can easily install it with your package manager.

sudo dnf install openconnect
sudo yum install openconnect
sudo pacman -S openconnect

Requirements

To follow this tutorial, you will need a VPS (Virtual Private Server) that can access blocked websites freely (Outside of your country or Internet filtering system). I recommend Kamatera VPS, which features:

• 30 days free trial.
• Starts at $4/month (1GB RAM)
• High-performance KVM-based VPS
• 9 data centers around the world, including United States, Canada, UK, Germany, The Netherlands, Hong Kong, and Isreal.

Follow the tutorial linked below to create your Linux VPS server at Kamatera.

• How to Create a Linux VPS Server on Kamatera

Once you have a VPS running Debian 12 buster, follow the instructions below.

You also need a domain name to enable HTTPS for OpenConnect VPN. I registered my domain name from NameCheap because the price is low and they give whois privacy protection free for life.

Step 1: Install OpenConnect VPN Server on Debian 12 Bookworm

Log into your Debian 12 Bookworm server via SSH. Then use apt to install the ocserv package from Debian repository.

sudo apt update
sudo apt install ocserv

Once installed, the OpenConnect VPN server is automatically started. You can check its status with:

systemctl status ocserv

Sample output:

● ocserv.service - OpenConnect SSL VPN server
   Loaded: loaded (/lib/systemd/system/ocserv.service; enabled; vendor preset: enabled
   Active: active (running) since Tue 2023-06-20 03:36:19 CEST; 11s ago
     Docs: man:ocserv(8)
 Main PID: 19235 (ocserv-main)
   CGroup: /system.slice/ocserv.service
           ├─19235 ocserv-main                                                  
           └─19242 ocserv-secm 

Hint: If the above command doesn’t quit immediately, you can press the Q key to gain back control of the terminal.

If it’s not running, then you can start it with:

sudo systemctl start ocserv

By default OpenConnect VPN server listens on TCP and UDP port 443. If it’s being used by web server, then the VPN server can’t be started. We will see how to change the port in OpenConnect VPN configuration file later.

If there’s a firewall running on your server, then you will need to open port 80 and 443. For example, if you use UFW, then run the following commands.

sudo ufw allow 80/tcp
sudo ufw allow 443/tcp

Step 2: Install Let’s Encrypt Client (Certbot) on Debian 12 Bookworm Server

The gnutls-bin package installed along with ocserv provides tools to create your own CA and server certificate, but we will obtain and install Let’s Encrypt certificate. The advantage of using Let’s Encrypt certificate is that it’s free, easier to set up and trusted by VPN client software.

Run the following commands to install the Let’s Encrypt client (certbot) on Debian 12.

sudo apt install certbot

To check version number, run

certbot --version

Sample output:

certbot 2.1.0

Step 3: Obtain a Trusted TLS Certificate from Let’s Encrypt

I recommend using the standalone or webroot plugin to obtain TLS certificate.

Standalone Plugin

If there’s no web server running on your Debian 12 Bookworm server and you want OpenConnect VPN server to use port 443, then you can use the standalone plugin to obtain TLS certificate from Let’s Encrypt. Set DNS A record for vpn.yourdomain.com at your domain registrar’s website, then run the following command to obtain certificate.

sudo certbot certonly --standalone --preferred-challenges http --agree-tos --email [email protected] -d vpn.example.com

Explanation:

• certonly: Obtain a certificate but don’t install it.
• --standalone: Use the standalone plugin to obtain a certificate
• --preferred-challenges http: Perform http-01 challenge to validate our domain, which will use port 80.
• --agree-tos: Agree to Let’s Encrypt terms of service.
• --email: Email address is used for account registration and recovery.
• -d: Specify your domain name.

As you can see from the following screenshot, I successfully obtained the certificate.

Using webroot Plugin

If your Debian 12 Bookworm server has a web server listening on port 80 and 443, then it’s a good idea to use the webroot plugin to obtain a certificate because the webroot plugin works with pretty much every web server and we don’t need to install the certificate in the web server.

First, you need to create a virtual host for vpn.example.com.

Apache

If you are using Apache web server, then create virtual host under /etc/apache2/sites-available/ directory.

sudo nano /etc/apache2/sites-available/vpn.example.com.conf

And paste the following lines into the file.

<VirtualHost *:80>        
        ServerName vpn.example.com

        DocumentRoot /var/www/ocserv
</VirtualHost>

Save and close the file. Then create the web root directory.

sudo mkdir /var/www/ocserv

Set www-data (Apache user) as the owner of the web root.

sudo chown www-data:www-data /var/www/ocserv -R

Enable this virtual host.

sudo a2ensite vpn.example.com

Reload Apache for the changes to take effect.

sudo systemctl reload apache2

Once virtual host is created and enabled, run the following command to obtain Let’s Encrypt certificate using webroot plugin.

sudo certbot certonly --webroot --agree-tos --email [email protected] -d vpn.example.com -w /var/www/ocserv/

Nginx

If you are using Nginx web server, then create virtual host under /etc/nginx/conf.d/.

sudo nano /etc/nginx/conf.d/vpn.example.com.conf

Paste the following lines into the file.

server {
      listen 80;
      server_name vpn.example.com;

      root /var/www/ocserv/;

      location ~ /.well-known/acme-challenge {
         allow all;
      }
}

Save and close the file. Then create the web root directory.

sudo mkdir /var/www/ocserv

Set www-data (Nginx user) as the owner of the web root.

sudo chown www-data:www-data /var/www/ocserv -R

Reload Nginx for the changes to take effect.

sudo systemctl reload nginx

Once virtual host is created and enabled, run the following command to obtain Let’s Encrypt certificate using webroot plugin.

sudo certbot certonly --webroot --agree-tos --email [email protected] -d vpn.example.com -w /var/www/ocserv/

Step 4: Edit OpenConnect VPN Server Configuration File

Edit ocserv configuration file.

sudo nano /etc/ocserv/ocserv.conf

First, find the following two lines.

tcp-port = 443
udp-port = 443

Comment out the UDP port. (We will use TCP BBR algorithm to boost TCP speed.)

tcp-port = 443 
#udp-port = 443

If you don’t want ocserv to use TCP port 443 (there’s a web server using port 443?), then change the TCP port number. Otherwise leave it alone.

Then find the following two lines. We need to change them.

#server-cert = /etc/ssl/certs/ssl-cert-snakeoil.pem
#server-key = /etc/ssl/private/ssl-cert-snakeoil.key

Uncomment the two lines and replace the default setting with the path of Let’s Encrypt server certificate and server key file.

server-cert = /etc/letsencrypt/live/vpn.example.com/fullchain.pem
server-key = /etc/letsencrypt/live/vpn.example.com/privkey.pem

Then, set the maximal number of clients. Default is 16. You can set it to zero for unlimited.

max-clients = 16

Set the number of devices a user is able to log in from at the same time. Default is 2. Set to zero for unlimited.

max-same-clients = 2

By default, keepalive packets are sent every 32400 seconds. I prefer to use a short time (30 seconds) to reduce the chance of VPN connection dropout.

keepalive = 32400

Next, find the following line. Change false to true to enable MTU discovery.

try-mtu-discovery = false

After that, set the default domain to vpn.example.com.

default-domain = vpn.example.com

The IPv4 network configuration is as follows by default. This will cause problems because most home routers also set the IPv4 network range to 192.168.1.0/24.

ipv4-network = 192.168.1.0
ipv4-netmask = 255.255.255.0

We can use another private IP address range (such as 10.10.10.0/24) to avoid IP address collision, so change the value of ipv4-network to

ipv4-network = 10.10.10.0

Find the following two lines and uncomment them, so VPN clients will be given private IPv6 addresses.

ipv6-network = fda9:4efe:7e3b:03ea::/48
ipv6-subnet-prefix = 64

If you see the following line

ipv6-network = fda9:4efe:7e3b:03ea::/64

Please change it to:

ipv6-network = fda9:4efe:7e3b:03ea::/48

Now uncomment the following line to tunnel all DNS queries via the VPN.

tunnel-all-dns = true

Change DNS resolver address. You can use Google’s public DNS server.

dns = 8.8.8.8
dns = 8.8.4.4

or Cloudflare’s public DNS server.

dns = 1.1.1.1
dns = 1.0.0.1

Note: If you are a VPN service provider, it’s a good practice to run your own DNS resolver. If there’s a DNS resolver running on the same server, then specify the DNS as

dns = 10.10.10.1

10.10.10.1 is the IP address of OpenConnect VPN server in the VPN LAN. This will speed up DNS lookups a little bit for clients because the network latency between the VPN server and the DNS resolver is eliminated.

Then comment out all the route parameters (add # character at the beginning of the following lines), which will set the server as the default gateway for the clients.

#route = 10.0.0.0/8
#route = 172.16.0.0/12
#route = 192.168.0.0/16
#route = fd00::/8
#route = default

#no-route = 192.168.5.0/255.255.255.0

Save and close the file  Then restart the VPN server for the changes to take effect.

sudo systemctl restart ocserv

Step 5: Creating VPN Accounts

Now use the ocpasswd tool to generate VPN accounts.

sudo ocpasswd -c /etc/ocserv/passwd username

You will be asked to set a password for the user and the information will be saved to /etc/ocserv/passwd file. To reset password, simply run the above command again.

Note: Ocserv supports client certificate authentication, but Let’s Encrypt does not issue client certificate. If you want to enable certificate authentication, you need to set up your own CA to issue client certificate.

Step 6: Enable IP Forwarding in Linux Kernel

In order for the VPN server to route packets between VPN clients and the Internet, we need to enable IP forwarding by running the following command.

echo "net.ipv4.ip_forward = 1" | sudo tee /etc/sysctl.d/60-custom.conf

Also, run the following two commands to enable TCP BBR algorithm to boost TCP speed.

echo "net.core.default_qdisc=fq" | sudo tee -a /etc/sysctl.d/60-custom.conf

echo "net.ipv4.tcp_congestion_control=bbr" | sudo tee -a /etc/sysctl.d/60-custom.conf

Then apply the changes with the below command. The -p option will load sysctl settings from /etc/sysctl.d/60-custom.conf file. This command will preserve our changes across system reboots.

sudo sysctl -p /etc/sysctl.d/60-custom.conf

Step 7: Configure IP Masquerading in Firewall

We need to set up IP masquerading in the server firewall, so that the server becomes a virtual router for VPN clients. I will use UFW, which is a front end to the iptables firewall. Install UFW on Debian with:

sudo apt install ufw

First, you need to allow SSH traffic.

sudo ufw allow 22/tcp

Next, find the name of your server’s main network interface.

ip addr

As you can see, it’s named eth0 on my Debian server.

To configure IP masquerading, we have to add iptables command in a UFW configuration file.

sudo nano /etc/ufw/before.rules

By default, there are some rules for the filter table. Add the following lines at the end of this file. Replace eth0 with your own network interface name.

# NAT table rules
*nat
:POSTROUTING ACCEPT [0:0]
-A POSTROUTING -s 10.10.10.0/24 -o eth0 -j MASQUERADE

# End each table with the 'COMMIT' line or these rules won't be processed
COMMIT

In Nano text editor, you can go to the end of the file by pressing Ctrl+W, then pressing Ctrl+V.

The above lines will append (-A) a rule to the end of of POSTROUTING chain of nat table. It will link your virtual private network with the Internet. And also hide your network from the outside world. So the Internet can only see your VPN server’s IP, but can’t see your VPN client’s IP, just like your home router hides your private home network.

By default, UFW forbids packet forwarding. We can allow forwarding for our private network. Find the ufw-before-forward chain in this file and add the following 3 lines, which will accept packet forwarding if the source IP or destination IP is in the 10.10.10.0/24 range.

# allow forwarding for trusted network
-A ufw-before-forward -s 10.10.10.0/24 -j ACCEPT
-A ufw-before-forward -d 10.10.10.0/24 -j ACCEPT

Save and close the file. Then enable UFW.

sudo ufw enable

If you have enabled UFW before, then you can use systemctl to restart UFW.

sudo systemctl restart ufw

Now if you list the rules in the POSTROUTING chain of the NAT table by using the following command:

sudo iptables -t nat -L POSTROUTING

You can see the Masquerade rule.

It can take some time for UFW to process the firewall rules. If the masquerade rule doesn’t show up, then restart UFW again (sudo systemctl restart ufw).

Step 8: Open Port 443 in Firewall

Run the following command to open TCP and UDP port 443. If you configured a different port for ocserv, then change 443 to your configured port.

sudo ufw allow 443/tcp
sudo ufw allow 443/udp

Now OpenConnect VPN server is ready to accept client connections.

For those of you who run a local DNS resolver, if you specified 10.10.10.1 as the DNS server for VPN clients, then you must allow VPN clients to connect to port 53 with the following UFW rule.

sudo ufw insert 1 allow in from 10.10.10.0/24

You also need to edit the BIND DNS server’s configuration to allow VPN clients to send recursive DNS queries like below.

allow-recursion { 127.0.0.1; 10.10.10.0/24; };

How to Install and Use OpenConnect VPN client on Debian 12 Bookworm Desktop

Run the following command to install OpenConnect VPN command line client on Debian desktop.

sudo apt install openconnect

You can Connect to VPN from the command line like below. -b flag will make it run in the background after connection is established.

sudo openconnect -b vpn.example.com

By default, openconnect client will contact port 443 of the server. If you configured a different port for the server, then you can add the port number.

sudo openconnect -b vpn.example.com:port-number

You will be asked to enter VPN username and password. If connection is successfully established, you will see the following message.

Got CONNECT response: HTTP/1.1 200 CONNECTED
CSTP connected. DPD 60, Keepalive 300
Connected as 10.10.10.139, using SSL + LZ4, with DTLS + LZ4 in progress
Continuing in background; pid 17050

If the connection failed, you can check the ocserv log to find out why. (Perhaps you didn’t enter the password correctly.)

sudo journaltcl -eu ocserv

To stop the connection, run:

sudo pkill openconnect

To run the client non-interactively, use the following syntax.

echo -n password | sudo openconnect -b vpn.example.com -u username --passwd-on-stdin

If you want to use Network Manager to manage VPN connection, then you need to install these packages.

sudo apt install network-manager-openconnect network-manager-openconnect-gnome

If you are successfully connected to the VPN server, but your public IP address doesn’t change, that’s because IP forwarding or IP masquerading is not working. I once had a typo in my iptables command, which caused my computer not being able to browse the Internet.

Auto-Connect on System Startup

To let OpenConnect VPN client automatically connect to the server at boot time, we can create a systemd service unit.

sudo nano /etc/systemd/system/openconnect.service

Put the following lines to the file. Replace the red text.

[Unit]
  Description=OpenConnect VPN Client
  After=network-online.target
  Wants=network-online.target

[Service]
  Type=simple
  ExecStart=/bin/bash -c '/bin/echo -n password | /usr/sbin/openconnect vpn.example.com -u username --passwd-on-stdin'
  KillSignal=SIGINT
  Restart=always
  RestartSec=2

[Install]
  WantedBy=multi-user.target

Save and close the file. Then enable this service so that it will start at boot time.

sudo systemctl enable openconnect.service

Explanation of the file content:

• After=network-online.target and Wants=network-online.target make this service run after network is up.
• In reality, this service can still run before network is up. We add Restart=always and RestartSec=2 to restart this service after 2 seconds if this service fails.
• Systemd doesn’t recognise pipe redirection. So in the ExecStart directive, we wrap the command in single quotes and run it with the Bash shell.
• Since OpenConnect VPN client will run as a systemd service, which runs in the background, there’s no need to add -b flag to the openconnect command.
• The KillSignal directive tells Systemd to send the SIGINT signal when the systemctl stop openconnect command is issued. This will performs a clean shutdown by logging the session off, and restoring DNS server settings and the Linux kernel routing table.

To start this Systemd service immediately, run

sudo systemctl start openconnect

To stop this Systemd service, run

sudo systemctl stop openconnect

Automatic-Restart When VPN Connection Drops

Sometimes the VPN connection would drop due to various reasons. To make the VPN client automatically restart, edit the root user’s crontab file.

sudo crontab -e

Add the following line at the end of this file.

* * * * * ping -c 10 10.10.10.1 > /dev/null || systemctl restart openconnect

This Cron job will run every minute to check if the VPN client can ping the VPN server’s private IP address (10.10.10.1). If the ping is unsuccessful, then the command on the right will be executed to restart the VPN client. || is the OR operator in Bash. It executes the command on the right only if the command on the left returned an error.

Save and close the file.

OpenConnect GUI Client for Windows and MacOS

They can be downloaded from OpenConnect GUI Github Page.

Speed

OpenConnect VPN is pretty fast. I can use it to watch 4k videos on YouTube. As you can see, the connection speed is 63356 Kbps, which translates to 61 Mbit/s.

And here’s the test results on speedtest.net.

Speed Optimization

OpenConnect by default uses TLS over UDP protocol (DTLS) to achieve faster speed, but UDP can’t provide reliable transmission. TCP is slower than UDP but can provide reliable transmission. One optimization tip I can give you is to disable DTLS, use standard TLS (over TCP), then enable TCP BBR to boost TCP speed.

To disable DTLS, comment out (add # symbol at the beginning) the following line in ocserv configuration file.

udp-port = 443

Save and close the file. Then restart ocserv service.

sudo systemctl restart ocserv.service

To enable TCP BBR, please check out the following tutorial. Note that you need to disable DTLS in ocserv, or TCP BBR won’t work.

• How to Easily boost Debian Network Performance by enabling TCP BBR

In my test, standard TLS with TCP BBR enabled is two times faster than DTLS.

Another very important factor affecting speed is how good the connection between your local computer and the VPN server is. If you live in the middle east and the VPN server is located in the U.S, the speed would be slow. Choose a data center that’s close to where you live.

Auto-Renew Let’s Encrypt Certificate

Edit root user’s crontab file.

sudo crontab -e

Add the following line at the end of the file to run the Cron job daily. If the certificate is going to expire in 30 days, certbot will try to renew the certificate. It’s necessary to restart ocserv service for the VPN server to pick up new certificate and key file.

@daily certbot renew --quiet && systemctl restart ocserv

Troubleshooting Tips

Note that if you are using OpenVZ VPS, make sure you enable the TUN virtual networking device in VPS control panel. (If you use Kamatera VPS, then you have KVM-based VPS, so you don’t have to worry about this.)

If you encounter any problem, then check OpenConnect VPN server log.

sudo journalctl -eu ocserv.service

I found that if I change port 443 to a different port, the great firewall of China will block this VPN connection.

If ocserv tells you that it can’t load the /etc/ocserv/ocserv.conf file, you can stop ocserv.

sudo systemctl stop ocserv

Then run it in the foreground with debugging enabled.

sudo /usr/sbin/ocserv --foreground --pid-file /run/ocserv.pid --config /etc/ocserv/ocserv.conf --debug=10

Then output might give you some clues why ocserv isn’t working.

No Tap-Windows Adapter installed on this system

If you use the OpenConnect GUI Windows client and found the “No Tap-Windows Adapter installed on this system” error in the logs (View – Log Window), this is likely because you have install other VPN clients afterward like OpenVPN.

To fix this error, you need to uninstall OpenConnect GUI client and reinstall it. In the setup wizard, you will have the option to install the TAP driver.

Make OpenConnect VPN Server and Web Server Use Port 443 at the Same Time

Please read the following article:

• Run OpenConnect VPN Server & Apache/Nginx on the Same Box with HAProxy

How to Disable TLS 1.0 and TLS 1.1 in ocserv

The PCI council deprecated TLS 1.0 in June 30, 2018 and main stream web browsers are going to disable TLS 1.0 and TLS 1.1 in 2020. We should do the same with VPN server. Edit the main configuration file.

sudo nano /etc/ocserv/ocserv.conf

Find the following line:

tls-priorities = "NORMAL:%SERVER_PRECEDENCE:%COMPAT:-RSA:-VERS-SSL3.0:-ARCFOUR-128"

To disable TLS 1.0 and TLS 1.1 in OpenConnect VPN server, just add -VERS-TLS1.0 and -VERS-TLS1.1 in the line.

tls-priorities = "NORMAL:%SERVER_PRECEDENCE:%COMPAT:-RSA:-VERS-SSL3.0:-ARCFOUR-128:-VERS-TLS1.0:-VERS-TLS1.1"

Save and close the file. Then restart ocserv.

sudo systemctl restart ocserv

Now ocserv will only accept TLS 1.3 and TLS 1.2. For further information on configuring the TLS parameter in ocserv, please see GnuTLS priority strings.

To check if TLS 1.0 is supported in your OpenConnect VPN server, run the following command.

openssl s_client -connect vpn.your-domain.com:443 -tls1

And check TLS 1.1

openssl s_client -connect vpn.your-domain.com:443 -tls1_1

If you see the following message in the output, that means the TLS version is not supported.

New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported

Per-User or Per Group Configuration

Ocserv allows per user and per group configurations. To enable this feature, uncomment the following two lines in /etc/ocserv/ocserv.conf file.

config-per-user = /etc/ocserv/config-per-user/
config-per-group = /etc/ocserv/config-per-group/

Save and close the file. Then create the per user and per group config directory.

sudo mkdir /etc/ocserv/config-per-user/
sudo mkdir /etc/ocserv/config-per-group/

Next, you can create a file under these two directories. For example, create the user1 file to allow custom configuration for user1.

sudo nano /etc/ocserv/config-per-user/user1

You can also create the group1 file to allow custom configuration for the group named group1.

sudo nano /etc/ocserv/config-per-group/group1

You can add something like below in the file.

route = 10.10.10.0/255.255.255.0

This means that after user1 connect to this VPN server, only traffic to the 10.10.10.0/24 network will be routed via the VPN server. Traffic to other IP addresses is routed via the original gateway. This is known as split tunneling, which is useful when:

• You just want the VPN clients to be able to browse internal resources and don’t want all the traffic to go through the VPN server.
• You need to build a private network for cloud servers.
• The client needs to connect to multiple VPNs. One VPN could use split tunneling and the other use a full tunnel.

Save and close the file. Restart ocserv for the changes to take effect.

How to Run Multiple Instances of ocserv

One ocserv process can bind to only one TCP or UDP port on your server. If you want to allow ocserv to bind to multiple TCP or UDP ports, then you need to run multiple ocserv processes. It’s very simple. Copy the /lib/systemd/system/ocserv.service to a new file.

sudo cp /lib/systemd/system/ocserv.service /etc/systemd/system/ocserv2.service

Then edit the new file.

sudo nano /etc/systemd/system/ocserv2.service

Change

/etc/ocserv/ocserv.conf

to

/etc/ocserv/ocserv2.conf

Save and close the file. Next, you can edit the /etc/ocserv/ocserv2.conf file and add your custom configurations. Once you are done, start the second ocserv service.

sudo systemctl start ocserv2

Wrapping Up

That’s it! I hope this tutorial helped you install and configure OpenConnect VPN on Debian 12 Bookworm server. As always, if you found this post useful, then subscribe to our free newsletter to get more tips and tricks. Take care 🙂