Fast Scalable Basemap with TileServer GL/OpenMapTiles (Ubuntu 22.04)
TileServer GL is an open-source tile server for OpenStreetMap. Previously we explained the process of setting up OSM tile server with mapnik and mod_tile, which is a raster-based tile server. This tutorial is going to show you how to set up TileServer GL vector tile server on Ubuntu 22.04.
TileServer GL
TileServer GL can serve both vector tiles (pbf) and raster tiles (png, jpg, webp). It’s simple to use. You just need to feed it a .mbtiles file, which will be generated by OpenMaptiles from a PostGIS database. The overall Steps are as follows:
- Import OpenStreetMap data to PostgreSQL
- Use OpenMapTiles to generate
.mbtilesfile - Start TileServer GL with the
.mbtilesfile to serve tiles
MBTiles are like pre-rendered map tiles. The PostGIS database won’t be needed anymore once the .mbtiles file is generated. That’s why it’s very fast to serve tiles with TileServer GL.
TileServer GL is designed to work with MBTiles in openmaptiles format, so we must use OpenMapTiles tools to generate the .mbtiles file with the latest map data.
Demo
Mapbox Streets style: https://www.linuxbabe.com/maps/mapbox-street.html
Maplibre Streets style: https://www.linuxbabe.com/maps/maplibre-streets.html
Benefits of Vector Tiles
- Better display quality for high DPI devices (retina display)
- Small efficient format (No 512 * 512 images needed)
- Fast map rendering
- Clearer, more readable text
- On-the-fly labeling for heads-up display
- Separate content and styling, which allows for creating multiple styles pointing to the same tile stack.
- More fine-grained zoom levels such as 8.43 and 9.57. With raster tiles, you can only have integer zoom levels such as 8, 9, 10.
- Day and night mode
Vector Tile Formats
There are several formats for vector tiles.
- GeoJSON
- TopoJSON
- Mapbox Vector Tile (MVT)
- 05m
- OpenScienceMap binary
- Arc GeoServices JSON
OpenMapTiles uses the Mapbox vector tile format.
Hardware Requirements
It takes a long time to import large map data to the PostgreSQL database and generate .mbtiles file from the database. Your server should have a powerful CPU and fast SSD. Also consider adding more RAM.
If you are going to host the entire world map, I recommend you buy the extra-large VPS from Contabo, which has
- A 10-core CPU
- 60 GB RAM
- 1.6 TB Intel Optane SSD
It costs just 26.99 €/month.
Step 1: Upgrade Software
It’s always a good practice to update server software before doing any major work on your server. Log into your server via SSH and run the following command.
sudo apt update; sudo apt upgrade -y
Step 2: Install PostgreSQL Database Server and Some Extensions
We will use PostgreSQL to store map data. The PostgreSQL team always strives to make performance improvements with every new version. Run the following commands to install the latest version of PostgreSQL.
echo "deb [signed-by=/etc/apt/keyrings/postgresql.asc] http://apt.postgresql.org/pub/repos/apt $(lsb_release -cs)-pgdg main" | sudo tee /etc/apt/sources.list.d/pgdg.list sudo mkdir -p /etc/apt/keyrings/ wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | sudo tee /etc/apt/keyrings/postgresql.asc sudo apt update sudo apt install -y postgresql postgresql-contrib postgresql-15 postgresql-client-15
Then install PostGIS, which is a geospatial extension to PostgreSQL.
sudo apt install postgis postgresql-15-postgis-3
Note: If you have installed PostgreSQL 14 before, and now you install PostgreSQL 15, then you should upgrade it by following the instructions in the “How to Upgrade PostgreSQL Database Server” section at the end of this article before continuing the steps below.
Install PostgreSQL GZIP extension.
cd ~ sudo apt install git git clone https://github.com/pramsey/pgsql-gzip cd pgsql-gzip/ sudo apt install build-essential zlib1g-dev postgresql-server-dev-all pkg-config make sudo make install
The extension will be installed under /usr/share/postgresql/15/extension/.
Install OSM Localization Extension (osml10n)
cd ~ git clone https://github.com/giggls/mapnik-german-l10n cd mapnik-german-l10n/ sudo apt install devscripts equivs python3 python3-pip -y sudo pip3 install tltk sudo mk-build-deps -i debian/control wget -O country_osm_grid.sql.gz https://www.nominatim.org/data/country_grid.sql.gz gunzip country_osm_grid.sql.gz make deb sudo apt install ../postgresql*osml10n*amd64.deb
The extension will be installed under /usr/share/postgresql/15/extension/.
Step 3: Create PostgreSQL Database For OpenStreetMap
PostgreSQL database server automatically starts and listens on 127.0.0.1:5432. The postgres user is the super user for PostgreSQL database server. By default, this user has no password and there’s no need to set one because you can use sudo to switch to the postgres user and log into PostgreSQL server.
sudo -u postgres -i
Now you can create a PostgreSQL database user osm.
createuser osm
Set a password for the osm user. The password should not contain any special characters, or it might prevent OSM tools to access PostgreSQL database.
psql -c "ALTER USER osm WITH PASSWORD 'secret_password';"
Then create a database named openstreetmap and at the same time make osm as the owner of the database.
createdb -E UTF8 -O osm openstreetmap
Next, create the postgis and hstore extension for the openstreetmap database.
psql -d openstreetmap -c "CREATE EXTENSION postgis;" psql -d openstreetmap -c "CREATE EXTENSION hstore;"
Enable the GZIP and osml10n extension on the openstreetmap database.
psql -d openstreetmap -c "CREATE EXTENSION gzip;" psql -d openstreetmap -c "CREATE EXTENSION osml10n CASCADE;"
Set osm as the table owner.
psql -d openstreetmap -c "ALTER TABLE spatial_ref_sys OWNER TO osm;"
Grant permission to the osm user.
psql -d openstreetmap -c "GRANT ALL ON ALL TABLES IN SCHEMA public TO osm;" psql -d openstreetmap -c "ALTER DEFAULT PRIVILEGES FOR USER osm IN SCHEMA public GRANT ALL ON TABLES TO osm;"; psql -d openstreetmap -c "ALTER DEFAULT PRIVILEGES FOR USER osm IN SCHEMA public GRANT ALL ON SEQUENCES TO osm;";
Create a database named natural_earth and at the same time make osm as the owner of the database.
createdb -E UTF8 -O osm natural_earth
Next, create the postgis and hstore extension for the natural_earth database.
psql -c "CREATE EXTENSION postgis;" -d natural_earth psql -c "CREATE EXTENSION hstore;" -d natural_earth
Exit from the postgres user.
exit
Step 4: Optimize PostgreSQL Server Performance
The import process can take some time. To speed up this process, we can tune some PostgreSQL server settings to improve performance. Edit PostgreSQL main configuration file.
sudo nano /etc/postgresql/15/main/postgresql.conf
First, we should change the value of shared_buffer. The default setting is:
shared_buffers = 128MB
This is too small. The rule of thumb is to set it to 25% of your total RAM (excluding swap space). For example, my VPS has 60G RAM, so I set it to:
shared_buffers = 15GB
Find the following line.
#work_mem = 4MB #maintenance_work_mem = 64MB
Again, the value is too small. I use the following settings.
work_mem = 1GB maintenance_work_mem = 8GB
Then find the following line.
#effective_cache_size = 4GB
If you have lots of RAM like I do, you can set a higher value for the effective_cache_size like 20G.
effective_cache_size = 20GB
Save and close the file. Restart PostgreSQL for the changes to take effect.
sudo systemctl restart postgresql
By default, PostgreSQL would try to use huge pages in RAM. However, Linux by default does not allocate huge pages. Check the process ID of PostgreSQL.
sudo head -1 /var/lib/postgresql/15/main/postmaster.pid
Sample output:
7031
Then check the VmPeak value of this process ID.
grep ^VmPeak /proc/7031/status
Sample output:
VmPeak: 16282784 kB
This is the peak memory size that will be used by PostgreSQL. Now check the size of huge page in Linux.
cat /proc/meminfo | grep -i huge
Sample output:
AnonHugePages: 0 kB ShmemHugePages: 0 kB HugePages_Total: 0 HugePages_Free: 0 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 2048 kB Hugetlb: 0 kB
We can calculate how many huge pages we need. Divide the VmPeak value by the size of huge page: 16282784 kB / 2048 kB = 7950. Then we need to edit Linux kernel parameters.
sudo nano /etc/sysctl.d/60-custom.conf
Add the following line to allocate 7950 huge pages.
vm.nr_hugepages = 7950
Save and close the file. Then apply the changes.
sudo sysctl -p /etc/sysctl.d/60-custom.conf
If you check the meminfo again,
cat /proc/meminfo | grep -i huge
We can see there are 7950 huge pages available.
AnonHugePages: 0 kB ShmemHugePages: 0 kB HugePages_Total: 7950 HugePages_Free: 7950 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 2048 kB
Restart PostgreSQL to use huge pages.
sudo systemctl restart postgresql
Step 5: Install Imposm3
Imposm is an importer for OpenStreetMap data.
cd ~ wget https://github.com/omniscale/imposm3/releases/download/v0.11.1/imposm-0.11.1-linux-x86-64.tar.gz tar xvf imposm-0.11.1-linux-x86-64.tar.gz sudo mv imposm-0.11.1-linux-x86-64 /opt/imposm
Now you invoke imposm with:
/opt/imposm/imposm
Step 6: Install OpenMapTiles Tools
OpenMapTiles relies on Docker, so we need to install Docker.
sudo apt install docker.io docker-compose
Add your user account to the docker group.
sudo usermod -aG docker username
Log out and log back in for the change to take effect. Then run the following command to check running Docker containers.
docker ps
It should at least output the following texts.
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
Clone the OpenMapTiles Github repo.
cd ~ git clone https://github.com/openmaptiles/openmaptiles.git
Create the data.yml, mapping.yml and SQL files.
cd openmaptiles/ sudo make
Edit the docker-compose.yml file.
nano docker-compose.yml
Change the port nubmer to 2345.
postgres:
image: "${POSTGIS_IMAGE:-openmaptiles/postgis}:${TOOLS_VERSION}"
# Use "command: postgres -c jit=off" for PostgreSQL 11+ because of slow large MVT query processing
volumes:
- pgdata:/var/lib/postgresql/data
networks:
- postgres
ports:
- "2345"
env_file: .env
Save and close the file. Then edit the .env file.
nano .env
Find the following lines.
# Make sure these values are in sync with the ones in .env-postgres file PGDATABASE=openmaptiles PGUSER=openmaptiles PGPASSWORD=openmaptiles PGHOST=postgres PGPORT=5432
Change them to the following. 172.17.0.1 is the Docker network interface.
PGDATABASE=openstreetmap
PGUSER=osm
PGPASSWORD=osm_password
PGHOST=172.17.0.1
PGPORT=5432
Find the following lines.
# Which zooms to generate with make generate-tiles-pg MIN_ZOOM=0 MAX_ZOOM=7
Set max zoom level to 14.
# Which zooms to generate with make generate-tiles-pg MIN_ZOOM=0 MAX_ZOOM=14
Save and close the file. Create a .osmenv file in your home directory.
nano ~/.osmenv
Add the following lines.
export PGDATABASE=openstreetmap
export PGUSER=osm
export PGPASSWORD=osm_password
export PGHOST=172.17.0.1
export PGPORT=5432
Save and close the file. Then run the following command to set the above environment variables.
source ~/.osmenv chmod 700 ~/.osmenv
OpenMapTiles will start a Docker container and access PostgreSQL database via 172.17.0.1, so we need to configure PostgreSQL to listen on this IP address.
sudo nano /etc/postgresql/15/main/postgresql.conf
Find the following line.
#listen_addresses = 'localhost'
Change it to:
listen_addresses = 'localhost,172.17.0.1'
Save and close the file. Then edit pg_hba.conf file.
sudo nano /etc/postgresql/15/main/pg_hba.conf
Find the following lines.
# IPv4 local connections: host all all 127.0.0.1/32 scram-sha-256
This allows users to log in to PostgreSQL from the 127.0.0.1 IP address. We need to add the 172.17.0.0/24 and 172.18.0.0/24 network to allow login from Docker.
# IPv4 local connections: host all all 127.0.0.1/32 scram-sha-256 host all all 172.17.0.0/24 scram-sha-256 host all all 172.18.0.0/24 scram-sha-256
Save and close the file. Restart PostgreSQL for the changes to take effect.
sudo systemctl restart postgresql
If you enabled firewall on the server, you should also allow connections from the 172.17.0.0/24 and 172.18.0.0/24 network. For example, if you use the UFW firewall, run the following command.
sudo ufw insert 1 allow in from 172.17.0.0/24 sudo ufw insert 1 allow in from 172.18.0.0/24
You also need to allow SSH.
sudo ufw allow ssh
Enable and restart UFW for the changes to take effect.
sudo ufw enable sudo systemctl restart ufw
List the PostgreSQL listening addresses:
sudo ss -lnpt | grep postgres
You should see that PostgreSQL listens on both 127.0.0.1 and 172.17.0.1.
LISTEN 0 244 172.17.0.1:5432 0.0.0.0:* users:(("postgres",pid=19767,fd=6))
LISTEN 0 244 127.0.0.1:5432 0.0.0.0:* users:(("postgres",pid=19767,fd=5))
Step 7: Use Screen on Remote Servers
It can take a long time to import the OSM pbf file and your computer might be disconnected from Internet, it’s recommended to use the screen utility to keep your session alive. Install screen on the Ubuntu 22.04 server:
sudo apt install screen
Then start screen:
screen
Upon the first launch, you will see an introduction text, simply press Enter to end. Then you will be able to run commands as usual.
The GNU screen session will reset environment variables, so we need to set them again.
source ~/.osmenv
Run the following command to check the PostgreSQL environment variables are set.
env
Step 8: Import the Map Data to PostgreSQL
Now we can import external OSM data from OpenStreetMapData, Natural Earth and OpenStreetMap Lake Labels into the PostgreSQL database.
cd ~/openmaptiles/ sudo make import-data
If you encounter the following error, it’s because you didn’t change the port number in docker-compose.yml file.
Error starting userland proxy: listen tcp4 0.0.0.0:5432: bind: address already in use
To fix this error, you should change the port number, then remove the existing Docker container.
sudo docker rm openmaptiles_postgres_1
And run sudo make import-data again.
Next, change to the data directory.
cd ~/openmaptiles/data/
Then run one of the following commands to download the map data in PBF (ProtoBufBinary) format to the data directory.
Britain and Ireland (1.7G)
sudo wget -c http://download.geofabrik.de/europe/britain-and-ireland-latest.osm.pbf
Europe (25.8G)
sudo wget -c http://download.geofabrik.de/europe-latest.osm.pbf
North America (11.8G)
sudo wget -c http://download.geofabrik.de/north-america-latest.osm.pbf
South America (2.9G)
sudo wget -c http://download.geofabrik.de/south-america-latest.osm.pbf
Central America (570MB)
sudo wget -c http://download.geofabrik.de/central-america-latest.osm.pbf
Asia (11.2G)
sudo wget -c http://download.geofabrik.de/asia-latest.osm.pbf
Africa (5.5G)
sudo wget -c http://download.geofabrik.de/africa-latest.osm.pbf
Whole planet (66G). Note: I recommend only downloading the whole plant map when you really need to display the map of the whole world, or you will waste time waiting for the tile server to process unnecessary data.
sudo wget -c http://planet.openstreetmap.org/pbf/planet-latest.osm.pbf
or
sudo wget -c https://download.bbbike.org/osm/planet/planet-latest.osm.pbf
If you want other map of individual country/state/province/city, go to http://download.geofabrik.de. Also, BBBike.org provides extracts of more than 200 cities and regions worldwide in different formats.
Once the map data is downloaded, run the following commands to import the PBF file. If your map data is very large, then this process will take some time.
cd ~/openmaptiles/ sudo make import-osm
Now you probably don’t need to do other things on your server. Since you are using GNU Screen, you can press Ctrl+A, release those keys, and then press D key to detach from the current Screen session. You will see a message like below.
[detached from 32113.pts-1.focal]
This tells you that the previous Screen session ID is 32113. You can log out from the SSH session and even shut down your computer. Don’t worry, the OSM import process is still running. When you need to come back and check the import progress, SSH into your server and run the following command to get the previous Screen Session ID.
screen -ls
Sample output:
There is a screen on: 32113.pts-1.focal (05/19/2020 03:45:29 PM) (Detached) 1 Socket in /run/screen/S-linuxbabe.
Then you can re-attach to the previous Screen session.
screen -r 32113
And you will be able to continue your work.
Once the OSM map import is finished, import labels from Wikidata.
sudo make import-wikidata
Extract bbox (bounding box) from the .osm.pbf file.
sudo make generate-bbox-file
Import SQL files.
sudo make import-sql
Generate vector tiles (.mbtiles file). If your map data is very large, then this process will take some time. The generated file will be saved as ~/openmaptiles/data/tile.mbtiles.
sudo make generate-tiles-pg
Step 9: Install TileServer GL
Docker
The easiest way is to use Docker. It will listen on TCP port 8080.
sudo docker run --restart=always -it -d -v /home/username/openmaptiles/data:/data -p 8080:8080 maptiler/tileserver-gl
TileServer GL will automatically detect .mbtiles file in the data directory.
Native Binary
If you want to run TileServer GL natively, you can install it with npm (NodeJS package manager), but I don’t recommend this method. It’s error-prone.
TileServer GL is compatible with NodeJS v14 and v16.
curl -fsSL https://deb.nodesource.com/setup_16.x | sudo -E bash - sudo apt install nodejs
Then install the tileserver-gl package. (Please don’t run this command as root user. You should create a sudo user.)
sudo npm install -g tileserver-gl
Now you can start the tile server.
sudo tileserver-gl --mbtiles /home/username/openmaptiles/data/tiles.mbtiles
Firewall
Open TCP port 8080 in firewall.
sudo ufw allow 8080/tcp sudo systemctl restart ufw
Then in your web browser address bar, type
your-server-ip-address:8080
You should see the vector tile map. Congrats! You just successfully built your own vector tile server. Note that old versions of Firefox can’t display these vector tiles. You need to use a third-party library to display vector tile based maps, which is explained at the end of this tutorial.
Step 10: Generate Fonts for OpenStreetMap
Download the OpenMapTiles fonts generation tool.
cd ~ git clone https://github.com/openmaptiles/fonts.git
Install NodeJS.
sudo apt install nodejs npm
Install required nodejs module.
cd fonts/ npm install
Generate fonts, which will be saved in the _output directory.
node ./generate.js
Move this directory to /var/www/.
sudo mkdir -p /var/www/ sudo mv _output/ /var/www/font-family/
It’s also recommeneded to download the KlokanTech Noto Sans font family.
cd ~ git clone https://github.com/klokantech/klokantech-gl-fonts.git
Move it to /var/www/font-family/ directory.
sudo mv klokantech-gl-fonts/* /var/www/font-family/
Some map styles expect the font name Klokantech Noto Sans, so we need to make a copy.
cd /var/www/font-family/ sudo cp -r KlokanTech\ Noto\ Sans\ Regular/ Klokantech\ Noto\ Sans\ Regular/ sudo cp -r KlokanTech\ Noto\ Sans\ Bold/ Klokantech\ Noto\ Sans\ Bold sudo cp -r KlokanTech\ Noto\ Sans\ CJK\ Bold/ Klokantech\ Noto\ Sans\ CJK\ Bold/ sudo cp -r KlokanTech\ Noto\ Sans\ CJK\ Regular/ Klokantech\ Noto\ Sans\ CJK\ Regular/ sudo cp -r KlokanTech\ Noto\ Sans\ Italic/ Klokantech\ Noto\ Sans\ Italic/
Step 11: Setting Up Reverse Proxy
To access TileServer GL using a domain name, we can set up a reverse proxy for TileServer GL with Nginx or Apache. This will also allow us to enable HTTPS with free Let’s Encrypt certificate.
Open TCP ports 80 and 443 in the firewall.
sudo ufw allow 80,443/tcp sudo systemctl restart ufw
Nginx
Nginx is a very popular web server and reverse proxy. If you prefer to use Nginx, run the following command to install it.
sudo apt install nginx
Then create a server block file for TileServer GL.
sudo nano /etc/nginx/conf.d/tileserver-gl.conf
Add the following content to this file. Replace tileserver.example.com with your own domain name. You should also create DNS A record for this sub-domain. If you don’t have a real domain name, I recommend going to NameCheap to buy one. The price is low and they give whois privacy protection free for life.
server {
listen 80;
listen [::]:80;
server_name tileserver.example.com;
access_log /var/log/nginx/openstreetmap.access;
error_log /var/log/nginx/openstreetmap.error;
location /font-family/ {
alias /var/www/font-family/;
}
location / {
proxy_pass http://127.0.0.1:8080;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Protocol $scheme;
proxy_set_header X-Forwarded-Host $http_host;
proxy_hide_header Access-Control-Allow-Origin;
}
# This header tells web browsers not to cache the CORS reponse header.
add_header 'Vary' "Origin";
# This controls which domains can use this tile server via CORS. It's like an access control list.
add_header "Access-Control-Allow-Origin" "https://www.example.com";
# If you want to allow all domains to use your tile server, then comment out the above line and uncomment the following line.
# add_header "Access-Control-Allow-Origin" "$http_origin";
}
Save and close this file. Then test Nginx configuration.
sudo nginx -t
If the test is successful, reload Nginx for the change to take effect.
sudo systemctl reload nginx
Now you can access TileServer GL via tileserver.example.com.
Apache
If you prefer Apache over Nginx, then install Apache web server by using the following command.
sudo apt install apache2
To use Apache as a reverse proxy, we need to enable the proxy modules and the header module.
sudo a2enmod proxy proxy_http headers substitute
Then create a virtual host file for TileServer GL.
sudo nano /etc/apache2/sites-available/tileserver-gl.conf
Put the following configurations into the file. Replace tileserver.example.com with your actual domain name. Don’t forget to create DNS A record for this sub-domain. If you don’t have a real domain name, I recommend going to NameCheap to buy one. The price is low and they give whois privacy protection free for life.
<VirtualHost *:80> ServerName tileserver.example.com ErrorDocument 404 /404.html # disable proxy for the /font-family sub-directory # must be placed on top of the other ProxyPass directive ProxyPass /font-family ! Alias "/font-family" "/var/www/font-family" #HTTP proxy ProxyPass / http://127.0.0.1:8080/ ProxyPassReverse / http://127.0.0.1:8080/ ProxyPreserveHost On # Only allow authorizied domains to fetch resources on this tile server SetEnvIf Origin "^http(s)?://(.+\.)?(example\.com|otherdomain\.tld)$" origin_is=$0 Header unset Access-Control-Allow-Origin Header always set Access-Control-Allow-Origin %{origin_is}e env=origin_is Header merge Vary Origin ErrorLog ${APACHE_LOG_DIR}/tileserver-gl.error.log CustomLog ${APACHE_LOG_DIR}/tileserver-gl.access.log combined </VirtualHost>
Save and close the file. Then enable this virtual host.
sudo a2ensite tileserver-gl.conf
Restart Apache
sudo systemctl restart apache2
Now you can access TileServer GL using the domain name tileserver.example.com.
Step 12: Enable HTTPS
To encrypt the HTTP traffic when you visit TileServer GL server from outside, we can enable HTTPS by installing a free TLS certificate issued from Let’s Encrypt. Run the following command to install Let’s Encrypt client (certbot) on Ubuntu 22.04.
sudo apt install certbot
If you use Nginx, then you also need to install the Certbot Nginx plugin.
sudo apt install python3-certbot-nginx
Next, run the following command to obtain and install TLS certificate.
sudo certbot --nginx --agree-tos --redirect --hsts --staple-ocsp --email [email protected] -d tileserver.example.com
If you use Apache, then you need to install the Certbot Apache plugin.
sudo apt install python3-certbot-apache
Next, run the following command to obtain and install TLS certificate.
sudo certbot --apache --agree-tos --redirect --hsts --staple-ocsp --uir --email [email protected] -d tileserver.example.com
Where:
--nginx: Use the nginx plugin.--apache: Use the Apache plugin.--agree-tos: Agree to terms of service.--redirect: Force HTTPS by 301 redirect.--hsts: Add the Strict-Transport-Security header to every HTTP response. Forcing browser to always use TLS for the domain. Defends against SSL/TLS Stripping.--staple-ocsp: Enables OCSP Stapling. A valid OCSP response is stapled to the certificate that the server offers during TLS.--uir: upgrade insecure requests.
The certificate should now be obtained and automatically installed.
And you can access TileServer GL via HTTPS: https://tileserver.example.com.
If you click the vector HTTP link, you will see an example map with the Klokantech Basic Style.
You can use the following HTML code to display the map on any web page.
<!DOCTYPE html> <html> <head> <meta charset="UTF-8"> <meta name="viewport" content="width=device-width, initial-scale=1"> <title>Basic preview - TileServer GL</title> <link rel="stylesheet" type="text/css" href="https://tileserver.example.com/mapbox-gl.css" /> <script src="https://tileserver.example.com/mapbox-gl.js"></script> <link rel="stylesheet" type="text/css" href="https://tileserver.example.com/mapbox.css" /> <script src="https://tileserver.example.com/mapbox.js"></script> <script src="https://tileserver.example.com/leaflet-hash.js"></script> <style> body { margin:0; padding:0; } #map { position:absolute; top:0; bottom:0; width:100%; } </style> </head> <body> <h1 style="display:none;">Basic preview</h1> <div id='map'></div> <script> var q = (location.search || '').substr(1).split('&'); var preference = q.indexOf('vector') >= 0 ? 'vector' : (q.indexOf('raster') >= 0 ? 'raster' : (mapboxgl.supported() ? 'vector' : 'raster')); if (preference == 'vector') { mapboxgl.setRTLTextPlugin('https://tileserver.example.com/mapbox-gl-rtl-text.js'); var map = new mapboxgl.Map({ container: 'map', style: 'https://tileserver.example.com/styles/basic-preview/style.json', zoom: 5.43, center: [-3.9, 54.5], hash: true }); map.addControl(new mapboxgl.NavigationControl()); } else { var map = L.mapbox.map('map', 'https://tileserver.example.com/styles/basic-preview.json', { zoomControl: false }); new L.Control.Zoom({ position: 'topright' }).addTo(map); setTimeout(function() { new L.Hash(map); }, 0); } </script> </body> </html>
Note that if you serve the map over HTTPS, you need to add the upgrade-insecure-requests header to convert all HTTP URLs to HTTPS URLs.
Nginx
add_header Content-Security-Policy upgrade-insecure-requests;
Apache
Header always set Content-Security-Policy upgrade-insecure-requests
Step 13 : Use Other Open-Source Map Styles
The following is a list of open-source map styles compatible with OpenMapTiles vector tile schema.
- MapTiler Basic:
- OSM Bright
- Positron
- Dark Matter
- MapTiler 3D
- MapTiler Terrain
- Fiord Color
- Toner
- OSM Liberty
Let me show you how to use these map styles. For example, I download the MapTiler 3D style.
git clone https://github.com/openmaptiles/maptiler-3d-gl-style.git
Then edit the style.json file.
cd maptiler-3d-gl-style/ nano style.json
Find the following lines.
"sources": {
"openmaptiles": {
"type": "vector",
"url": "https://api.maptiler.com/tiles/v3/tiles.json?key={key}"
}
},
"glyphs": "https://api.maptiler.com/fonts/{fontstack}/{range}.pbf?key={key}",
Change the tileset URL and glyphs URL:
"sources": {
"openmaptiles": {
"type": "vector",
"url": "https://tileserver.example.com/data/v3.json"
}
},
"glyphs": "https://tileserver.example.com/font-family/{fontstack}/{range}.pbf",
Save and close the file. Then upload this style.json file to a web server and create an HTML file.
nano maptiler-3d.html
Content:
<!DOCTYPE html> <html> <head> <meta charset="UTF-8"> <meta name="viewport" content="width=device-width, initial-scale=1"> <title>Maptiler 3D Style</title> <link rel="stylesheet" type="text/css" href="https://tileserver.example.com/mapbox-gl.css" /> <script src="https://tileserver.example.com/mapbox-gl.js"></script> <link rel="stylesheet" type="text/css" href="https://tileserver.example.com/mapbox.css" /> <script src="https://tileserver.example.com/mapbox.js"></script> <script src="https://tileserver.example.com/leaflet-hash.js"></script> <style> body { margin:0; padding:0; } #map { position:absolute; top:0; bottom:0; width:100%; } </style> </head> <body> <h1 style="display:none;">Basic preview</h1> <div id='map'></div> <script> var q = (location.search || '').substr(1).split('&'); var preference = q.indexOf('vector') >= 0 ? 'vector' : (q.indexOf('raster') >= 0 ? 'raster' : (mapboxgl.supported() ? 'vector' : 'raster')); if (preference == 'vector') { mapboxgl.setRTLTextPlugin('https://tileserver.example.com/mapbox-gl-rtl-text.js'); var map = new mapboxgl.Map({ container: 'map', style: 'https://www.example.com/style.json', zoom: 5.43, center: [-3.9, 54.5], hash: true }); map.addControl(new mapboxgl.NavigationControl()); } else { var map = L.mapbox.map('map', 'https://tileserver.example.com/styles/basic-preview.json', { zoomControl: false }); new L.Control.Zoom({ position: 'topright' }).addTo(map); setTimeout(function() { new L.Hash(map); }, 0); } </script> </body> </html>
Save and close the file. Here’s how the map looks like: https://www.linuxbabe.com/maps/maptiler-3d.html
You might also want to check out other tools provided by OpenMapTiles on Github.
Step 14: Create Custom Styles
You can use maputnik to edit map styles. Make sure it conforms to OpenMapTiles schema.
You need to know the TileServer GL endpoints when editing your style JSON file.
- TileJSON URL:
https://tileserver.example.com/data/v3.json - Tiles URL:
https://tileserver.example.com/data/v3/{z}/{x}/{y}.pbf - Glyphs:
https://tileserver.example.com/font-family/{fontstack}/{range}.pbf
There are two ways to specify Tile URL in the style JSON file.
TileJSON URL:
"sources": {
"openmaptiles": {
"type": "vector",
"url": "https://tileserver.linuxbabe.com/data/v3.json"
}
},
Tiles URL:
"sources": {
"openmaptiles": {
"type": "vector",
"tiles": ["https://tileserver.linuxbabe.com/data/v3/{z}/{x}/{y}.pbf"],
"attribution": "© OpenStreetMap contributors",
"minzoom": 0,
"maxzoom": 14
}
},
I recommend using the second method: Tiles URL. Using the TILEJSON URL might cause the NetworkError when attempting to fetch resource error in Firefox.
Note: When using the Tiles URL, you must set maxzoom to 14 like in the above code.
Step 15: Enable HTTP2
To improve map loading performance, you can enable HTTP2 protocol.
Nginx
Simply open the virtual host file and change
listen 443 ssl;
to
listen 443 ssl http2;
Save and close the file. Then reload Nginx.
sudo systemctl reload nginx
Apache
First, you need to enable the HTTP2 module.
sudo a2enmod http2
Then open the SSL virtual host file.
sudo nano /etc/apache2/sites-enabled/tileserver_site-le-ssl.conf
Put the following directive after the opening <VirtualHost *:443> tag.
Protocols h2 http/1.1
Save and close the file. Then restart Apache for the changes to take effect.
sudo systemctl restart apache2
Display Map with MapLibre GL
Maplibre GL is an open-source fork of Mapbox GL. Demo: https://www.linuxbabe.com/maps/maplibre-streets.html
HTML code:
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<title>vector tile map made with Tegola and Maplibre</title>
<meta name="viewport" content="initial-scale=1,maximum-scale=1,user-scalable=no">
<script src="https://unpkg.com/[email protected]/dist/maplibre-gl.js"></script>
<link href="https://unpkg.com/[email protected]/dist/maplibre-gl.css" rel="stylesheet" />
<style>
body { margin: 0; padding: 0; }
#map { position: absolute; top: 0; bottom: 0; width: 100%; }
</style>
</head>
<body>
<div id="map"></div>
<script>
var map = new maplibregl.Map({
container: 'map',
style:
'https://www.linuxbabe.com/maps/mapbox-street-style.json',
zoom: 5.43,
center: [-3.9, 54.5]
});
map.on('load', function () {
// Add a new vector tile source with ID 'linuxbabe'.
map.addSource('linuxbabe', {
'type': 'vector',
'tiles': [
'https://tileserver.linuxbabe.com/data/v3/{z}/{x}/{y}.pbf'
],
'minzoom': 6,
'maxzoom': 14
});
map.addLayer(
{
'id': 'default', // Layer ID
'type': 'line',
'source': 'linuxbabe', // ID of the tile source created above
// Source has several layers. We visualize the one with name 'sequence'.
'source-layer': 'sequence',
'layout': {
'line-cap': 'round',
'line-join': 'round'
},
'paint': {
'line-opacity': 0.6,
'line-color': 'rgb(53, 175, 109)',
'line-width': 2
}
},
);
});
map.addControl(new maplibregl.NavigationControl());
//set max zoom level (0-24)
map.setMaxZoom(19);
</script>
</body>
</html>
Integrate Maplibre with Nominatim Geocoder
If you want to provide address search functionality, you can set up a Nominatim geocoding server and integrate it with your map via the Maplibre GL Geocoder plugin.
Add the following lines in HTML header.
<script src="https://unpkg.com/@maplibre/[email protected]/dist/maplibre-gl-geocoder.min.js"></script> <link rel="stylesheet" href="https://unpkg.com/@maplibre/[email protected]/dist/maplibre-gl-geocoder.css" type="text/css" />
Then add the following code in the HTML body, above the </script> tag. Replace nominatim.example.com/nominatim with the URL of your own Nonimatim server.
var geocoder_api = {
forwardGeocode: async (config) => {
const features = [];
try {
let request =
'https://nominatim.example.com/nominatim/search?q=' +
config.query +
'&format=geojson&polygon_geojson=1&addressdetails=1';
const response = await fetch(request);
const geojson = await response.json();
for (let feature of geojson.features) {
let center = [
feature.bbox[0] +
(feature.bbox[2] - feature.bbox[0]) / 2,
feature.bbox[1] +
(feature.bbox[3] - feature.bbox[1]) / 2
];
let point = {
type: 'Feature',
geometry: {
type: 'Point',
coordinates: center
},
place_name: feature.properties.display_name,
properties: feature.properties,
text: feature.properties.display_name,
place_type: ['place'],
center: center
};
features.push(point);
}
} catch (e) {
console.error(`Failed to forwardGeocode with error: ${e}`);
}
return {
features: features
};
}
};
map.addControl(
new MaplibreGeocoder(geocoder_api, {
maplibregl: maplibregl
})
);
Display Map with Leaflet
By default, TileServer GL uses Mapbox GL to display maps. Some folks use Leaflet for rendering raster tile maps, but Leaflet doesn’t support vector tiles natively. You need to use the Maplibre GL leaflet plugin. Example map: https://www.linuxbabe.com/maps/leaflet-vector.html
Here’s the code. We need to load the Leaflet and Maplibre GL library in the HTML header, then use Mapbox GL Leaflet to render the map in the HTML body.
<html>
<head>
<title>Vector map loaded by Leaflet</title>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<style>
html, body, #map {
width: 100%;
height: 100%;
margin: 0;
}
</style>
<!-- Leaflet -->
<link rel="stylesheet" href="https://unpkg.com/[email protected]/dist/leaflet.css" />
<script src="https://unpkg.com/[email protected]/dist/leaflet.js"></script>
<!-- Maplibre GL -->
<script src="https://unpkg.com/[email protected]/dist/maplibre-gl.js"></script>
<link href="https://unpkg.com/[email protected]/dist/maplibre-gl.css" rel="stylesheet" />
</head>
<body>
<div id="map"></div>
<script src="https://www.linuxbabe.com/maps/leaflet-maplibre-gl.js"></script>
<script>
var map = L.map('map').setView([54.5, -3.9], 6.4);
var gl = L.maplibreGL({
style: 'https://www.linuxbabe.com/maps/mapbox-street-style.json'
}).addTo(map);
</script>
</body>
</html>
It’s compatible with the Nomination Geocoder.
Backend Performance Tuning
You use Varnish or MapProxy on the server to cache the tiles, so it will be faster to deliver tiles to clients.
Troubleshooting PostgreSQL Error
Check PostgreSQL status:
sudo systemctl status [email protected]
If you encounter the following error.
FATAL: could not map anonymous shared memory: Cannot allocate memory
It means you allocated too much RAM to PostgreSQL shared_buffers. Edit PostgreSQL config file.
sudo nano /etc/postgresql/15/main/postgresql.conf
Set shared_buffers to a smaller value like 5GB.
shared_buffers = 5GB
Save and close the file. Then restart PostgreSQL.
sudo systemctl restart [email protected]
How to Upgrade PostgreSQL Database Server
When a new version of PostgreSQL comes out, you can upgrade to take advantage of performance improvements. Simply run the following command, and the apt package manager will install the latest version of PostgreSQL from the apt.postgresql.org repository.
sudo apt update; sudo apt upgrade -y
After that, you also need to upgrade existing PostgreSQL clusters. Pro Tip: You should open a GNU Screen session to prevent connection drop because the upgrade will take some time to finish.
screen
Then list current clusters.
sudo pg_lsclusters
Sample output:
Ver Cluster Port Status Owner Data directory Log file
14 main 5432 online postgres /var/lib/postgresql/14/main /var/log/postgresql/postgresql-14-main.log
15 main 5433 online postgres /var/lib/postgresql/15/main /var/log/postgresql/postgresql-15-main.log
As you can see, PostgreSQL 14 is using the default 5432 port. PostgreSQL 15 is using port 5433. Stop PostgreSQL server.
sudo systemctl stop postgresql
Let’s check the cluster status again with: sudo pg_lsclusters. They are all down.
Ver Cluster Port Status Owner Data directory Log file 14 main 5432 down postgres /var/lib/postgresql/14/main /var/log/postgresql/postgresql-14-main.log 15 main 5433 down postgres /var/lib/postgresql/15/main /var/log/postgresql/postgresql-15-main.log
Rename the cluster name of PostgreSQL 15 from main to latest_stable.
sudo pg_renamecluster 15 main latest_stable
Then we need to shrink the PostgreSQL shared_buffer.
sudo nano /etc/postgresql/14/main/postgresql.conf
Decrease the value to prevent out-of-memory problem during upgrade. For example, I reduce the value from 15GB to 5GB.
shared_buffers = 5GB
Save and close the file.
Next, upgrade the old PostgreSQL 14 cluster.
sudo pg_upgradecluster 14 main
Start PostgreSQL server.
sudo systemctl start postgresql
Now check if your application is working. If it’s working well, then drop the old cluster.
sudo pg_dropcluster --stop 14 main
Check cluster status again:
sudo pg_lsclusters
If the upgrade is successful, you can change the shared_buffer back to the original value in the /etc/postgresql/15/main/postgresql.conf file. Don’t forget to restart PostgreSQL.
If the upgrade procedure can’t stop PostgreSQL,
pg_ctl: server does not shut down Error: Could not stop target cluster
then you need to run the following command to stop PostgreSQL.
pkill postgres
Then edit PostgreSQL 14 config file.
sudo nano /etc/postgresql/14/main/postgresql.conf
Set the listen port to 5432.
port = 5432
Save and close the file. Then start PostgreSQL 14.
sudo systemctl restart postgresql@14-main
Run the following command to make sure you can log into PostgreSQL console.
sudo -u postgres -i psql
Press Ctrl+D to log out.
Restart TileServer GL.
sudo systemctl restart openstreetmap
Next Step
For more information on TileServer GL, please check its documentation.
You may also want to set up Nominatim geocoding server to provide address lookup functionality.
I hope this article helped you set up TileServer GL Vector Tile Server on Ubuntu 22.04. As always, if you found this post useful, then subscribe to our free newsletter to get more tips and tricks. Take care 🙂
If you encounter the following error:
Then you need to reset osm password.
I get this error AFTER resetting osm password :
Solved
It was a problem of mismatching between PGPASSWORD ‘s in ~/.osmenv and in .env
I get this error whenever I execute this command in step 8: sudo make generate-tiles-pg
I can’t seem to get past this step. Any suggestion?
Server Info:
did you find any solution to this error?
When running ‘sudo make import-wikidata’, I meet this error. Could you help me?
“Connection reset by peer” means your network condition is unstable. Is your server located in China? The connection could be blocked by GFW (Great Firewall of China).
Thanks for your suggestion. Yes, my server is located in China, may I use vpn to solve it ?
Yes, you can use VPN to fix this error.
Hello, thank you for your work!
– in which directory?
Typically, you should create an HTML file on another web server to fetch the vector tiles.
Your website is the entry point for visitors to load the map in the web browser. That’s is to say, your visitor should load the map via
not
so you should upload the
style.jsonfile to the web server that hosts your website.If your website uses
/var/www/html/as web root directory, then you can put thestyle.jsonfile under/var/www/html/and also create the HTML file under this directory.The web root directory could be at another place. You need to check your web server configuration file (such as /etc/nginx/conf.d/website.conf) to find out.
Of course, if your website is hosted on the tile server, then you should create the HTML file on the tile server, but you still need to know the web root directory of your website.
I’m getting this error in the Step 8 : … gave up waiting for Postgres: PGHOST=172.17.0.1 PGDATABASE=openstreetmap PGUSER=osm PGPORT=5432 pg_isready
ERROR: 1
Step 8: Import the Map Data to PostgreSQL :
root@vmi1046457:~/openmaptiles# sudo make import-data Starting postgres docker compose target using default image (no recreate if exists) Creating openmaptiles_postgres_1 ... done Wait for PostgreSQL to start... docker-compose run --rm --user=0:0 openmaptiles-tools pgwait Creating openmaptiles_openmaptiles-tools_run ... done ... gave up waiting for Postgres: PGHOST=172.17.0.1 PGDATABASE=openstreetmap PGUSER=osm PGPORT=5432 pg_isready ERROR: 1 make: *** [Makefile:331: start-db] Error 1 root@vmi1046457:~/openmaptiles# root@vmi1046457:~/openmaptiles# sudo systemctl start postgresql root@vmi1046457:~/openmaptiles# systemctl status postgresql ● postgresql.service - PostgreSQL RDBMS Loaded: loaded (/lib/systemd/system/postgresql.service; enabled; vendor preset: enabled) Active: active (exited) since Tue 2022-10-18 13:33:54 CEST; 53min ago Process: 50104 ExecStart=/bin/true (code=exited, status=0/SUCCESS) Main PID: 50104 (code=exited, status=0/SUCCESS) CPU: 2ms Oct 18 13:33:54 vmi1046457.contaboserver.net systemd[1]: Starting PostgreSQL RDBMS... Oct 18 13:33:54 vmi1046457.contaboserver.net systemd[1]: Finished PostgreSQL RDBMS. root@vmi1046457:~/openmaptiles# sudo make import-data Starting postgres docker compose target using default image (no recreate if exists) Wait for PostgreSQL to start... docker-compose run --rm --user=0:0 openmaptiles-tools pgwait Creating openmaptiles_openmaptiles-tools_run ... done ... gave up waiting for Postgres: PGHOST=172.17.0.1 PGDATABASE=openstreetmap PGUSER=osm PGPORT=5432 pg_isready ERROR: 1 make: *** [Makefile:331: start-db] Error 1What am I doing wrongly? How to make it work?
You didn’t follow step 6 to make PostgreSQL listen on 172.17.0.1 and allow connection to this IP address?
I had to remove all docker images and containers.
Idk why, when i removed only the container didnt work… also maybe remove openmaptiles database and recreate. And check if there is not some error on the database when connecting it as a osm user.
But the above commands helped me!
I follow all steps but still meet this error, could you help me? Thanks a lot.
First, make sure PostgreSQL is listening on 172.17.0.1. List the PostgreSQL listening addresses:
You should see that PostgreSQL listens on both 127.0.0.1 and 172.17.0.1.
LISTEN 0 244 172.17.0.1:5432 0.0.0.0:* users:(("postgres",pid=19767,fd=6)) LISTEN 0 244 127.0.0.1:5432 0.0.0.0:* users:(("postgres",pid=19767,fd=5))Then configure UFW to allow connection from 172.17.0.0/24 and 172.18.0.0/24.
Run the import command again.
If you encounter the following error, it means you need to upgrade PostgreSQL from version 14 to version 15. Instructions are available at the end of this article.
hi, Installing fails for me at this step:
psql -d openstreetmap -c “CREATE EXTENSION osml10n CASCADE;”
ERROR: extension “osml10n” is not available
DETAIL: Could not open extension control file “/usr/share/postgresql/16/extension/osml10n.control”: No such file or directory.
HINT: The extension must first be installed on the system where PostgreSQL is running.
I tried to follow your steps relating to upgrade from 14 to 15, but going from 15 to 16.
Here are the clusters:
Ver Cluster Port Status Owner Data directory Log file
15 main 5434 down postgres /var/lib/postgresql/15/main /var/log/postgresql/postgresql-15-main.log
16 latest_stable 5433 online postgres /var/lib/postgresql/16/latest_stable /var/log/postgresql/postgresql-16-latest_stable.log
16 main 5432 online postgres /var/lib/postgresql/16/main /var/log/postgresql/postgresql-16-main.log
What should I do?
How long can reasonably be the phase `import-sql` for the `whole-map` with a `Cloud VPS` (Contabo) with `10 vCPU Cores` + `60 GB RAM` + `2 TB SSD`?
Till now, no error messages. But this sub-phase, the `import.sql` has already been running for two days
Europe map with same contabo config. It finished after 16 hours.
I have the same problem. The query “CREATE MATERIALIZED VIEW osm_building_block_gen1_dup AS …” is taking forever. Can I skip this somehow? How did you fix it?
Can we skip this step ?
Because I tried 5 times, because I got every time :
Error: Copy operation timed out at Timeout. (/usr/lib/node_modules/@mapbox/tilelive/lib/tilelive.js:392:22) at listOnTimeout (internal/timers.js:557:17) at processTimers (internal/timers.js:500:7) ERROR: 1 make: *** [Makefile:461: generate-tiles-pg] Error 1could you find any solution to this problem?
I have successfully built my own vector tile server. I want to add some elements in the page but I cannot find where its HTML is. Could you help me?
Click the “vector” or “raster” link and it will show you an example map. Then you can prepend
view-source:to the begining of the URL to view the HTML source code.There’s no need to alter the HTML code of the example map on the tileserver. You just copy the HTML code, change it as you see fit and upload it to your web server where it will show your map.
I clicked the “Viewer” and copied its HTML code. Then I changed the address of js or css in the code such as “http://47.122.xx.xx/maplibre-gl.js” but I found it can’t show the map.
Check your browser console to find out what went wrong.
Hey,
Great guide! I’ve got this sorted successfully, but I wanted to know how we could go about performing updates to the map data? I’ve seen a few updates using renderd, but we haven’t used that so far here.
I’m pretty sure I could just re-run the whole of step 8 (could I?) – but I’m sure there are faster methods?
What would you recommend?
Thanks!
Renderd doesn’t apply here, as it’s used for raster tile server.
All you need to do is follow step 8 again to download new map data and generate a new
.mbtilesfile, then restart the tile server.Step 9:
Output: This site can’t be reached
I didn’t enable `ufw` and set any rules because every time I enable ufw into Contabo Server, I cannot connect to server anymore via SSH
So…since there is no firewall set at the moment…. What might be the root cause of “This site can’t be reached” ?
This is the current situation:
And
If I put in the browser address bar: http://194.x.x.x/ -> I correctly get:
Apache2 Default Pagebut
If If put: http://194.x.x.x:8080/ -> I get :
This site can’t be reached194.x.x.x refused to connect. Try: Checking the connection Checking the proxy and the firewallHow to make it work?
Looking forward to your kind help
I changed from `8080:80` to `8080:8080` :
root@vmi1046457:~# sudo docker run --restart=always -it -d -v /root/openmaptiles/data:/data -p 8080:8080 maptiler/tileserver-gl 3219b5dcd97a182b101a6fa06f05eefb13e7f0b573b58b3ce4ffe9394d4469ee root@vmi1046457:~# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 3219b5dcd97a maptiler/tileserver-gl "/usr/src/app/docker…" 9 seconds ago Up 3 seconds (health: starting) 0.0.0.0:8080->8080/tcp adoring_shamir root@vmi1046457:~# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 3219b5dcd97a maptiler/tileserver-gl "/usr/src/app/docker…" 35 seconds ago Up 28 seconds (health: starting) 0.0.0.0:8080->8080/tcp adoring_shamirand now it works fine
I suggest you, Xiao, to update `Step 9` .
The correct way is :
Hi !!
After a very long processing, the execution of the tiles generation finally ended:
root@vmi1046457:~/openmaptiles# sudo make generate-tiles-pg area=planet-latest docker-compose run --rm --user=0:0 openmaptiles-tools bash -c 'style-tools recompose openmaptiles.yaml build/style/style.json \ style/style-header.json && \ spritezero build/style/sprite /style/icons && spritezero --retina build/style/sprite@2x /style/icons' Creating openmaptiles_openmaptiles-tools_run ... done Starting postgres docker compose target using default image (no recreate if exists) Wait for PostgreSQL to start... docker-compose run --rm --user=0:0 openmaptiles-tools pgwait Creating openmaptiles_openmaptiles-tools_run ... done Generating tiles into data/tiles.mbtiles (will delete if already exists) using PostGIS ST_MVT()... docker-compose run -T --rm --user=0:0 openmaptiles-tools generate-tiles Creating openmaptiles_openmaptiles-tools_run ... done water 0 14 id, class, intermittent, brunnel Water polygons representing oceans and l… waterway 0 14 name, name_en, name_de, name_int, class, brunnel, intermittent, name:* (am,ar,az,be,bg,br,bs…) OpenStreetMap [waterways](https://wiki.o… landcover 0 14 class, subclass Landcover is used to describe the physic… landuse 0 14 class Landuse is used to describe use of land … mountain_peak 0 14 osm_id, name, name_en, name_de, name_int, class, ele, ele_ft, customary_ft, rank, name:* (am,ar,az,be,bg,br,bs…) [Natural peaks](http://wiki.openstreetma… park 0 14 class, name, name_en, name_de, name_int, rank, name:* (am,ar,az,be,bg,br,bs…) The park layer contains parks from OpenS… boundary 0 14 admin_level, adm0_l, adm0_r, disputed, disputed_name, claimed_by, maritime Contains administrative boundaries as li… aeroway 0 14 ref, class Aeroway polygons based of OpenStreetMap … transportation 0 14 class, subclass, network, oneway, ramp, brunnel, service, access, toll, expressway, layer, level, indoor, bicycle, foot, horse, mtb_scale, surface **transportation** contains roads, railw… building 0 14 osm_id, render_height, render_min_height, colour, hide_3d All [OSM Buildings](http://wiki.openstre… water_name 0 14 osm_id, name, name_en, name_de, name_int, class, intermittent, name:* (am,ar,az,be,bg,br,bs…) Lake center lines for labelling lake bod… transportation_name 0 14 name, name_en, name_de, name_int, ref, ref_length, network, class, subclass, brunnel, layer, level, indoor, route_1, route_2, route_3, route_4, route_5, route_6, name:* (am,ar,az,be,bg,br,bs…) This is the layer for labelling the high… place 0 14 osm_id, name, name_en, name_de, name_int, class, rank, capital, iso_a2, name:* (am,ar,az,be,bg,br,bs…) The place layer consists out of [countri… housenumber 0 14 housenumber Everything in OpenStreetMap which contai… poi 0 14 osm_id, name, name_en, name_de, name_int, class, subclass, agg_stop, layer, level, indoor, rank, name:* (am,ar,az,be,bg,br,bs…) [Points of interests](http://wiki.openst… aerodrome_label 0 14 id, name, name_en, name_de, name_int, class, iata, icao, ele, ele_ft, name:* (am,ar,az,be,bg,br,bs…) [Aerodrome labels](http://wiki.openstree… Zoom Tile count Found tile ranges ------ ------------ ------------------- 0 1 0,0 x 0,0 1 4 0,0 x 1,1 2 16 0,0 x 3,3 3 64 0,0 x 7,7 4 256 0,0 x 15,15 5 1,024 0,0 x 31,31 6 4,096 0,0 x 63,63 7 16,384 0,0 x 127,127 8 65,536 0,0 x 255,255 9 262,144 0,0 x 511,511 10 1,048,576 0,0 x 1023,1023 11 4,194,304 0,0 x 2047,2047 12 16,777,216 0,0 x 4095,4095 13 67,108,864 0,0 x 8191,8191 14 268,435,456 0,0 x 16383,16383I then restarted the Apache Server :
And renewed the LetsEncrypt Certificate :
Now Clicking “Inspect” I get a blank empty page
In `tileserver-gl.error.log` I get a warning an an error:
In /var/log/apache2/tileserver-gl.access.log :
I do not understand what is the problem
Can you, please, point me in the right direction to spot and solve the issue?
Just an FYI, as found by Raphy above…
As of TileServer-GL v4.2.0 , the default docker port tileserver runs on internally has changed from 80 to 8080.
This means the command to run is now
*note ‘8080:80’ is now ‘8080:8080’ *
Thanks for your FYI 🙂
Hello. Thanks a lot for the tutorial. Everything went perfectly.
You gave an example in step “Step 13 : Use Other Open-Source Map Styles” how to use “MapTiler 3D style.” Could you please tell me how I could use “OSM Bright” but be on the same server? And, in which folder should the different styles that I would like to use be placed. I tried to put the osm-bright folder in “openmaptiles/data” but it doesn’t seem to work.
It’s already almost two days that in `sudo make import-sql` of `Step 8` for `planet-latest` it is stuck here:
I’m using the same Hardware Settings with the same Cloud Infrastructure Provider
What does that mean?
I had to quit for the second time `sudo make import-sql area=planet-latest`: it was stuck at this specific point:
HW used: extra-large VPS from Contabo with :
A 10-core CPU
60 GB RAM
2 TB Intel SSD
What can I do , Xiao, to make it work?
I’m still stuck on the step : sudo make import-sql area=planet-latest
This time I noticed this `warning`/`error`:
docker-compose run --rm --user=0:0 openmaptiles-tools sh -c 'pgwait && import-sql' | \ awk -v s=": WARNING:" '1{print; fflush()} $0~s{print "\n*** WARNING detected, aborting"; exit(1)}' | \ awk '1{print; fflush()} $0~".*ERROR" {txt=$0} END{ if(txt){print "\n*** ERROR detected, aborting:"; print txt; exit(1)} }': root@vmi1046457:~/openmaptiles# root@vmi1046457:~/openmaptiles# sudo make import-sql area=planet-latest docker-compose run --rm --user=0:0 openmaptiles-tools bash -c 'style-tools recompose openmaptiles.yaml build/style/style.json \ style/style-header.json && \ spritezero build/style/sprite /style/icons && spritezero --retina build/style/sprite@2x /style/icons' Creating openmaptiles_openmaptiles-tools_run ... done Starting postgres docker compose target using default image (no recreate if exists) docker-compose run --rm --user=0:0 openmaptiles-tools sh -c 'pgwait && import-sql' | \ awk -v s=": WARNING:" '1{print; fflush()} $0~s{print "\n*** WARNING detected, aborting"; exit(1)}' | \ awk '1{print; fflush()} $0~".*ERROR" {txt=$0} END{ if(txt){print "\n*** ERROR detected, aborting:"; print txt; exit(1)} }' Creating openmaptiles_openmaptiles-tools_run ... done Time: 14.412 ms CREATE FUNCTION Time: 2.649 ms CREATE FUNCTION Time: 1.222 ms CREATE FUNCTION Time: 1.025 ms CREATE FUNCTION Time: 5.570 ms CREATE FUNCTION Time: 8.536 ms CREATE FUNCTION Time: 1.213 ms CREATE FUNCTION Time: 0.974 ms CREATE FUNCTION Time: 14.770 ms psql:/sql/parallel/transportation__transportation_name.sql:208: NOTICE: relation "ne_10m_admin_0_bg_buffer" already exists, skipping CREATE TABLE AS Time: 2.390 ms CREATE FUNCTION Time: 29.095 ms CREATE FUNCTION Time: 3.096 ms DROP TRIGGER Time: 6.977 ms DROP TRIGGER Time: 0.980 ms CREATE VIEW Time: 16.132 ms DROP TRIGGER Time: 9.111 ms psql:/sql/parallel/place.sql:161: NOTICE: schema "place_continent_point" already exists, skipping CREATE SCHEMA Time: 0.517 ms psql:/sql/parallel/place.sql:166: NOTICE: relation "osm_ids" already exists, skipping CREATE TABLE Time: 0.336 ms CREATE FUNCTION Time: 7.725 ms update_osm_continent_point ---------------------------- (1 row) Time: 44.090 ms CREATE FUNCTION Time: 2.490 ms psql:/sql/parallel/place.sql:199: NOTICE: relation "updates" already exists, skipping CREATE TABLE Time: 1.235 ms CREATE FUNCTION Time: 6.577 ms CREATE FUNCTION Time: 2.405 ms CREATE TRIGGER Time: 6.338 ms CREATE TRIGGER Time: 2.358 ms CREATE TRIGGER Time: 8.123 ms DROP TRIGGER Time: 1.214 ms DROP TRIGGER Time: 0.819 ms DROP TRIGGER Time: 6.019 ms psql:/sql/parallel/place.sql:250: NOTICE: schema "place_country" already exists, skipping CREATE SCHEMA Time: 1.079 msStill stuck at this step: sudo make import-sql area=planet-latest
As you can see from the screenshot, this process uses all the 10 cpus available
What can I do in order to speed up the process? I’ve already done all the suggestions in Step 4: Optimize PostgreSQL Server Performance : https://www.linuxbabe.com/linux-server/osm-openstreetmap-tile-server-ubuntu-22-04
have you found a solution ?
Sorry for clogging this great post with my S.O.S. requests.. :
Still stuck at this step: sudo make import-sql area=planet-latest
Would you be so kind in explaining what can I do in order to speed up the process and finish it?
Hi there,
I’ve followed the steps to generate an mbtiles file. However, when it is served by tileserver-gl it is not displaying all details in the map. It is only showing some cities names. Train stations, road names, buildings … are not displayed on the map as showed in the attachement. Do you have an idea about that ?
Step `sudo make generate-tiles-pg area=planet-latest` :
Error getting 14/10743/7421: Error: Connection terminated unexpectedly Error getting 14/10741/7421: Error: Connection terminated unexpectedly Possible callback crash for 13/5370/3706: Error: read ECONNRESET Possible callback crash for 13/5371/3706: Error: read ECONNRESET Possible callback crash for 13/5374/3706: Error: read ECONNRESET Possible callback crash for 14/10741/7415: Error: read ECONNRESET Error getting 14/10751/7413: Error: read ECONNRESET Error getting 14/10749/7415: Error: read ECONNRESET Error getting 14/10745/7413: Error: read ECONNRESET Error getting 14/10743/7421: Error: read ECONNRESET Error getting 14/10741/7421: Error: read ECONNRESET Possible callback crash for 14/10741/7423: Error: read ECONNRESET Possible callback crash for 14/10751/7413: Error: connect ECONNREFUSED 172.17.0.1:5432 Possible callback crash for 14/10749/7415: Error: connect ECONNREFUSED 172.17.0.1:5432 Possible callback crash for 14/10745/7413: Error: connect ECONNREFUSED 172.17.0.1:5432 Possible callback crash for 14/10743/7421: Error: connect ECONNREFUSED 172.17.0.1:5432 Possible callback crash for 14/10741/7421: Error: connect ECONNREFUSED 172.17.0.1:5432 2022-12-25 04-12-23 Tile generation complete! Updating generated tile metadata ... docker-compose run --rm --user=0:0 openmaptiles-tools \ mbtiles-tools meta-generate "data/tiles.mbtiles" openmaptiles.yaml --auto-minmax --show-ranges Creating openmaptiles_openmaptiles-tools_run ... done Connecting to PostgreSQL at 172.17.0.1:5432, db=openstreetmap, user=osm... Unable to connect to Postgres database: [Errno 111] Connect call failed ('172.17.0.1', 5432) Traceback (most recent call last): File "/usr/src/app/mbtiles-tools", line 188, in main() File "/usr/src/app/mbtiles-tools", line 141, in main asyncio.run(Metadata(args[''], args['--show-json'], File "/usr/local/lib/python3.9/asyncio/runners.py", line 44, in run return loop.run_until_complete(main) File "/usr/local/lib/python3.9/asyncio/base_events.py", line 647, in run_until_complete return future.result() File "/usr/src/app/openmaptiles/mbtile_tools.py", line 278, in generate raise err File "/usr/src/app/openmaptiles/mbtile_tools.py", line 265, in generate async with asyncpg.create_pool( File "/usr/local/lib/python3.9/site-packages/asyncpg/pool.py", line 966, in __aenter__ await self._async__init__() File "/usr/local/lib/python3.9/site-packages/asyncpg/pool.py", line 413, in _async__init__ await self._initialize() File "/usr/local/lib/python3.9/site-packages/asyncpg/pool.py", line 441, in _initialize await first_ch.connect() File "/usr/local/lib/python3.9/site-packages/asyncpg/pool.py", line 133, in connect self._con = await self._pool._get_new_connection() File "/usr/local/lib/python3.9/site-packages/asyncpg/pool.py", line 511, in _get_new_connection con = await connection.connect( File "/usr/local/lib/python3.9/site-packages/asyncpg/connection.py", line 2085, in connect return await connect_utils._connect( File "/usr/local/lib/python3.9/site-packages/asyncpg/connect_utils.py", line 895, in _connect raise last_error File "/usr/local/lib/python3.9/site-packages/asyncpg/connect_utils.py", line 881, in _connect return await _connect_addr( File "/usr/local/lib/python3.9/site-packages/asyncpg/connect_utils.py", line 781, in _connect_addr return await __connect_addr(params, timeout, True, *args) File "/usr/local/lib/python3.9/site-packages/asyncpg/connect_utils.py", line 825, in __connect_addr tr, pr = await compat.wait_for(connector, timeout=timeout) File "/usr/local/lib/python3.9/site-packages/asyncpg/compat.py", line 66, in wait_for return await asyncio.wait_for(fut, timeout) File "/usr/local/lib/python3.9/asyncio/tasks.py", line 479, in wait_for return fut.result() File "/usr/local/lib/python3.9/site-packages/asyncpg/connect_utils.py", line 691, in _create_ssl_connection tr, pr = await loop.create_connection( File "/usr/local/lib/python3.9/asyncio/base_events.py", line 1065, in create_connection raise exceptions[0] File "/usr/local/lib/python3.9/asyncio/base_events.py", line 1050, in create_connection sock = await self._connect_sock( File "/usr/local/lib/python3.9/asyncio/base_events.py", line 961, in _connect_sock await self.sock_connect(sock, address) File "/usr/local/lib/python3.9/asyncio/selector_events.py", line 500, in sock_connect return await fut File "/usr/local/lib/python3.9/asyncio/selector_events.py", line 535, in _sock_connect_cb raise OSError(err, f'Connect call failed {address}') ConnectionRefusedError: [Errno 111] Connect call failed ('172.17.0.1', 5432) ERROR: 1 make: *** [Makefile:463: generate-tiles-pg] Error 1 root@vmi1046457:~/openmaptiles#What could be the causes of this `ECONNREFUSED` ?
How to make it work?
Looking forward to your kind help and suggestions.
Seems like PostgreSQL is killed due to OOM (Out of Memory)
I wrote to Contabo Team about my problems with Step `sudo make generate-tiles-pg area=planet-latest` and they answered my as follows:
Since the type of VPS I’m using is the same used by Xiao: I’ve already followed Xiao’s suggestions on Postgresql’s server optimization….. so what can I do to further improve the “optmization” of the postgresql server ?
74 days for processing `sudo make generate-tiles-pg area=planet-latest`?? How is it possible??? How to speed it up? Your expertise Xiao is needed!!!
The timing is increasing…. from the already remarkable 74 days to 133 days left to finish… How to speed everything up?
sudo make generate-tiles-pg area=planet-latest
Thanks for this tutorial! However, Mapnik doesn’t work anymore. Repository on github is also archived. Any thoughts to fix this?
just love your CMS 😉
Found the problem. It looks like they have restricted downloading https://www.nominatim.org/data/country_grid.sql.gz using curl, so the make script fails. The file isn’t downloaded, so nothing is fed to gunzip, yielding the “gzip: stdin: not in gzip format” error.
Solution: you can download the file in a browser and upload it with SFTP to your server. Put it in the mapnik-german-l10n folder and gunzip. The make script checks the directory for a local version before attempting to download it with curl. So just run “make deb” afterwards and you’re fine.
Hi Eveline,
You said “put it in the mapnik-german-l10n folder and gunzip”. Does that mean put country_grid.sql.gz file in mapnik-german-l10n folder?
Thanks for the solution for this issue.
Hm… I put the country_grid.sql.gz file in mapnik-german-l10n folder after downloading it from a browser and gunzip the gz file, which depressed as sql file. Afterwards I type the command “make deb” but still shows the same error message “gzip: stdin: not in gzip format”.
If anyone know me how to fix this error, let me know. Thanks
Hi Shrush,
You’re right. Just put it in the mapnik-german-l10n folder and unzip. Forgot to mention I renamed the file, because the build script checks for country_osm_grid.sql
By default it will unzip to country_grid.sql, if you don’t rename it you’ll be stuck with the same error message.
Thank a lot, I had the same problem on Ubuntu 22.10, and your solution saved my day.
Unfortunately only for bumping in the next one. When creating extensions, postgis and hstore had been no problem.
However, gzip and osm110n look for /usr/share/postgresql/14/extension/gzip.control which does not exist, only in /usr/share/postgresql/15/extension/gzip.control (postgresql v15 installed).
-> ERROR: could not open extension control file “/usr/share/postgresql/14/extension/gzip.control”: Datei oder Verzeichnis nicht gefunden
Copying gzip.control and gzi–1.0.sql to the /14/extension folder is of no avail.
-> ERROR: could not access file “$libdir/gzip”: Datei oder Verzeichnis nicht gefunden
Anyone knowing how to deal with that?
Looks like you have installed PostgreSQL 14 before. You installed PostgreSQL 15 but didn’t configure it to use the new version. You should upgrade it by following the instructions in the “How to Upgrade PostgreSQL Database Server” section.
I have the problem that https is not working, browser console says that mixed content is not allowed.
I am using apache and have a bought certificate, https://tiles2.myserver.net works fine. But:
If i read https://tiles2.myserver.net/styles/basic-preview.json
it replies with a json file with http-Addresses, not with https-Addresses. for example:
the tiles-Address is: “http://tiles2.myserver.net/styles/basic-preview/{z}/{x}/{y}.png”
It seems that the Server in the docker container is sending this wrong addresses.
How could i fix that?
The Solution was to add 2 lines into the apache-conf-file of the domain:
I get this error on sql import :
*** ERROR detected, aborting:
psql:/sql/parallel/transportation__transportation_name.sql:447: ERROR: only superuser can define a leakproof function
make: *** [Makefile:453: import-sql] Error 1
Any idea what to do here?
I have the same problem while starting command:
with message:
I have found a solution: It seems, that the file transportation__transportation_name.sql (out of github.com/openmaptiles/openmaptiles.git)has changed significantly in the last 3 months (and other files too). I have had a Version from December 2022 from github which is working fine. In the current Version there are severeal commands with “IMMUTABLE LEAKPROOF PARALLEL SAFE;” which are not working as osm user in postgresql.
Maybe something like this will work:
git log –since=2022-12-01 –until=2022-12-31
To find one of the commit ids you like there then do a git checkout
Does anyone know how to use the current version of openmaptiles as a superuser in postgresql?
There is no git commit in december 2022 as I can see: https://github.com/openmaptiles/openmaptiles/commits?after=91dd853a89492e81987a717e3fcfe1b50836fc94+34
Do I need to clean up the tables before I try ‘sudo make import-sql’ again? How?
thank you 🙂
I had the same problem and as a temporary solution I changed the permissions of the user ‘osm’ to be superuser. After finishing the ‘import-sql’ process, the superuser permission can be revoked.
Example: https://chartio.com/resources/tutorials/how-to-change-a-user-to-superuser-in-postgresql/
Ok … after hours of tears I have it:
1. after step
do:
2. after step
and
do:
good luck
there is a typo:
However, this is the status:
LISTEN 0 244 172.17.0.1:5432 0.0.0.0:* users:(("postgres",pid=7470,fd=6)) LISTEN 0 244 127.0.0.1:5432 0.0.0.0:* users:(("postgres",pid=7470,fd=5)) LISTEN 0 244 127.0.0.1:5433 0.0.0.0:* users:(("postgres",pid=7471,fd=5))I am not in China (East Coast of the US)
And the ports are enabled in the firewall.
Also using the latest version of postgres15.
after this command sudo ss -lnpt | grep postgres I cannot see ip address I added
Hi, the tutorial is fine! I loaded an European country map and now it is completed step 9 and more and works fine. I can see the map but all the html example where I modified with my domain doesn’t work, it is a html empty page or with + and – in top right.
I can’t see the map in that files neither with leaflet example or all the others example. I need some useful details. Many thanks!
Hello, first thanks for your tutorial
I stuck at step 2 last part
it seems osml10n git page not maintain anymore
I installed osml10n from newer page but i cant add osml10n extension to PostgreSQL 16
any help?
thanks
I had the same issue. I stayed with PostgreSQL 15, and it works well.
how can I stick with postgresql 15, it downloads extension for 16
I removed all the Postgres first:
Then I only installed Postgres 15:
I was also having trouble compiling mapnik-german-l10n. User makemap’s instructions were useful but didn’t get me all the way there because, I think, postgresql 17 is still available in the various apt packages.
Purge postgresql.
Only install postgresql 15 (but no generic package names, like postgresql):
Then I modified the debian/control.in to explicitly make it use and output v15:
Source: osml10n Section: misc Priority: extra Maintainer: Sven Geggus Build-Depends: debhelper (>= 8.0.0), libicu-dev, postgresql-server-dev-15, curl, libkakasi2-dev, libutf8proc-dev, pandoc Standards-Version: 3.9.2 Package: postgresql-15-osml10n Architecture: any # "postgis" is used as some kind of meta package here, because it # contains all dependencies for whatever version of PostGIS is available # on the Debian or Ubuntu intallation this code is going to get built Depends: ${shlibs:Depends}, ${misc:Depends}, postgresql-15, icu-devtools Recommends: postgresql-plpython3-15, python3-matplotlib, python3-pip, postgis Description: l10n package for generation of OSM based maps from PostGIS localisation package for generation of OSM based maps from PostGIS. This is currently used in german mapnik style but can be used in a renderer independant way because ist is based on PostgreSQL stored procedures. This includes a transliteration function based on ICU any-latin, a kanji transcription library based on libkakasi and a couple of pl/pgSQL funktions.And you can modify debian/rules as well for good measure to override and make it version 15:
Then run the compile stuff again:
Hopefully this helps someone.
when I try to create .mbtiles file with running: sudo make generate-tiles-pg
I got this error:
Error: Copy operation timed out
at Timeout. (/usr/lib/node_modules/@mapbox/tilelive/lib/tilelive.js:392:22)
at listOnTimeout (internal/timers.js:557:17)
at processTimers (internal/timers.js:500:7)
ERROR: 1
make: *** [Makefile:461: generate-tiles-pg] Error 1
How can I solve this problem? I tried twice and this is the result
Thanks for the great tutorial, it really complimented the the official docs nicely.
However I had some issues following this on the lastest Ubuntu 22.04.3 LTS , due to a couple of things:
ubuntu now seems to install postgres16 as well as v15, this causes the gzip extension to pick up on v16 and install there, so adding the extension fails in step 3. To fix this you need to set the path so that the V15 version of pg_config gets used for the extension build in step 2:
cd pgsql-gzip/ sudo apt install build-essential zlib1g-dev postgresql-server-dev-all pkg-config export PATH=/usr/lib/postgresql/15/bin:$PATH make sudo make installAlso I couldn’t get the localisation extension to build at all due to some dependencies seemingly missing upstream, so I downloaded the prebuilt deb from
https://github.com/kirimaru-jp/mapnik-german-l10n/releases/download/v2.5.10/postgresql-15-osml10n_2.5.10_amd64.deb
and that seemed to install fine.
Hi ,
thank you for the great post , btw i got stuck on step make import-sql it’s throw error column source_ids not exist on relation osm_transportation_merge_linestring_gen_z11 , all step before this was successfully executed , not sure where the error is , i have tried create manually the column using pgadmin but it got error ” column source_ids is type bigint but expression is of type bigint[] ” , hopefully you can give me advice for the error
thank you
regards
This post might help you:
https://github.com/openmaptiles/openmaptiles/issues/1556#issuecomment-1741928857
hi
thank you , now i passed the import sql step , it seems they commit a fix for column_ids issue
hi all 🙂
sorry for ask again , so we are trying deploy with whole planet pbf , when on step generate mbtiles file from postgres the counter time said we need about thousand days to finish the generate process XD , i think it’s not normal ,
btw we use vm with 30 core amd opteron processor and 220 GB ram , also i have tried tune the .env file with increase value of max_paralel_copy and copy_concurrency , and i see the process use all resources but i didn’t any significant progress on counter estimating finish time , it still show we need thousands days to finish ,
maybe some one can give us an advice 🙂
thank you
I previously built the OSM|OSRM for the planet with mod_tile, renderd, etc… https://www.linuxbabe.com/ubuntu/openstreetmap-tile-server-ubuntu-20-04-osm
I run a service that makes calls for tiles as x/y/z.png I was looking to upgrade and go vector with this new solution but I can’t seem to figure out how to make the same calls. Is this possible with Vector tiles? I’ve been searching around and started playing with these api endpoints but I can’t seem to make any of them work where I can make a call for the png https://tileserver.readthedocs.io/en/latest/endpoints.html#rendered-tiles
Any thoughts on this?
I couldn’t remove my comment, but I got it working. Thanks for the tutorials. Also, anyone trying to get it going on PostgreSQL 16, you have to manuall include the varatt.h in the postgres.h file
#include “varatt.h”
Thanks Paul,
You beat me to it. Your solution will work but I suggest the following instead:
# Add: #include “varatt.h” under #include in both programs to compile properly with PostgreSQL v16:
/home/osm/mapnik-german-l10n/icutranslit/osml10n_translit.cpp
/home/osm/mapnik-german-l10n/kanjitranscript/kanjitranscript.c
I prefer the above changes, it does not overwrite the vendors software, though I do not know why they moved some of its definitions to server/varatt.h.
Great amendment, this is a much better solution. I was under a deadline and I just did what I could to get things rolling.
This is 2 (and last) part of my short commands
> sudo docker run --restart=always -it -d -v /home/username/openmaptiles/data:/data -p 8080:8080 maptiler/tileserver-gl > cd ~ > git clone https://github.com/openmaptiles/fonts.git ### node js use this (latest version) LTS > curl -fsSL https://deb.nodesource.com/setup_20.x | sudo -E bash - &&\ sudo apt-get install -y nodejs > cd fonts/ > vim package.json # EDIT "fontnik": "0.7.2" > npm install > node ./generate.js > sudo mkdir -p /var/www/ > sudo mv _output/ /var/www/font-family/ > cd ~ > git clone https://github.com/klokantech/klokantech-gl-fonts.git > sudo mv klokantech-gl-fonts/* /var/www/font-family/ > cd /var/www/font-family/ > sudo cp -r KlokanTech\ Noto\ Sans\ Regular/ Klokantech\ Noto\ Sans\ Regular/ > sudo cp -r KlokanTech\ Noto\ Sans\ Bold/ Klokantech\ Noto\ Sans\ Bold > sudo cp -r KlokanTech\ Noto\ Sans\ CJK\ Bold/ Klokantech\ Noto\ Sans\ CJK\ Bold/ > sudo cp -r KlokanTech\ Noto\ Sans\ CJK\ Regular/ Klokantech\ Noto\ Sans\ CJK\ Regular/ > sudo cp -r KlokanTech\ Noto\ Sans\ Italic/ Klokantech\ Noto\ Sans\ Italic/ > sudo vim /etc/nginx/sites-enabled/tileserver.example.it.conf # EDIT - suppose you've created your virtual host, created ssl certificate and enabled http2 server { server_name tileserver.example.it; listen YOUR-IP; root /home/your-user/public_html; index index.php index.htm index.html; access_log /var/log/virtualmin/tileserver.example.it_access_log; error_log /var/log/virtualmin/tileserver.example.it_error_log; fastcgi_param GATEWAY_INTERFACE CGI/1.1; fastcgi_param SERVER_SOFTWARE nginx; fastcgi_param QUERY_STRING $query_string; fastcgi_param REQUEST_METHOD $request_method; fastcgi_param CONTENT_TYPE $content_type; fastcgi_param CONTENT_LENGTH $content_length; fastcgi_param SCRIPT_FILENAME "/home/your-user/public_html$fastcgi_script_name"; fastcgi_param SCRIPT_NAME $fastcgi_script_name; fastcgi_param REQUEST_URI $request_uri; fastcgi_param DOCUMENT_URI $document_uri; fastcgi_param DOCUMENT_ROOT /home/your-user/public_html; fastcgi_param SERVER_PROTOCOL $server_protocol; fastcgi_param REMOTE_ADDR $remote_addr; fastcgi_param REMOTE_PORT $remote_port; fastcgi_param SERVER_ADDR $server_addr; fastcgi_param SERVER_PORT $server_port; fastcgi_param SERVER_NAME $server_name; fastcgi_param PATH_INFO $fastcgi_path_info; fastcgi_param HTTPS $https; location /font-family/ { alias /var/www/font-family/; } location / { proxy_pass http://127.0.0.1:8080; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; proxy_set_header X-Forwarded-Protocol $scheme; proxy_set_header X-Forwarded-Host $http_host; proxy_hide_header Access-Control-Allow-Origin; } add_header 'Vary' "Origin"; add_header "Access-Control-Allow-Origin" "$http_origin"; location ^~ /.well-known/ { try_files $uri /; } location ~ "\.php(/|$)" { try_files $uri $fastcgi_script_name =404; default_type application/x-httpd-php; fastcgi_pass unix:/run/php/1708801756124837.sock; } fastcgi_split_path_info "^(.+\.php)(/.+)$"; location /cgi-bin/ { gzip off; root /home/your-user/cgi-bin; fastcgi_pass unix:/var/fcgiwrap/1708801756124837.sock/socket; fastcgi_param SCRIPT_FILENAME "/home/your-user$fastcgi_script_name"; fastcgi_param GATEWAY_INTERFACE CGI/1.1; fastcgi_param SERVER_SOFTWARE nginx; fastcgi_param QUERY_STRING $query_string; fastcgi_param REQUEST_METHOD $request_method; fastcgi_param CONTENT_TYPE $content_type; fastcgi_param CONTENT_LENGTH $content_length; fastcgi_param SCRIPT_NAME $fastcgi_script_name; fastcgi_param REQUEST_URI $request_uri; fastcgi_param DOCUMENT_URI $document_uri; fastcgi_param DOCUMENT_ROOT /home/your-user/public_html; fastcgi_param SERVER_PROTOCOL $server_protocol; fastcgi_param REMOTE_ADDR $remote_addr; fastcgi_param REMOTE_PORT $remote_port; fastcgi_param SERVER_ADDR $server_addr; fastcgi_param SERVER_PORT $server_port; fastcgi_param SERVER_NAME $server_name; fastcgi_param PATH_INFO $fastcgi_path_info; fastcgi_param HTTPS $https; } listen YOUR-IP:443 ssl http2; ssl_certificate /home/your-user/ssl.combined; ssl_certificate_key /home/your-user/ssl.key; if ($scheme = http) { rewrite "^/(?!.well-known)(.*)$" "https://$host/$1" break; } } > sudo systemctl reload nginx That's all;My database is hosted outside of the ubuntu VM. I updated .env and .osmenv with my database connection, and run “sudo make import-osm”, I got the error (attachment).
I even hardcoded the PGHOST variable in the Makefile, and specify PGHOST environment variable in openmaptiles-tools (Docker-compose).
Still no luck of getting it solved.
Let me know how many beer you need to make you awake in the night to help me on this 🙂
I have a question regarding your step – ‘Install OSM Localization Extension (osml10n)’. I want to use newer ‘osml10n’ as earlier ‘mapnik-german-l10n’ is depreciated.
Earlier:
https://github.com/giggls/mapnik-german-l10n
Newer:
https://github.com/giggls/osml10n
So, what are the changes do I need to make newer osml10n to work?
after retry again and again the command runs
sudo make generate-tiles-pg
2 o 3 days after the generation tile was complete, but at Updating generate tile metada i got this
……….
[1d 23h 54m 42.8s] 100.0000% 15.50m/15.50m @ 93/s | ✓ 15.50m □ 0 | 0.0s left
2024-10-09 17-34-26 Tile generation complete!
Updating generated tile metadata …
docker-compose run –rm –user=0:0 openmaptiles-tools \
mbtiles-tools meta-generate “data/tiles.mbtiles” openmaptiles.yaml –auto-minmax –show-ranges
Creating openmaptiles_openmaptiles-tools_run … done
Connecting to PostgreSQL at 172.17.0.1:5432, db=openstreetmap, user=osm…
Traceback (most recent call last):
File “/usr/src/app/mbtiles-tools”, line 188, in
main()
File “/usr/src/app/mbtiles-tools”, line 141, in main
asyncio.run(Metadata(args[”], args[‘–show-json’],
File “/usr/local/lib/python3.9/asyncio/runners.py”, line 44, in run
return loop.run_until_complete(main)
File “/usr/local/lib/python3.9/asyncio/base_events.py”, line 647, in run_until_complete
return future.result()
File “/usr/src/app/openmaptiles/mbtile_tools.py”, line 275, in generate
json_data = dict(vector_layers=await get_vector_layers(conn, mvt))
File “/usr/src/app/openmaptiles/pgutils.py”, line 137, in get_vector_layers
fields = await mvt.validate_layer_fields(conn, layer_id, layer)
File “/usr/src/app/openmaptiles/sqltomvt.py”, line 267, in validate_layer_fields
query_field_map = await self.get_sql_fields(connection, layer)
File “/usr/src/app/openmaptiles/sqltomvt.py”, line 299, in get_sql_fields
st = await connection.prepare(f’SELECT * FROM {query} WHERE false LIMIT 0′)
File “/usr/local/lib/python3.9/site-packages/asyncpg/connection.py”, line 566, in prepare
return await self._prepare(
File “/usr/local/lib/python3.9/site-packages/asyncpg/connection.py”, line 584, in _prepare
stmt = await self._get_statement(
File “/usr/local/lib/python3.9/site-packages/asyncpg/connection.py”, line 398, in _get_statement
statement = await self._protocol.prepare(
File “asyncpg/protocol/protocol.pyx”, line 168, in prepare
asyncpg.exceptions.UndefinedColumnError: column “route_1” does not exist
ERROR: 1
make: *** [Makefile:463: generate-tiles-pg] Error 1
how can fix this? how I can trace table with missing column?
Hi all,
I’m trying to deploy this system in a ubuntu server 22.04 with postgresql 17. (I already tried with the 15).
In the “Step 8: Import the Map Data to PostgreSQL” when I do “sudo make import-sql” I got an error saying:
“ERROR: 124
*** ERROR detected, aborting:
psql:/sql/parallel/building.sql:100: ERROR: function zres(integer) does not exist
make: *** [Makefile:453: import-sql] Error 1
”
Anyone can help me?
Thanks a lot.