My Own Mapserver

Aus DL8RDS Wiki
Wechseln zu: Navigation, Suche

1 Scope

We have a really great high performance computer at DB0HSR, and since we're going to do several geography based experiments with it, we need our own mapserver to display positions.

There are other mapservers in the HAMNET:

2 Some basics

3 Mapserver Setup

I am replicating this report:

Other interesting guides:

3.1 Preparations

$ sudo apt-get install libmapnik-dev mapnik-utils git subversion \
dh-autoreconf apache2-dev apache2 unzip postgis make cmake g++ \
libboost-dev libboost-system-dev libboost-filesystem-dev \
libexpat1-dev zlib1g-dev libbz2-dev libpq-dev libproj-dev \
lua5.2 liblua5.2-dev postgis imposm git postgresql postgresql-9.5-postgis-2.2 \

3.2 Get and Prepare the Mapnik style

$ mkdir ~/src
$ cd ~/src

If behind a firewall, subversion will consider the file ~./subversion/servers. There check the settings:

http-proxy-host = (your proxy IP address)
http-proxy-port = (your proxy portnumber)

Then check out the style:

$ svn co mapnik-style
$ cd ~/src/mapnik-style

If behind a firewall, wget will consider the environment variable http_proxy, in the format http://(proxy IP number):(proxy port number)

$ sudo ./ /usr/local/share

will do something like this:

root@mapserver:/home/dl8rds/src/mapnik-style# ./ /usr/local/share
--2017-12-29 03:50:41--
Connecting to connected.
Proxy request sent, awaiting response... 301 Moved Permanently
Location: [following]
--2017-12-29 03:50:41--
Reusing existing connection to
Proxy request sent, awaiting response... 200 OK
Length: 52857349 (50M) [application/x-gzip]
Saving to: ‘/usr/local/share/world_boundaries-spherical.tgz’

/usr/local/share/ 100%[==================================================================================================>]  50,41M  1,12MB/s    in 44s     

2017-12-29 03:51:25 (1,14 MB/s) - ‘/usr/local/share/world_boundaries-spherical.tgz’ saved [52857349/52857349]

--2017-12-29 03:51:25--
Connecting to connected.
Proxy request sent, awaiting response... 301 Moved Permanently
Location: [following]
--2017-12-29 03:51:26--
Reusing existing connection to
Proxy request sent, awaiting response... 200 OK
Length: 409468857 (390M) [application/x-bzip2]
Saving to: ‘/usr/local/share/processed_p.tar.bz2’

/usr/local/share/processed_p.tar.bz2           100%[==================================================================================================>] 390,50M  1,16MB/s    in 6m 21s  

2017-12-29 03:57:47 (1,03 MB/s) - ‘/usr/local/share/processed_p.tar.bz2’ saved [409468857/409468857]

--2017-12-29 03:57:47--
Connecting to connected.
Proxy request sent, awaiting response... 301 Moved Permanently
Location: [following]
--2017-12-29 03:57:47--
Reusing existing connection to
Proxy request sent, awaiting response... 200 OK
Length: 43867136 (42M) [application/x-bzip2]
Saving to: ‘/usr/local/share/shoreline_300.tar.bz2’

/usr/local/share/shoreline_300.tar.bz2         100%[==================================================================================================>]  41,83M  1,29MB/s    in 34s     

2017-12-29 03:58:21 (1,23 MB/s) - ‘/usr/local/share/shoreline_300.tar.bz2’ saved [43867136/43867136]

--2017-12-29 03:58:21--
Connecting to connected.
Proxy request sent, awaiting response... 302 Moved Temporarily
Location: [following]
--2017-12-29 03:58:22--
Reusing existing connection to
Proxy request sent, awaiting response... 200 OK
Length: 2002618 (1,9M) [application/x-zip-compressed]
Saving to: ‘/usr/local/share/’

/usr/local/share/   100%[==================================================================================================>]   1,91M   184KB/s    in 11s     

2017-12-29 03:58:34 (178 KB/s) - ‘/usr/local/share/’ saved [2002618/2002618]

--2017-12-29 03:58:34--
Connecting to connected.
Proxy request sent, awaiting response... 302 Moved Temporarily
Location: [following]
--2017-12-29 03:58:35--
Reusing existing connection to
Proxy request sent, awaiting response... 200 OK
Length: 45804 (45K) [application/x-zip-compressed]
Saving to: ‘/usr/local/share/’

/usr/local/share/ne_110m_admin_0_boundary_line 100%[==================================================================================================>]  44,73K  68,1KB/s    in 0,7s    

2017-12-29 03:58:35 (68,1 KB/s) - ‘/usr/local/share/’ saved [45804/45804]

Archive:  /usr/local/share/
  inflating: /usr/local/share/world_boundaries/ne_10m_populated_places.README.html  
 extracting: /usr/local/share/world_boundaries/ne_10m_populated_places.VERSION.txt  
 extracting: /usr/local/share/world_boundaries/ne_10m_populated_places.cpg  
  inflating: /usr/local/share/world_boundaries/ne_10m_populated_places.dbf  
  inflating: /usr/local/share/world_boundaries/ne_10m_populated_places.prj  
  inflating: /usr/local/share/world_boundaries/ne_10m_populated_places.shp  
  inflating: /usr/local/share/world_boundaries/ne_10m_populated_places.shx  
Archive:  /usr/local/share/
  inflating: /usr/local/share/world_boundaries/ne_110m_admin_0_boundary_lines_land.README.html  
 extracting: /usr/local/share/world_boundaries/ne_110m_admin_0_boundary_lines_land.VERSION.txt  
 extracting: /usr/local/share/world_boundaries/ne_110m_admin_0_boundary_lines_land.cpg  
  inflating: /usr/local/share/world_boundaries/ne_110m_admin_0_boundary_lines_land.dbf  
  inflating: /usr/local/share/world_boundaries/ne_110m_admin_0_boundary_lines_land.prj  
  inflating: /usr/local/share/world_boundaries/ne_110m_admin_0_boundary_lines_land.shp  
  inflating: /usr/local/share/world_boundaries/ne_110m_admin_0_boundary_lines_land.shx  

Now modify some mapnik definitions:

$ cd inc
$ cp
$ cp
$ cp

Modify the file like this:

Settings for symbols, the spatial reference of your postgis tables, coastline s$

<!-- use 'symbols' unless you have moved the symbols directory -->
<!ENTITY symbols "symbols">

<!-- use the '&srs900913;' entity if you have called osm2pgsql without special $
<!ENTITY osm2pgsql_projection "&srs900913;">

<!-- used for 'node in way' ST_DWithin spatial operations -->
<!-- Use 0.1 (meters) when your database is in 900913     -->
<!-- Use 0.000001 (degrees) when your database is in 4326 -->
<!ENTITY dwithin_900913 "0.1">
<!ENTITY dwithin_4326 "0.00001">
<!ENTITY dwithin_node_way "&dwithin_900913;">

<!-- use 'world_boundaries', which is the usual naming for the local folder the$
<!ENTITY world_boundaries "/usr/local/share/world_boundaries">

<!-- use 'planet_osm' unless you have customized your database table prefix usi$
<!ENTITY prefix "planet_osm">

Modify the file like this:

Settings for your postgres setup.

Note: feel free to leave password, host, port, or use blank

<Parameter name="type">postgis</Parameter>
<!-- <Parameter name="password">%(password)s</Parameter> -->
<!-- <Parameter name="host"></Parameter> -->
<!-- <Parameter name="port">5432</Parameter> -->
<!-- <Parameter name="user">gis</Parameter> -->
<Parameter name="dbname">gis</Parameter>
<!-- this should be 'false' if you are manually providing the 'extent' -->
<Parameter name="estimate_extent">false</Parameter>
<!-- manually provided extent in epsg 900913 for whole globe -->
<!-- providing this speeds up Mapnik database queries -->
<Parameter name="extent">-20037508,-19929239,20037508,19929239</Parameter>

Finally copy the mapnik-style directory over to its default location at /etc/mapnik-osm-data:

mv ~./src/mapnik-styles /etc/mapnik-osm-data

3.3 Install and Configure mod_tile and renderd

There are several recommendations to simply build and use mod_tile from source. I decided to generate a debian package first and then install the package properly. I followed this recommendation:

After you have installed mod_tile, there should be the following files:

root@mapserver:/home/dl8rds/src# dpkg -L libapache2-mod-tile 

And the same with renderd:

root@mapserver:/home/dl8rds/src# dpkg -L renderd

Now edit the file /etc/renderd.conf like this:



DESCRIPTION=This is the standard osm mapnik style

Finally ensure that the modules are used by Apache:

$ sudo sh -c 'echo "LoadModule tile_module /usr/lib/apache2/modules/" > /etc/apache2/mods-available/tile.load'

Activate it without an Apache restart:

$ sudo a2enmod tile

Now modify the website definition. Decide what to do with the default website config 000-default.conf. I simply deleted it.

$ sudo nano /etc/apache2/sites-available/tileserver_site.conf

Make sure that the file looks like this:

<VirtualHost *:80>
    DocumentRoot /var/www
    LogLevel info
    AddTileConfig /tiles/ osm
    LoadTileConfigFile /etc/renderd.conf
    ModTileRequestTimeout 3
    ModTileMissingRequestTimeout 10
    ModTileMaxLoadOld 2
    ModTileMaxLoadMissing 5
    ModTileRenderdSocketName /var/run/renderd/renderd.sock
    ModTileCacheDurationMax 604800
    ModTileCacheDurationDirty 900
    ModTileCacheDurationMinimum 10800
    ModTileCacheDurationMediumZoom 13 86400
    ModTileCacheDurationLowZoom 9 518400
    ModTileCacheLastModifiedFactor 0.20
    ModTileEnableTileThrottling Off
    ModTileEnableTileThrottlingXForward 0
    ModTileThrottlingTiles 10000 1 
    ModTileThrottlingRenders 128 0.2
        <Directory />
		Options FollowSymLinks
		AllowOverride None
	<Directory /var/www/>
		Options Indexes FollowSymLinks MultiViews
		AllowOverride None
		Order allow,deny
		allow from all
	ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/
	<Directory "/usr/lib/cgi-bin">
		AllowOverride None
		Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch
		Order allow,deny
		Allow from all
    Alias /doc/ "/usr/share/doc/"
    <Directory "/usr/share/doc/">
        Options Indexes MultiViews FollowSymLinks
        AllowOverride None
        Order deny,allow
        Deny from all
        Allow from ::1/128

Finally restart Apache:

$ sudo apachectl -t

and it should say something like this:

Syntax OK

Oh yes, don't forget to create the runtime directory for renderd:

$ sudo mkdir /var/run/renderd /var/lib/mod_tile $ sudo chown www-data:www-data /var/run/renderd /var/lib/mod_tile

3.4 Download Map Data

Get the dataset for Germany. A company called Geofabrik is providing these files:

Now download:

$ mkdir ~/osm && cd ~/osm wget

3.5 Prepare the Database

There are packages for Ubuntu. You can compile the program yourself, too. I prefer the prefabricated package.

$ sudo apt-get install osm2pgsql

Then prepare the database:

$ sudo -u postgres createuser gis
$ sudo -u postgres createdb -E UTF8 -O gis gis
$ sudo -u postgres createlang plpgsql gis

Edit the file /etc/postgresql/9.5/main/pg_hba.conf:

# "local" is for Unix domain socket connections only
local   all         all                               trust
local   gis         gis                               trust

There is a script that also installs postgis:


Unfortunately it does not work correctly. So let's do it by hand:

$ sudo -u postgres psql -d gis -c 'CREATE EXTENSION postgis; CREATE EXTENSION hstore;'

Correct table ownerships:

$ echo 'ALTER TABLE geometry_columns OWNER TO gis; ALTER TABLE spatial_ref_sys OWNER TO gis;' | sudo -u postgres psql gis

And then import the database:

$ osm2pgsql --create --database gis --username gis --slim --cache 4096 --hstore /data/germany-latest.osm.pbf

Now wait for a couple of days.

3.6 Start everything

Make sure you have the output directory hierarchy:

$ mkdir /var/www/tiles/osm
$ chown -R www-data:www-data /var/www/tiles/osm

Simply make sure all the services are up and running:

$ sudo service renderd start
$ sudo systemctl restart apache2

3.7 Precompute Tiles

First of all, let me state that I have simply decided to base all my maps on Mapnik in order to keep the prcess comparable and simple. It works, and never change a winning team. Not for now.

3.7.1 Method 1: Tirex with Mapnik -- Abandoned.

This could be done with Tirex, but I still have some problems with it. So let's check render_list...

3.7.2 Method 2: render_list with Mapnik

This is my render_list invocation:

root@mapserver:~# render_list -v -n 6 -all --socket=/var/run/renderd/renderd.sock --min-zoom=6 --max-zoom=15 -x 2112 -X 2208 -y 2624 -Y 2880

... but I'm still playing around with it. Our machine is powerful, but it's not a supercomputer. :-) Here is some good advice (in German):

3.7.3 Method 3: with Mapnik

Here is the decription how to do that:

This is the region I'm interested in:

Here is my python file: /usr/share/osm-mapnik/

And I see that it can be modified via environment variables, notably:

  • MAPNIK_MAP_FILE, which is /etc/mapnik-osm-data/osm.xml
  • MAPNIK_TILE_DIR, which is /var/www/tiles/osm

and this section of the file is also interesting:

    bbox = (-180.0,-90.0, 180.0,90.0)

    render_tiles(bbox, mapfile, tile_dir, 0, 5, "World")

    minZoom = 10
    maxZoom = 16
    bbox = (-2, 50.0,1.0,52.0)
    render_tiles(bbox, mapfile, tile_dir, minZoom, maxZoom)

    # Muenchen
    bbox = (11.4,48.07, 11.7,48.22)
    render_tiles(bbox, mapfile, tile_dir, 1, 12 , "Muenchen")

    # Muenchen+
    bbox = (11.3,48.01, 12.15,48.44)
    render_tiles(bbox, mapfile, tile_dir, 7, 12 , "Muenchen+")

    # Muenchen++
    bbox = (10.92,47.7, 12.24,48.61)
    render_tiles(bbox, mapfile, tile_dir, 7, 12 , "Muenchen++")

    # Nuernberg
    render_tiles(bbox, mapfile, tile_dir, 10, 16, "Nuernberg")

    # Karlsruhe
    render_tiles(bbox, mapfile, tile_dir, 10, 16, "Karlsruhe")

    # Karlsruhe+
    bbox = (8.3,48.95,8.5,49.05)
    render_tiles(bbox, mapfile, tile_dir, 1, 16, "Karlsruhe+")

    # Augsburg
    bbox = (8.3,48.95,8.5,49.05)
    render_tiles(bbox, mapfile, tile_dir, 1, 16, "Augsburg")

    # Augsburg+
    render_tiles(bbox, mapfile, tile_dir, 10, 14, "Augsburg+")

    # Europe+
    bbox = (1.0,10.0, 20.6,50.0)
    render_tiles(bbox, mapfile, tile_dir, 1, 11 , "Europe+")

I select Munich++ first to see the result. Here is the invocation:

root@mapserver:/data/tiles# /usr/share/osm-mapnik/ &
[1] 11966
root@mapserver:/data/tiles# render_tiles( (10.92, 47.7, 12.24, 48.61) /etc/mapnik-osm-data/osm.xml /var/www/tiles/osm/ 7 12 Muenchen++ )
Muenchen++ : 8 135 89  
Muenchen++ : 8 135 88  
Muenchen++ : 7 68 44  
Muenchen++ : 8 136 89  
Muenchen++ : 9 271 176  
Muenchen++ : 9 271 178  
Muenchen++ : 9 271 177  
Muenchen++ : 8 136 88  
Muenchen++ : 9 272 176  
Muenchen++ : 9 272 178  
Muenchen++ : 9 273 176  
Muenchen++ : 9 272 177  
Muenchen++ : 10 543 353  
Muenchen++ : 9 273 178  
Muenchen++ : 9 273 177  
Muenchen++ : 10 543 356  
Muenchen++ : 10 543 355  
Muenchen++ : 10 543 354  
Muenchen++ : 10 543 357  
Muenchen++ : 10 544 353  
Muenchen++ : 10 544 356  
Muenchen++ : 10 544 354  
Muenchen++ : 10 544 355  
Muenchen++ : 10 544 357  
Muenchen++ : 10 545 353

Looks good. Still need to

chown -R www-data:www-data /var/www/tiles/osm/

Same, same, but different: I modify the generator like this in order to keep the multicore CPU busy:

   # Bayern++
   render_tiles(bbox, mapfile, tile_dir, 8, 16, "Bayern++")

And starting it with

root@mapserver:~# nohup time /usr/share/osm-mapnik/ &

Works fine.

4 Create a OpenLayers Application

Create a file with this content at /var/www/map.html:

<!DOCTYPE html>
    <link rel="stylesheet" href="ol.css" type="text/css">
      .map { height: 95vh; width: 100%; }
    <script src="ol.js" type="text/javascript"></script>
    <title>OpenLayers example</title>
    <div id="map" class="map"></div>
    <script type="text/javascript">
    var map = new ol.Map({
      target: 'map',
      layers: [
        new ol.layer.Tile({
            source: new ol.source.OSM({url: '/tiles/osm/{z}/{x}/{y}.png', maxZoom: 20})
      view: new ol.View({
        center: ol.proj.fromLonLat([8, 53]),
        zoom: 8

I decided to download the CSS and JS files onto my document root so that a client will not need to access the internet for it:

$ cd /var/www
$ wget
$ wget
$ chown -R www-data:www-data *

4.1 Check it out!