My Own Mapserver: Unterschied zwischen den Versionen
Dl8rds (Diskussion | Beiträge) (→Prepare the Database) |
Dl8rds (Diskussion | Beiträge) (→Prepare the Database) |
||
Zeile 432: | Zeile 432: | ||
# "local" is for Unix domain socket connections only | # "local" is for Unix domain socket connections only | ||
+ | local all all trust | ||
local gis gis trust | local gis gis trust | ||
− | |||
There is a script that also installs postgis: | There is a script that also installs postgis: | ||
Zeile 442: | Zeile 442: | ||
$ sudo -u postgres psql -d gis -c 'CREATE EXTENSION postgis; CREATE EXTENSION hstore;' | $ sudo -u postgres psql -d gis -c 'CREATE EXTENSION postgis; CREATE EXTENSION hstore;' | ||
− | |||
− | |||
− | |||
Correct table ownerships: | Correct table ownerships: | ||
Zeile 450: | Zeile 447: | ||
$ echo 'ALTER TABLE geometry_columns OWNER TO gis; ALTER TABLE spatial_ref_sys OWNER TO gis;' | sudo -u postgres psql gis | $ echo 'ALTER TABLE geometry_columns OWNER TO gis; ALTER TABLE spatial_ref_sys OWNER TO gis;' | sudo -u postgres psql gis | ||
− | + | And then import the database: | |
− | $ | + | $ osm2pgsql --create --database gis --username gis --slim --cache 4096 --hstore /data/germany-latest.osm.pbf |
− | |||
− | |||
− | + | Now wait for a couple of days. | |
=== Start everything === | === Start everything === |
Version vom 29. Dezember 2017, 21:26 Uhr
Inhaltsverzeichnis
1 Scope
We have a really great high performance computer at DB0HSR, and since we're going to do several geography based experiments with it, we need our own mapserver to display positions.
There are other mapservers in the HAMNET:
2 Implementation
I am replicating this report:
Other interesting guides:
2.1 Preparations
$ sudo apt-get install libmapnik-dev mapnik-utils git subversion \ dh-autoreconf apache2-dev apache2 unzip postgis make cmake g++ \ libboost-dev libboost-system-dev libboost-filesystem-dev \ libexpat1-dev zlib1g-dev libbz2-dev libpq-dev libproj-dev \ lua5.2 liblua5.2-dev postgis imposm git postgresql postgresql-9.5-postgis-2.2 \ postgresql-9.5-postgis-scripts
2.2 Get and Prepare the Mapnik style
$ mkdir ~/src $ cd ~/src
If behind a firewall, subversion will consider the file ~./subversion/servers. There check the settings:
http-proxy-host = (your proxy IP address) http-proxy-port = (your proxy portnumber)
Then check out the style:
$ svn co http://svn.openstreetmap.org/applications/rendering/mapnik mapnik-style $ cd ~/src/mapnik-style
If behind a firewall, wget will consider the environment variable http_proxy, in the format http://(proxy IP number):(proxy port number)
$ sudo ./get-coastlines.sh /usr/local/share
will do something like this:
root@mapserver:/home/dl8rds/src/mapnik-style# ./get-coastlines.sh /usr/local/share --2017-12-29 03:50:41-- http://tile.openstreetmap.org/world_boundaries-spherical.tgz Connecting to 44.225.41.33:8080... connected. Proxy request sent, awaiting response... 301 Moved Permanently Location: http://planet.openstreetmap.org/historical-shapefiles/world_boundaries-spherical.tgz [following] --2017-12-29 03:50:41-- http://planet.openstreetmap.org/historical-shapefiles/world_boundaries-spherical.tgz Reusing existing connection to 44.225.41.33:8080. Proxy request sent, awaiting response... 200 OK Length: 52857349 (50M) [application/x-gzip] Saving to: ‘/usr/local/share/world_boundaries-spherical.tgz’ /usr/local/share/world_boundaries-spherical.tg 100%[==================================================================================================>] 50,41M 1,12MB/s in 44s 2017-12-29 03:51:25 (1,14 MB/s) - ‘/usr/local/share/world_boundaries-spherical.tgz’ saved [52857349/52857349] --2017-12-29 03:51:25-- http://tile.openstreetmap.org/processed_p.tar.bz2 Connecting to 44.225.41.33:8080... connected. Proxy request sent, awaiting response... 301 Moved Permanently Location: http://planet.openstreetmap.org/historical-shapefiles/processed_p.tar.bz2 [following] --2017-12-29 03:51:26-- http://planet.openstreetmap.org/historical-shapefiles/processed_p.tar.bz2 Reusing existing connection to 44.225.41.33:8080. Proxy request sent, awaiting response... 200 OK Length: 409468857 (390M) [application/x-bzip2] Saving to: ‘/usr/local/share/processed_p.tar.bz2’ /usr/local/share/processed_p.tar.bz2 100%[==================================================================================================>] 390,50M 1,16MB/s in 6m 21s 2017-12-29 03:57:47 (1,03 MB/s) - ‘/usr/local/share/processed_p.tar.bz2’ saved [409468857/409468857] --2017-12-29 03:57:47-- http://tile.openstreetmap.org/shoreline_300.tar.bz2 Connecting to 44.225.41.33:8080... connected. Proxy request sent, awaiting response... 301 Moved Permanently Location: http://planet.openstreetmap.org/historical-shapefiles/shoreline_300.tar.bz2 [following] --2017-12-29 03:57:47-- http://planet.openstreetmap.org/historical-shapefiles/shoreline_300.tar.bz2 Reusing existing connection to 44.225.41.33:8080. Proxy request sent, awaiting response... 200 OK Length: 43867136 (42M) [application/x-bzip2] Saving to: ‘/usr/local/share/shoreline_300.tar.bz2’ /usr/local/share/shoreline_300.tar.bz2 100%[==================================================================================================>] 41,83M 1,29MB/s in 34s 2017-12-29 03:58:21 (1,23 MB/s) - ‘/usr/local/share/shoreline_300.tar.bz2’ saved [43867136/43867136] --2017-12-29 03:58:21-- http://www.naturalearthdata.com/http//www.naturalearthdata.com/download/10m/cultural/ne_10m_populated_places.zip Connecting to 44.225.41.33:8080... connected. Proxy request sent, awaiting response... 302 Moved Temporarily Location: http://naciscdn.org/naturalearth/10m/cultural/ne_10m_populated_places.zip [following] --2017-12-29 03:58:22-- http://naciscdn.org/naturalearth/10m/cultural/ne_10m_populated_places.zip Reusing existing connection to 44.225.41.33:8080. Proxy request sent, awaiting response... 200 OK Length: 2002618 (1,9M) [application/x-zip-compressed] Saving to: ‘/usr/local/share/ne_10m_populated_places.zip’ /usr/local/share/ne_10m_populated_places.zip 100%[==================================================================================================>] 1,91M 184KB/s in 11s 2017-12-29 03:58:34 (178 KB/s) - ‘/usr/local/share/ne_10m_populated_places.zip’ saved [2002618/2002618] --2017-12-29 03:58:34-- http://www.naturalearthdata.com/http//www.naturalearthdata.com/download/110m/cultural/ne_110m_admin_0_boundary_lines_land.zip Connecting to 44.225.41.33:8080... connected. Proxy request sent, awaiting response... 302 Moved Temporarily Location: http://naciscdn.org/naturalearth/110m/cultural/ne_110m_admin_0_boundary_lines_land.zip [following] --2017-12-29 03:58:35-- http://naciscdn.org/naturalearth/110m/cultural/ne_110m_admin_0_boundary_lines_land.zip Reusing existing connection to 44.225.41.33:8080. Proxy request sent, awaiting response... 200 OK Length: 45804 (45K) [application/x-zip-compressed] Saving to: ‘/usr/local/share/ne_110m_admin_0_boundary_lines_land.zip’ /usr/local/share/ne_110m_admin_0_boundary_line 100%[==================================================================================================>] 44,73K 68,1KB/s in 0,7s 2017-12-29 03:58:35 (68,1 KB/s) - ‘/usr/local/share/ne_110m_admin_0_boundary_lines_land.zip’ saved [45804/45804] world_boundaries/ world_boundaries/places.shx world_boundaries/world_boundaries_m.index world_boundaries/world_bnd_m.shx world_boundaries/builtup_area.shx world_boundaries/world_bnd_m.dbf world_boundaries/builtup_area.prj world_boundaries/places.shp world_boundaries/world_boundaries_m.shx world_boundaries/world_boundaries_m.shp world_boundaries/places.dbf world_boundaries/places.prj world_boundaries/builtup_area.dbf world_boundaries/world_bnd_m.shp world_boundaries/world_bnd_m.prj world_boundaries/world_boundaries_m.dbf world_boundaries/builtup_area.shp world_boundaries/world_boundaries_m.prj world_boundaries/world_bnd_m.index world_boundaries/builtup_area.index processed_p.dbf processed_p.index processed_p.shp processed_p.shx shoreline_300.dbf shoreline_300.index shoreline_300.shp shoreline_300.shx Archive: /usr/local/share/ne_10m_populated_places.zip inflating: /usr/local/share/world_boundaries/ne_10m_populated_places.README.html extracting: /usr/local/share/world_boundaries/ne_10m_populated_places.VERSION.txt extracting: /usr/local/share/world_boundaries/ne_10m_populated_places.cpg inflating: /usr/local/share/world_boundaries/ne_10m_populated_places.dbf inflating: /usr/local/share/world_boundaries/ne_10m_populated_places.prj inflating: /usr/local/share/world_boundaries/ne_10m_populated_places.shp inflating: /usr/local/share/world_boundaries/ne_10m_populated_places.shx Archive: /usr/local/share/ne_110m_admin_0_boundary_lines_land.zip inflating: /usr/local/share/world_boundaries/ne_110m_admin_0_boundary_lines_land.README.html extracting: /usr/local/share/world_boundaries/ne_110m_admin_0_boundary_lines_land.VERSION.txt extracting: /usr/local/share/world_boundaries/ne_110m_admin_0_boundary_lines_land.cpg inflating: /usr/local/share/world_boundaries/ne_110m_admin_0_boundary_lines_land.dbf inflating: /usr/local/share/world_boundaries/ne_110m_admin_0_boundary_lines_land.prj inflating: /usr/local/share/world_boundaries/ne_110m_admin_0_boundary_lines_land.shp inflating: /usr/local/share/world_boundaries/ne_110m_admin_0_boundary_lines_land.shx root@mapserver:/home/dl8rds/src/mapnik-style#
Now modify some mapnik definitions:
$ cd inc $ cp fontset-settings.xml.inc.template fontset-settings.xml.inc $ cp datasource-settings.xml.inc.template datasource-settings.xml.inc $ cp settings.xml.inc.template settings.xml.inc
Modify the file settings.xml.inc like this:
<!-- Settings for symbols, the spatial reference of your postgis tables, coastline s$ --> <!-- use 'symbols' unless you have moved the symbols directory --> <!ENTITY symbols "symbols"> <!-- use the '&srs900913;' entity if you have called osm2pgsql without special $ <!ENTITY osm2pgsql_projection "&srs900913;"> <!-- used for 'node in way' ST_DWithin spatial operations --> <!-- Use 0.1 (meters) when your database is in 900913 --> <!-- Use 0.000001 (degrees) when your database is in 4326 --> <!ENTITY dwithin_900913 "0.1"> <!ENTITY dwithin_4326 "0.00001"> <!ENTITY dwithin_node_way "&dwithin_900913;"> <!-- use 'world_boundaries', which is the usual naming for the local folder the$ <!ENTITY world_boundaries "/usr/local/share/world_boundaries"> <!-- use 'planet_osm' unless you have customized your database table prefix usi$ <!ENTITY prefix "planet_osm">
Modify the file datasource-settings.xml.inc like this:
<Parameter name="type">postgis</Parameter>
<Parameter name="dbname">gis</Parameter>
<Parameter name="estimate_extent">false</Parameter>
<Parameter name="extent">-20037508,-19929239,20037508,19929239</Parameter>
</pre>
Finally copy the mapnik-style directory over to its default location at /etc/mapnik-osm-data:
mv ~./src/mapnik-styles /etc/mapnik-osm-data
2.3 Install and Configure mod_tile and renderd
There are several recommendations to simply build and use mod_tile from source. I decided to generate a debian package first and then install the package properly. I followed this recommendation:
https://switch2osm.org/serving-tiles/building-a-tile-server-from-packages/
After you have installed mod_tile, there should be the following files:
root@mapserver:/home/dl8rds/src# dpkg -L libapache2-mod-tile /. /var /var/www /var/www/osm /var/www/osm/slippymap.html /usr /usr/bin /usr/bin/openstreetmap-tiles-update-expire /usr/bin/osmosis-db_replag /usr/share /usr/share/munin /usr/share/munin/plugins /usr/share/munin/plugins/mod_tile_fresh /usr/share/munin/plugins/mod_tile_latency /usr/share/munin/plugins/mod_tile_zoom /usr/share/munin/plugins/mod_tile_response /usr/share/doc /usr/share/doc/libapache2-mod-tile /usr/share/doc/libapache2-mod-tile/copyright /usr/share/doc/libapache2-mod-tile/changelog.Debian.gz /usr/share/doc/libapache2-mod-tile/readme.txt.gz /usr/lib /usr/lib/apache2 /usr/lib/apache2/modules /usr/lib/apache2/modules/mod_tile.so /etc /etc/apache2 /etc/apache2/mods-available /etc/apache2/mods-available/tile.load /etc/apache2/sites-available /etc/apache2/sites-available/tileserver_site.conf
And the same with renderd:
root@mapserver:/home/dl8rds/src# dpkg -L renderd /. /usr /usr/bin /usr/bin/render_old /usr/bin/render_expired /usr/bin/renderd /usr/bin/render_speedtest /usr/bin/render_list /usr/share /usr/share/munin /usr/share/munin/plugins /usr/share/munin/plugins/renderd_processed /usr/share/munin/plugins/renderd_zoom_time /usr/share/munin/plugins/renderd_zoom /usr/share/munin/plugins/renderd_queue /usr/share/munin/plugins/renderd_queue_time /usr/share/doc /usr/share/doc/renderd /usr/share/doc/renderd/copyright /usr/share/doc/renderd/changelog.Debian.gz /usr/share/doc/renderd/readme.txt.gz /usr/share/man /usr/share/man/man8 /usr/share/man/man8/renderd.8.gz /usr/share/man/man1 /usr/share/man/man1/render_expired.1.gz /usr/share/man/man1/render_speedtest.1.gz /usr/share/man/man1/render_old.1.gz /usr/share/man/man1/render_list.1.gz /usr/lib /usr/lib/libiniparser.so.3.0.0 /usr/lib/libiniparser.la /usr/lib/libiniparser.a /etc /etc/renderd.conf /etc/init.d /etc/init.d/renderd /usr/lib/libiniparser.so.3 /usr/lib/libiniparser.so
Now edit the file /etc/renderd.conf like this:
... plugins_dir=/usr/lib/mapnik/3.0/input/ ... [default] URI=/osm_tiles/ XML=/etc/mapnik-osm-data/osm.xml DESCRIPTION=This is the standard osm mapnik style ;ATTRIBUTION=©<a href=\"http://www.openstreetmap.org/\">OpenStreetMap</a> and <a href=\"http://wiki.openstreetmap.org/wiki/Contributors\">contributors</a>, <a href=\"http://creativecommons.org/licenses/by-sa/2.0/\">CC-BY-SA</a> HOST=mapserver.db0hsr.apr.org ;SERVER_ALIAS=http://a.tile.openstreetmap.org ;SERVER_ALIAS=http://b.tile.openstreetmap.org ;HTCPHOST=proxy.openstreetmap.org
Finally ensure that the modules are used by Apache:
$ sudo sh -c 'echo "LoadModule tile_module /usr/lib/apache2/modules/mod_tile.so" > /etc/apache2/mods-available/tile.load'
Activate it without an Apache restart:
$ sudo a2enmod tile
Now modify the website definition. Decide what to do with the default website config 000-default.conf. I simply deleted it.
$ sudo nano /etc/apache2/sites-available/tileserver_site.conf
Make sure that the file looks like this:
root@mapserver:/home/dl8rds/src# cat /etc/apache2/sites-enabled/tileserver_site.conf | grep -v "#" | grep -v "^$" <VirtualHost *:80> ServerAdmin dl8rds@darc.de ServerName mapserver.db0hsr.ampr.org DocumentRoot /var/www LogLevel info LoadTileConfigFile /etc/renderd.conf ModTileRequestTimeout 3 ModTileMissingRequestTimeout 10 ModTileMaxLoadOld 2 ModTileMaxLoadMissing 5 ModTileRenderdSocketName /var/run/renderd/renderd.sock ModTileCacheDurationMax 604800 ModTileCacheDurationDirty 900 ModTileCacheDurationMinimum 10800 ModTileCacheDurationMediumZoom 13 86400 ModTileCacheDurationLowZoom 9 518400 ModTileCacheLastModifiedFactor 0.20 ModTileEnableTileThrottling Off ModTileEnableTileThrottlingXForward 0 ModTileThrottlingTiles 10000 1 ModTileThrottlingRenders 128 0.2 <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options Indexes FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/ <Directory "/usr/lib/cgi-bin"> AllowOverride None Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch Order allow,deny Allow from all </Directory> Alias /doc/ "/usr/share/doc/" <Directory "/usr/share/doc/"> Options Indexes MultiViews FollowSymLinks AllowOverride None Order deny,allow Deny from all Allow from 127.0.0.0/255.0.0.0 ::1/128 </Directory> </VirtualHost>
Finally restart Apache:
$ sudo apachectl -t
and it should say something like this:
Syntax OK
Oh yes, don't forget to create the runtime directory for renderd:
$ sudo mkdir /var/run/renderd /var/lib/mod_tile $ sudo chown www-data:www-data /var/run/renderd /var/lib/mod_tile
2.4 Download Map Data
Get the dataset for Germany. A company called Geofabrik is providing these files:
http://download.geofabrik.de/europe/germany.html
Now download:
$ mkdir ~/osm && cd ~/osm wget http://download.geofabrik.de/europe/germany-latest.osm.pbf
2.5 Prepare the Database
There are packages for Ubuntu. You can compile the program yourself, too. I prefer the prefabricated package.
$ sudo apt-get install osm2pgsql
Then prepare the database:
$ sudo -u postgres createuser gis $ sudo -u postgres createdb -E UTF8 -O gis gis $ sudo -u postgres createlang plpgsql gis
Edit the file /etc/postgresql/9.5/main/pg_hba.conf:
# "local" is for Unix domain socket connections only local all all trust local gis gis trust
There is a script that also installs postgis:
/usr/share/doc/osm2pgsql/examples/install-postgis-osm-db.sh
Unfortunately it does not work correctly. So let's do it by hand:
$ sudo -u postgres psql -d gis -c 'CREATE EXTENSION postgis; CREATE EXTENSION hstore;'
Correct table ownerships:
$ echo 'ALTER TABLE geometry_columns OWNER TO gis; ALTER TABLE spatial_ref_sys OWNER TO gis;' | sudo -u postgres psql gis
And then import the database:
$ osm2pgsql --create --database gis --username gis --slim --cache 4096 --hstore /data/germany-latest.osm.pbf
Now wait for a couple of days.
2.6 Start everything
$ sudo service renderd start $ sudo systemctl restart apache2