28 May 2010

Geoserver rendering Openstreetmap update1 (windows)

Two years ago, I started using Geoserver and PostGIS. I was very
exciting and satisfied about performance. Since 2009, I don't have a
lot of work about Geoserver. Time flies, last week I have some
assignment about Geoserver again. So I can refresh my geoserver
memory.

For today, what I am going to do :
1.import latest Openstreetmap data into PostGIS
2.Render it by Geoserver 2.0.1

Here is information about local settings.

--Platform: windows XP SP3
--Postgresql 8.3 + PostGIS 1.3.5
--uDig 1.1.1
--Tomcat 6.0 + Geoserver 2.0.1

To import osm data into PostGIS:
I suggest that you read following articles before you start.

--general information about osm2pgsql tool
http://wiki.openstreetmap.org/wiki/Osm2pgsql

-- There is some important information about osm2pgsql tool and PostGIS
http://wiki.openstreetmap.org/wiki/Mapnik

-- general information about OpenStreetmap dataset
http://wiki.openstreetmap.org/wiki/Planet.osm

Data and tools:

osm2pgsql windows version
-- http://tile.openstreetmap.org/osm2pgsql.zip

osm dataset(I used planet-nl-100423.osm.gz , around 500MB)
http://hypercube.telascience.org/planet/
(sometimes, their service is not available.)

before start:
I have already created one postgis database.

Step1: unzip osm2pgsql to c:\ osm2pgsql\ and add the osm2pgsql path to
system path.
http://farm5.static.flickr.com/4055/4647620497_c8424365c8_o.jpg
add c:\osm2pgsql\osm2pgsql to system path
windows Key + E, show file explorer.
Then on file tree, right click mouse, then click properties.
On properties windows, click 'Advance 'Tab
click Environment variable
http://farm5.static.flickr.com/4041/4647620813_ccc8d9a41d_o.jpg
Edit path :
http://farm4.static.flickr.com/3493/4647621133_9989696093_o.jpg
click ok and ok...

Step2: In the osm2pgsql folder, there is a sql query which named 900913.sql.
Run it on your database, you can run it via pgadmin or psql tool.
Step 3, import data into database.
Open one command window,
type "osm2pgsql -h"

http://farm5.static.flickr.com/4025/4648235534_3700e68f3e_o.jpg

then type "osm2pgsql -c -d YOURDATABASENAME -U postgres -s -v -S c
:\osm2pgsql\osm2pgsql\default.style D:\openstreetmap\planet-nl-
100423.osm.gz"

(When you import data, better switch off all other programs, because
importing would take a lot of your ram.)

After a while you should see something like this:
osm2pgsql SVN version 0.69-20874M

Using projection SRS 900913 (Spherical Mercator)
Setting up table: planet_osm_point
NOTICE: table "planet_osm_point" does not exist, skipping
NOTICE: table "planet_osm_point_tmp" does not exist, skipping
Setting up table: planet_osm_line
NOTICE: table "planet_osm_line" does not exist, skipping
NOTICE: table "planet_osm_line_tmp" does not exist, skipping
Setting up table: planet_osm_polygon
NOTICE: table "planet_osm_polygon" does not exist, skipping
NOTICE: table "planet_osm_polygon_tmp" does not exist, skipping
Setting up table: planet_osm_roads
NOTICE: table "planet_osm_roads" does not exist, skipping
NOTICE: table "planet_osm_roads_tmp" does not exist, skipping
Mid: pgsql, scale=100, cache=800MB, maxblocks=102401*8192
Setting up table: planet_osm_nodes
*** WARNING: intarray contrib module not installed
*** The resulting database will not be usable for applying diffs.
NOTICE: table "planet_osm_nodes" does not exist, skipping
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "planet_osm_nodes
_pkey" for table "planet_osm_nodes"
Setting up table: planet_osm_ways
NOTICE: table "planet_osm_ways" does not exist, skipping
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "planet_osm_ways_
pkey" for table "planet_osm_ways"
Setting up table: planet_osm_rels
NOTICE: table "planet_osm_rels" does not exist, skipping
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "planet_osm_rels_
pkey" for table "planet_osm_rels"

!! You are running this on 32bit system, so at most
!! 3GB of RAM can be used. If you encounter unexpected
!! exceptions during import, you should try running in slim
!! mode using parameter -s.

Reading in file: D:\openstreetmap\planet-nl-100423.osm.gz
Processing: Node(28102k) Way(4756k) Relation(30k)
Node stats: total(28102656), max(704144199)
Way stats: total(4756486), max(56116903)
Relation stats: total(30289), max(571919)

Going over pending ways
processing way (2981k)

Going over pending relations

node cache: stored: 26780303(95.29%), storage efficiency: 25.54%, hit rate: 94.1
1%
Stopping table: planet_osm_nodes
Stopped table: planet_osm_nodes
Stopping table: planet_osm_ways
Stopped table: planet_osm_ways
Stopping table: planet_osm_rels
Stopped table: planet_osm_rels
Committing transaction for planet_osm_point
Sorting data and creating indexes for planet_osm_point
Completed planet_osm_point
Committing transaction for planet_osm_line
Sorting data and creating indexes for planet_osm_line
Completed planet_osm_line
Committing transaction for planet_osm_polygon
Sorting data and creating indexes for planet_osm_polygon
Completed planet_osm_polygon
Committing transaction for planet_osm_roads
Sorting data and creating indexes for planet_osm_roads
Completed planet_osm_roads

With pgadmin you should see like this:
http://farm5.static.flickr.com/4072/4648235806_d464e37db9_o.jpg
note: if there is error during import, there is a lot of how-to-fix in
the wiki from openstreemap


The second part: link PostGIS to Geoserver:


before start :
download one example sld for osm_line,
http://blog.geoserver.org/wp-content/uploads/osm_roadssld.zip

add this sld to your geoserver.
There is some information about SLD :
http://blog.geoserver.org/2010/04/09/sld-cookbook/


Let start!
Go http://localhost:8080/geoserver/
login with admin

Step 1: click store

step 2: add new store and click postgis(not postigs jndi)

select workspace topp
fill in information for your database.
You see something like this:
because I have already publish some layers, you wont see same screen like this.

Step3 publish new layer planet_osm_line
Just click publish planet_osm_line

then click compute from data and compute from native bounds.

Then then click publishing tab
chang default style to osm_line.
Click save.
Then go to your demo page

there are more screen dumps here
http://www.flickr.com/photos/43687674@N07/
--
Xiaoyu Guan (Sam)
http://guanxiaoyu.blogspot.com

September 7th, 2008

Recently, I setup GeoServer on Ubuntu 8.04. It would be nice to share
some information with everybody. it is not easy to install everything.
Firstly, you need Ubuntu, you can download and install it by yourself
from www.ubuntu.org. beside that, you need JDK6, Tomcat6 and
Geoserver.war.

about how to install JDK and tomcat

here is a very good article from howtogeek.com. how to setup tomcat 6 on ubuntu.
http://www.howtogeek.com/howto/linux/installing-tomcat-6-on-ubuntu/

Step1. install JDK

open terminal , type 'sudo apt-get install sun-java6-jdk'

do not switch off the terminal.

step2. install Tomcat6

use commands

'wget http://apache.hoxt.com/tomcat/tomcat-6/v6.0.xx/bin/apache-tomcat-6.0.14.tar.gz'

'tar xvzf apache-tomcat-6.0.xx.tar.gz'

there are three versions tomcat, 14,16,18, choose your version, replace 'xx'

follow the article, move all the installation file to '/usr/local/tomcat'

use commands

'sudo mv apache-tomcat-6.0.14 /usr/local/tomcat'

open another terminal , use command 'sudo nautilus'

it will open a new fie explorer , use the file explorer open this file

'/usr/local/tomat/bin/shartup.sh'

at the 7th line add

'export JAVA_HOME=/usr/lib/jvm/java-6-sun'

save it and close the explorer.

go back to the articles. setup automatically startup .

easy way is download the file, put it at '/etc/init.d/'

in the terminal, use command 'sudo /etc/init.d/tomcat restart' , btw,
if you want to stop tomcat, use command 'sudo /etc/init.d/tomcat stop'

after restart it, if everything fine. open our Firefox , via tomcat
by address http://127.0.0.1:8180/

then you depoly your geoserver.war files. everything done.

--
Xiaoyu Guan (Sam)
http://guanxiaoyu.blogspot.com

Now OpenStreetmap covers most euro regions. I am surprised when I am
the first saw OpenStreetMap. It looks just fantastic. Actually, in
2007, one of openstreetmap developer gave us a lecture about their
ideas. How openstreetmap goings. But at that time, I have no idea
about it. It just like copying ideas from wikipedia. Now it is very
nice 'Copying'. Everybody have free and comprehensive spatial
information for free.

I would like to compare Google Maps, Microsoft maps or Yahoo maps with
OpenStreetMap. I think some other people must be already wrote
something about it. There is no need to repeat it again.

Back to today's point, using Geoserver rendering OpenStreetMap.
Honestly, it is not easy to rendering OSM by Geoserver. Because there
is no offical OSM SLD file for GeoServer. In the mailinglist there are
few people asked whether there is a SLD of OpenStreetMap. I did not
find any SLD file of OSM map until last month. One geoserver user
wrote a tutorial to teach people how to write Google style SLD. Based
on this turtorial, we can easily write an OSM SLD for GeoServer.

Data:

http://t1.hypercube.telascience.org/planet/?C=S;O=A

Tools

GeoServer

PostGIS/PostgreSQL

Useful links:

http://oegeo.wordpress.com/

http://www.perrygeo.net/wordpress/

http://geoserver.org/display/GEOSDOC/Google+Maps+SLD

http://geoserver.org/display/GEOSDOC/Geoserver+and+Google+SLDs

Here I would like to spend sometime to complain about the quality of
OSM. You could say OSM it is not perfect datasets. THE ATTRIBUTE TABLE
IS A MESS. There is no free lunch. So we still need to adopt to the
bad quality datasets.

The prepare job would be setup a GeoServer and a PostGIS. And import
OSM map into PostGIS. If you are a beginner of PostGIS or GeoServer or
SLD. Then you need time to read some material before continue reading
this article.

Something for windows users, I don't know how did you import osm file
into PostGIS. There is a tool osm2pgsql.exe which can easily import
osm data to PostGIS. From my experience, this tool is quite nasty. I
had two version of datasets. I could not import them to PostGIS by
osm2pgsql.exe. Even one of them did been import into database, but it
with some other problems. Actually, there are some errors when import
it. Anyway, I recommend use LINUX version osm2pgsql.exe to import
data. After that, you can back up your database to a backup file. Then
restore it on windows PostGIS. It is works fine for me. When you have
error with osm2pgsql.exe, please, follow me.

There will be four layers of openstreetmap, road, lines, polygon and
points. Here I only used three layers: lines ploygon and points.

Here is some example about how to write lines featuers.

A secondary line with labels

<sld:Rule>
<sld:Name>rule008</sld:Name>
<sld:Title>secondary trunk</sld:Title>
<sld:Abstract>Abstract</sld:Abstract>
<ogc:Filter>
<ogc:Or>

<ogc:PropertyIsEqualTo>
<ogc:PropertyName>highway</ogc:PropertyName>
<ogc:Literal>secondary</ogc:Literal>
</ogc:PropertyIsEqualTo>

<ogc:PropertyIsEqualTo>
<ogc:PropertyName>highway</ogc:PropertyName>
<ogc:Literal>secondary_link</ogc:Literal>
</ogc:PropertyIsEqualTo>

</ogc:Or>

</ogc:Filter>
<sld:MinScaleDenominator>150000</sld:MinScaleDenominator>
<sld:MaxScaleDenominator>400000</sld:MaxScaleDenominator>
<sld:LineSymbolizer>
<sld:Stroke>
<sld:CssParameter name="stroke">
<ogc:Literal>#d69c51</ogc:Literal>
</sld:CssParameter>
<sld:CssParameter name="stroke-linecap">
<ogc:Literal>butt</ogc:Literal>
</sld:CssParameter>
<sld:CssParameter name="stroke-linejoin">
<ogc:Literal>miter</ogc:Literal>
</sld:CssParameter>
<sld:CssParameter name="stroke-opacity">
<ogc:Literal>1.0</ogc:Literal>
</sld:CssParameter>
<sld:CssParameter name="stroke-width">
<ogc:Literal>5.0</ogc:Literal>
</sld:CssParameter>
<sld:CssParameter name="stroke-dashoffset">
<ogc:Literal>0.0</ogc:Literal>
</sld:CssParameter>
</sld:Stroke>
</sld:LineSymbolizer>
<sld:LineSymbolizer>
<sld:Stroke>
<sld:CssParameter name="stroke">
<ogc:Literal>#fdbf6f</ogc:Literal>
</sld:CssParameter>
<sld:CssParameter name="stroke-linecap">
<ogc:Literal>round</ogc:Literal>
</sld:CssParameter>
<sld:CssParameter name="stroke-linejoin">
<ogc:Literal>round</ogc:Literal>
</sld:CssParameter>
<sld:CssParameter name="stroke-opacity">
<ogc:Literal>1.0</ogc:Literal>
</sld:CssParameter>
<sld:CssParameter name="stroke-width">
<ogc:Literal>4.0</ogc:Literal>
</sld:CssParameter>
<sld:CssParameter name="stroke-dashoffset">
<ogc:Literal>0.0</ogc:Literal>
</sld:CssParameter>
</sld:Stroke>
</sld:LineSymbolizer>

<sld:TextSymbolizer>
<sld:Label>
<ogc:PropertyName>ref</ogc:PropertyName>
</sld:Label>
<sld:Font>
<sld:CssParameter name="font-family">
<ogc:Literal>Lucida Sans</ogc:Literal>
</sld:CssParameter>
<sld:CssParameter name="font-size">
<ogc:Literal>6.0</ogc:Literal>
</sld:CssParameter>
<sld:CssParameter name="font-style">
<ogc:Literal>normal</ogc:Literal>
</sld:CssParameter>

<sld:CssParameter name="font-weight">
<ogc:Literal>bold</ogc:Literal>
</sld:CssParameter>
</sld:Font>
<sld:LabelPlacement>
<sld:PointPlacement>
<sld:AnchorPoint>
<sld:AnchorPointX>0.5</sld:AnchorPointX>
<sld:AnchorPointY>0.5</sld:AnchorPointY>
</sld:AnchorPoint>
</sld:PointPlacement>
</sld:LabelPlacement>
<sld:Fill>
<sld:CssParameter name="fill">
<ogc:Literal>#fdbf6f</ogc:Literal>
</sld:CssParameter>
<sld:CssParameter name="fill-opacity">
<ogc:Literal>1.0</ogc:Literal>
</sld:CssParameter>
</sld:Fill>

<sld:VendorOption name="group">yes</sld:VendorOption>
<sld:VendorOption name="spaceAround">2</sld:VendorOption>

<sld:Graphic>
<sld:ExternalGraphic>
<sld:OnlineResource xlink:type="simple" xlink:href="secondary_shield3.png"/>
<sld:Format>image/png</sld:Format>
</sld:ExternalGraphic>

<sld:Opacity>
<ogc:Literal>1.0</ogc:Literal>
</sld:Opacity>
<sld:Size>
<ogc:Literal>18.0</ogc:Literal>
</sld:Size>
<sld:Rotation>
<ogc:Literal>0.5</ogc:Literal>
</sld:Rotation>
</sld:Graphic>
</sld:TextSymbolizer>

</sld:Rule>

A railway in Geoserver, you should switch off, "fully SLD schema validator…."


<sld:Rule>
<sld:Name>rule06</sld:Name>
<sld:Title>railway rail</sld:Title>
<sld:Abstract>Abstract</sld:Abstract>
<ogc:Filter>

<ogc:PropertyIsEqualTo>
<ogc:PropertyName>railway</ogc:PropertyName>
<ogc:Literal>rail</ogc:Literal>
</ogc:PropertyIsEqualTo>

</ogc:Filter>
<sld:MinScaleDenominator>150000</sld:MinScaleDenominator>
<sld:MaxScaleDenominator>400000</sld:MaxScaleDenominator>

<sld:LineSymbolizer>
<sld:Stroke>
<sld:CssParameter name="stroke">
<ogc:Literal>#000000</ogc:Literal>
</sld:CssParameter>

<sld:CssParameter name="stroke-linejoin">
<ogc:Literal>round</ogc:Literal>
</sld:CssParameter>
<sld:CssParameter name="stroke-opacity">
<ogc:Literal>1.0</ogc:Literal>
</sld:CssParameter>
<sld:CssParameter name="stroke-width">
<ogc:Literal>4.0</ogc:Literal>
</sld:CssParameter>

</sld:Stroke>
</sld:LineSymbolizer>
<sld:LineSymbolizer>
<sld:Stroke>
<sld:CssParameter name="stroke">
<ogc:Literal>#ffffff</ogc:Literal>
</sld:CssParameter>

<sld:CssParameter name="stroke-linejoin">
<ogc:Literal>round</ogc:Literal>
</sld:CssParameter>
<sld:CssParameter name="stroke-opacity">
<ogc:Literal>1.0</ogc:Literal>
</sld:CssParameter>
<sld:CssParameter name="stroke-width">
<ogc:Literal>3.0</ogc:Literal>
</sld:CssParameter>
<sld:CssParameter name="stroke-dasharray">10 10</sld:CssParameter>

</sld:Stroke>
</sld:LineSymbolizer>

<sld:TextSymbolizer>
<sld:Label>
<ogc:PropertyName>name</ogc:PropertyName>
</sld:Label>
<sld:Font>
<sld:CssParameter name="font-family">
<ogc:Literal>Lucida Sans</ogc:Literal>
</sld:CssParameter>
<sld:CssParameter name="font-size">
<ogc:Literal>10.0</ogc:Literal>
</sld:CssParameter>
<sld:CssParameter name="font-style">
<ogc:Literal>normal</ogc:Literal>
</sld:CssParameter>
<sld:CssParameter name="font-weight">
<ogc:Literal>bold</ogc:Literal>
</sld:CssParameter>
</sld:Font>
<sld:LabelPlacement>
<sld:LinePlacement>
<sld:PerpendicularOffset>
<ogc:Literal>10.0</ogc:Literal>
</sld:PerpendicularOffset>
</sld:LinePlacement>
</sld:LabelPlacement>
<sld:Halo>
<sld:Radius>
<ogc:Literal>2</ogc:Literal>
</sld:Radius>
<sld:Fill>
<sld:CssParameter name="fill">#ffffff</sld:CssParameter>
<sld:CssParameter name="fill-opacity">1.0</sld:CssParameter>
</sld:Fill>
</sld:Halo>
<sld:Fill>
<sld:CssParameter name="fill">
<ogc:Literal>#000000</ogc:Literal>
</sld:CssParameter>
<sld:CssParameter name="fill-opacity">
<ogc:Literal>1.0</ogc:Literal>
</sld:CssParameter>
</sld:Fill>
<sld:VendorOption name="spaceAround">2</sld:VendorOption>
<sld:VendorOption name="group">yes</sld:VendorOption>
</sld:TextSymbolizer>

--
Xiaoyu Guan (Sam)
http://guanxiaoyu.net