OpenSeaMap Forum

For new registrations: Please create and activate an account. Afterwards send an email to forum.openseamap@gmail.com including a short message and your username to get write access to this forum. Thanks!
Für neue Registrierungen: Bitte erstelle einen neuen Account und schalte ihn frei. Anschließend sende eine kurze Mail mit deinem Nutzernamen an forum.openseamap@gmail.com um Schreibzugriff für dieses Forum zu bekommen. Danke!

You are not logged in.

#31 2016-03-15 22:12:33

gshegosh
Member
From: Szczecin, Poland
Registered: 2016-03-12
Posts: 18
Website

Re: OpenSeaMap symbols

I've also had to remove & from the end of line 43 in tilegen, so it looks like:

java -jar jtile.jar tmp/ tiles/ $z $tx $ty

-- otherwise the script started too many jtile.jar processes at the same time and the memory run out.

I'm generating the tiles now and hopefully no more surprises await smile

Offline

#32 2016-03-15 23:02:57

lepipasd
Member
Registered: 2016-02-16
Posts: 36

Re: OpenSeaMap symbols

3. diff world.osm next.osm | grep id= | grep -v "<tag" > diffs
4. java --jar jsearch.jar ./

This is a part from render script. So in order to make it clear:

cd path to /work
mkdir tmp
mkdir tiles
touch done
touch world.osm

edit tilegen

./render
open a new terminal
./tilegen

and then tiles are stored in work/tiles folder?

I store the tiles of my base map (openstreetmap) at /var/lib/mod_tile. Should I customize tilegen(?) in order to store the tiles generated from searenderer at the same location?

Last edited by lepipasd (2016-03-16 08:46:54)

Offline

#33 2016-03-16 09:23:05

malcolmh
Member
From: Hull, UK
Registered: 2012-09-05
Posts: 194

Re: OpenSeaMap symbols

Notes about the scripts:

1. Another computer (where OverPass is running) pushes next.osm to the work directory & touches work/done as a semaphore to indicate completion. render then processes the diffs using Jsearch & then sends a semaphore back to the OverPass machine. So that part of the render script will need adapting to your particular environment.

2. tilegen & Jtile create the PNG tiles in work/tiles/ and also the work/tmp/*.send files, which are batch files for the SFTP upload. As you will not be uploading, simply modify tilegen to delete the .send files. The .osm * .svg files that appear in work/tmp/ will be automatically deleted.


Malcolm Herring, Developer

Offline

#34 2016-03-16 09:55:54

lepipasd
Member
Registered: 2016-02-16
Posts: 36

Re: OpenSeaMap symbols

"Another computer (where OverPass is running) pushes next.osm to the work directory & touches work/done as a semaphore to indicate completion"

is this part of the code that should be edited? : (I have commented it out)

# while true; do
#       while [ ! -e next.osm ]; do
#    sleep 10
#       done
#       while [ ! -e done ]; do
#    sleep 10
#       done
#       rm done
#       echo "$(date) Parse world data" >> log.txt
#       if [ $(echo $(date "+%M"|sed 's/^0//')) -lt 5 ]; then
#       java -jar jharbour.jar <world.osm >harbours.xml
#         scp -q -C -P 2215 harbours.xml mherring@t1.openseamap.org:~/tiles/
#         echo "$(date) Harbours uploaded" >> log.txt
#       if [ $(echo $(date "+%H"|sed 's/^0//')) -eq 0 ]; then
#         scp -q -C world.osm 195.37.132.70:~/tiles/
#         echo "$(date) World uploaded" >> log.txt
#       fi
#       fi


What is this line doing?  : scp -q hs mini5.local:~/openseamap/work/done


Should I first run ./render and wait until finishes and then run ./tilegen or should I run them simultaneously in different sessions?

"simply modify tilegen to delete the .send files"

gshegosh suggested commented these lines in tilegen, are these lines that you mentioned also?

#      while [ $(ls tmp | grep -c "\.send") -gt 30 ]; do
#        sleep 30
#      done

Last edited by lepipasd (2016-03-16 11:06:27)

Offline

#35 2016-03-16 11:11:30

malcolmh
Member
From: Hull, UK
Registered: 2012-09-05
Posts: 194

Re: OpenSeaMap symbols

"is this part of the code that should be edited? :
if [ $(echo $(date "+%H"|sed 's/^0//')) -eq 0 ]; then
..."

Delete this. This is where I upload world.osm to the tile server at midnight.

""then sends a semaphore back to the OverPass machine"

is this line doing this?  : scp -q hs mini5.local:~/openseamap/work/done"

Yes

"Should I first run ./render and wait until finishes and then run ./tilegen or should I run them simultaneously in different sessions? "

I run both at once as I am processing the minutely updates as they arrive. If you are only doing a once-per-day update of world.osm, then you can run one after the other & stop. You would have to remove the "while true; do ... done" outer loop on both scripts.

""simply modify tilegen to delete the .send files"

gshegosh suggested commented these lines in tilegen, are these lines that you mentioned also?"

You can delete those lines as that is simply a flow control mechanism to prevent the upload backlog getting too large.


Malcolm Herring, Developer

Offline

#36 2016-03-16 11:25:59

lepipasd
Member
Registered: 2016-02-16
Posts: 36

Re: OpenSeaMap symbols

I commented out the following:

1. tilgen script
 
    # while true; do
    #  if [ $(ls tmp | grep -c "\.osm") -eq 0 ]; then
    #    sleep 10
    #  else
    #    while [ -e next.osm ]; do
    #      sleep 10
    #    done
    #         for file in $(ls tmp | grep "\.osm"); do
    #      while [ $(ls tmp | grep -c "\.send") -gt 30 ]; do
    #        sleep 30
    #      done
    #      while [ $(ps -a | grep jtile | grep -c java) -gt 3 ]; do
    #                   sleep 1
    #      done
    #         if [ -e hold ]; then
    #               echo "Holding..."
    #                         while [ -e hold ]; do
    #                               sleep 10
    #                       done
    #                       echo "Resuming"
    #               break
    #       fi

and the done in the end of the script  # done

2. render script

    # while true; do
    #       while [ ! -e next.osm ]; do
    #    sleep 10
    #       done
    #       while [ ! -e done ]; do
    #    sleep 10
    #       done
    #       rm done
    #       echo "$(date) Parse world data" >> log.txt
    #       if [ $(echo $(date "+%M"|sed 's/^0//')) -lt 5 ]; then
    #       java -jar jharbour.jar <world.osm >harbours.xml
    #         scp -q -C -P 2215 harbours.xml mherring@t1.openseamap.org:~/tiles/
    #         echo "$(date) Harbours uploaded" >> log.txt
    #       if [ $(echo $(date "+%H"|sed 's/^0//')) -eq 0 ]; then
    #         scp -q -C world.osm 195.37.132.70:~/tiles/
    #         echo "$(date) World uploaded" >> log.txt
    #       fi
    #       fi

and the two lines in the end of the script

    #       scp -q hs mini5.local:~/openseamap/work/done
    # done

Let me try and I will revert back. Fortunately tiles should be stores in work/tiles...

Offline

#37 2016-03-16 12:05:35

lepipasd
Member
Registered: 2016-02-16
Posts: 36

Re: OpenSeaMap symbols

render script (after commented the lines mentioned in my last post) seems to work ok. after running ./render and wait I get no connection closed. Instead the process finishes and I get a new prompt. In tmp/ files with the extension .osm are created.

In tilegen script I finally commented  the following lines:

# while true; do //in the beginning

//lines 11-13
#      while [ $(ls tmp | grep -c "\.send") -gt 30 ]; do
#        sleep 30
#      done

# done //the last line of the script ("the outer loop that you mentioned")

then run ./tilegen and I have the following output: (.svg files are generated in tmp/ but nothing in tiles/ directory).

What is wrong and no tiles are generated in tiles/ folder?


4)
    at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:3135)
    at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl$PrologDriver.next(XMLDocumentScannerImpl.java:880)
    at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:606)
    at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:118)
    at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:510)
    at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:848)
    at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:777)
    at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141)
    at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1213)
    at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:643)
    at org.apache.batik.dom.util.SAXDocumentFactory.createDocument(Unknown Source)
    at org.apache.batik.dom.util.SAXDocumentFactory.createDocument(Unknown Source)
    at org.apache.batik.dom.svg.SAXSVGDocumentFactory.createDocument(Unknown Source)
    at org.apache.batik.dom.svg.SAXSVGDocumentFactory.createDocument(Unknown Source)
    at org.apache.batik.transcoder.XMLAbstractTranscoder.transcode(Unknown Source)
    at org.apache.batik.transcoder.SVGAbstractTranscoder.transcode(Unknown Source)
    at jtile.Jtile.tile91011(Unknown Source)
    at jtile.Jtile.main(Unknown Source)
Caused by: java.lang.ClassNotFoundException: org.w3c.dom.svg.SVGDocument
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    ... 35 more
Exception in thread "main" java.lang.NoClassDefFoundError: org/w3c/dom/svg/SVGDocument
    at java.lang.ClassLoader.defineClass1(Native Method)
    at java.lang.ClassLoader.defineClass(ClassLoader.java:760)
    at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
    at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
    at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    at org.apache.batik.dom.svg.SVGDOMImplementation.createDocument(Unknown Source)
    at org.apache.batik.dom.util.SAXDocumentFactory.startElement(Unknown Source)
    at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.startElement(AbstractSAXParser.java:509)
    at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.scanStartElement(XMLNSDocumentScannerImpl.java:380)
    at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl$NSContentDriver.scanRootElementHook(XMLNSDocumentScannerImpl.java:61

Here is the log file

Τετ 16 Μάρ 2016 01:37:15 μμ EET New rendering queued
Τετ 16 Μάρ 2016 01:39:05 μμ EET rendering
Τετ 16 Μάρ 2016 01:49:25 μμ EET rendering 11 1041 776
Τετ 16 Μάρ 2016 01:49:25 μμ EET rendering 11 1048 675
Τετ 16 Μάρ 2016 01:49:25 μμ EET rendering 11 1052 663
Τετ 16 Μάρ 2016 01:49:25 μμ EET rendering 11 1052 664
Τετ 16 Μάρ 2016 01:49:26 μμ EET rendering 11 1052 668
Τετ 16 Μάρ 2016 01:49:26 μμ EET rendering 11 1053 663
Τετ 16 Μάρ 2016 01:49:26 μμ EET rendering 11 1053 664
Τετ 16 Μάρ 2016 01:49:26 μμ EET rendering 11 1053 665
Τετ 16 Μάρ 2016 01:49:26 μμ EET rendering 11 1053 676
Τετ 16 Μάρ 2016 01:49:26 μμ EET rendering 11 1054 664
Τετ 16 Μάρ 2016 01:49:27 μμ EET rendering 11 1054 665
Τετ 16 Μάρ 2016 01:49:27 μμ EET rendering 11 1054 666
Τετ 16 Μάρ 2016 01:49:27 μμ EET rendering 11 1054 671
Τετ 16 Μάρ 2016 01:49:27 μμ EET rendering 11 1055 664
Τετ 16 Μάρ 2016 01:49:28 μμ EET rendering 11 1067 691
Τετ 16 Μάρ 2016 01:49:28 μμ EET rendering 11 1069 659
Τετ 16 Μάρ 2016 01:49:28 μμ EET rendering 11 1084 670
Τετ 16 Μάρ 2016 01:49:28 μμ EET rendering 11 1087 665
Τετ 16 Μάρ 2016 01:49:28 μμ EET rendering 11 1094 628
Τετ 16 Μάρ 2016 01:49:29 μμ EET rendering 11 1094 733
Τετ 16 Μάρ 2016 01:49:29 μμ EET rendering 11 1097 731
Τετ 16 Μάρ 2016 01:49:29 μμ EET rendering 11 1101 731
Τετ 16 Μάρ 2016 01:49:29 μμ EET rendering 11 1102 731
Τετ 16 Μάρ 2016 01:49:30 μμ EET rendering 12 1415 2068
Τετ 16 Μάρ 2016 01:49:30 μμ EET rendering 12 1415 2069
Τετ 16 Μάρ 2016 01:49:31 μμ EET rendering 12 1415 2070
Τετ 16 Μάρ 2016 01:49:32 μμ EET rendering 12 1416 2068
Τετ 16 Μάρ 2016 01:49:32 μμ EET rendering 12 1416 2069
Τετ 16 Μάρ 2016 01:49:32 μμ EET rendering 12 1416 2070
Τετ 16 Μάρ 2016 01:49:33 μμ EET rendering 12 1417 2068
Τετ 16 Μάρ 2016 01:49:33 μμ EET rendering 12 1417 2069
Τετ 16 Μάρ 2016 01:49:33 μμ EET rendering 12 1417 2070
Τετ 16 Μάρ 2016 01:49:33 μμ EET rendering 11 1712 940
Τετ 16 Μάρ 2016 01:49:33 μμ EET rendering 9 177 258
and more....

Offline

#38 2016-03-16 12:10:20

gshegosh
Member
From: Szczecin, Poland
Registered: 2016-03-12
Posts: 18
Website

Re: OpenSeaMap symbols

It's more or less the same I did. Instead of commenting most of render and tilegen scripts, I simply run their non-commented parts manually :-)

I'm glad to report that I've managed to render OpenSeaMap tiles at zooms 9-14 (I've modifed JTile and tilegen to skip zooms 15-18 since I have no disk space for them).

I've also managed to render the "background" OpenStreetMap using OSMBright style. If you need any assistance with it, just ask and I'll try to help. To set up PostGIS, I've used https://github.com/wilsaj/everything-is-osm modified to use local file extract instead of downloading it. I've used http://extract.bbbike.org/ to extract part of world file that is only for Baltic Sea region (choose PBF format). I've done what's described at https://switch2osm.org/serving-tiles/bu … -packages/ and built OSMBright using instructions at their manual installation tutorial, because it was missing in packages. I've written a simple Java code to download chosen range of tiles (render_list gave me too much trouble). That's all smile

Offline

#39 2016-03-16 12:14:42

gshegosh
Member
From: Szczecin, Poland
Registered: 2016-03-12
Posts: 18
Website

Re: OpenSeaMap symbols

lepipasd, see build.xml in jtile directory. There's a property:

<property name="batik.dir" value="/work/westernrose/batik-1.7.1/"/>

I've changed its value to point to my copy of batik 1.7.1 which I've downloaded from https://xmlgraphics.apache.org/batik/download.html

To make this change work, run "ant" again in jtile directory and copy the jtile.jar to work.

Offline

#40 2016-03-16 12:20:51

lepipasd
Member
Registered: 2016-02-16
Posts: 36

Re: OpenSeaMap symbols

you downloaded the bin version unzip to some location edit the build.xml and then ant?

Last edited by lepipasd (2016-03-16 12:23:36)

Offline

Board footer

Powered by FluxBB