OpenSeaMap Forum

For new registrations: Please create and activate an account. Afterwards send an email to forum.openseamap@gmail.com including a short message and your username to get write access to this forum. Thanks!
Für neue Registrierungen: Bitte erstelle einen neuen Account und schalte ihn frei. Anschließend sende eine kurze Mail mit deinem Nutzernamen an forum.openseamap@gmail.com um Schreibzugriff für dieses Forum zu bekommen. Danke!

You are not logged in.

#41 2016-03-16 12:23:53

gshegosh
Member
From: Szczecin, Poland
Registered: 2016-03-12
Posts: 18
Website

Re: OpenSeaMap symbols

Yep.

Offline

#42 2016-03-16 13:10:17

lepipasd
Member
Registered: 2016-02-16
Posts: 36

Re: OpenSeaMap symbols

ok! I generated tiles in tiles/ this my folder structure

rnd@Archimidis:~/src/renderer/work/tiles$ ls -la
total 36
drwxrwxr-x  9 rnd rnd 4096 Μάρ  16 14:50 .
drwxrwxr-x  4 rnd rnd 4096 Μάρ  16 14:45 ..
drwxrwxr-x 10 rnd rnd 4096 Μάρ  16 14:46 11
drwxrwxr-x  4 rnd rnd 4096 Μάρ  16 14:46 12
drwxrwxr-x  4 rnd rnd 4096 Μάρ  16 14:46 14
drwxrwxr-x  4 rnd rnd 4096 Μάρ  16 14:47 15
drwxrwxr-x  4 rnd rnd 4096 Μάρ  16 14:47 16
drwxrwxr-x  3 rnd rnd 4096 Μάρ  16 14:49 17
drwxrwxr-x  3 rnd rnd 4096 Μάρ  16 14:50 18
rnd@Archimidis:~/src/renderer/work/tiles$ cd 11
rnd@Archimidis:~/src/renderer/work/tiles/11$ ls -la
total 40
drwxrwxr-x 10 rnd rnd 4096 Μάρ  16 14:46 .
drwxrwxr-x  9 rnd rnd 4096 Μάρ  16 14:50 ..
drwxrwxr-x  2 rnd rnd 4096 Μάρ  16 14:46 1041
drwxrwxr-x  2 rnd rnd 4096 Μάρ  16 14:46 1048
drwxrwxr-x  2 rnd rnd 4096 Μάρ  16 14:46 1052
drwxrwxr-x  2 rnd rnd 4096 Μάρ  16 14:46 1053
drwxrwxr-x  2 rnd rnd 4096 Μάρ  16 14:46 1094
drwxrwxr-x  2 rnd rnd 4096 Μάρ  16 14:46 1101
drwxrwxr-x  2 rnd rnd 4096 Μάρ  16 14:46 1102
drwxrwxr-x  2 rnd rnd 4096 Μάρ  16 14:46 1712
rnd@Archimidis:~/src/renderer/work/tiles/11$ cd 1041
rnd@Archimidis:~/src/renderer/work/tiles/11/1041$ ls -la
total 12
drwxrwxr-x  2 rnd rnd 4096 Μάρ  16 14:46 .
drwxrwxr-x 10 rnd rnd 4096 Μάρ  16 14:46 ..
-rw-rw-r--  1 rnd rnd  757 Μάρ  16 14:46 776.png
rnd@Archimidis:~/src/renderer/work/tiles/11/1041$

I stoped the process because my laptop "freezed" due to lack of ram probably, I have to move away the & in tilegen script as you mentioned

In order to serve these tiles to leaflet what should be done?

The tiles for the base map are located in /var/lib/mod_tile/ ? Should I copy paste the tiles folder under /var/lib/mod_tile/ ?

Offline

#43 2016-03-16 13:33:56

gshegosh
Member
From: Szczecin, Poland
Registered: 2016-03-12
Posts: 18
Website

Re: OpenSeaMap symbols

Assuming you have tile server setup (see switch2osm site for details), under /var/lib/mod_tile is something, but it looks more like a cache. There's render_list (http://www.volkerschatz.com/net/osm/render_list.html) but I wasn't able to figure out how it works -- it didn't save any tiles in the directory I pointed it to. And, it would render all the tiles in the world and I needed x and y subrange only. So I ended up writing a simple Java code (I'll paste it below and also my Leaflet index.html). Java code's dependencies are commons-io and guava; You can download whole project from http://krop.pl/pliki/download-osm-tiles.tar.gz - to build it, use maven (command is mvn clean install) or open the directory in NetBeans (that's how I do it) and run the project. You'll need to change the OSM_TILES_DIR constant to the path where you want your tiles stored. Also see lines 99 and 100 -- there is minimum and maximum X and Y tile coordinate at zoom level 9 set there. You should put minimum and maximum numbers from your openseamap tile directory. So if you have 9/120/271.png, then 120 is X and 271 is Y.

You can also set SEA_TILES_DIR to the directory where your openseamap tiles are and at line 29 change "go" to "go2" -- this will list all existing openseamap tiles and download appropriate openstreetmap tiles. But then you'll have holes since where there are no sea marks, openseamap renders no tile.

If it all sounds complex, give render_list a try, maybe it'll work for you.


As to using Leaflet, it's quite easy. Take my index.html below. Line 28 has "osm/{z}/{x}/{y}.png" -- it's a path (relative to index.html) to openstreet tiles. Line 35 has similarly "sea/{z}/{x}/{y}.png" -- it's a path to opensea tiles. I've put my tiles under "osm" and "sea" directories and next to those directories is index.html. You also need leaflet.css and leaflet.js there. Download them from http://cdn.leafletjs.com/leaflet/v0.7.7/leaflet.js and http://cdn.leafletjs.com/leaflet/v0.7.7/leaflet.css and put next to index.html. It'll work even if you open it in Firefox or Chrome from a file, no need to upload anywhere.



Leaflet index.html:

<!DOCTYPE html>
<html>
<head>
  <title>Western Rose :: Bałtyk</title>
  <meta charset="utf-8" />

  <meta name="viewport" content="width=device-width, initial-scale=1.0">

  <link rel="stylesheet" href="leaflet.css" />
  <style>
  html, body, #mapid {
    width: 100%;
    height: 100%;
    border: 0;
    padding: 0;
    margin: 0;
  }
  </style>
</head>
<body>
  <div id="mapid"></div>

  <script src="leaflet.js"></script>
  <script>

    var mymap = L.map('mapid').setView([53.45, 14.55], 9);

    L.tileLayer('osm/{z}/{x}/{y}.png', {
      minZoom: 4,
      maxZoom: 14,
      attribution: 'Mapa &copy; <a href="http://openstreetmap.org">OpenStreetMap</a>',
      id: 'osm'
    }).addTo(mymap);

    L.tileLayer('sea/{z}/{x}/{y}.png', {
      minZoom: 9,
      maxZoom: 14,
      attribution: 'Warstwa morska &copy; <a href="http://openseamap.org">OpenSeaMap</a>, ' +
        '<a href="http://creativecommons.org/licenses/by-sa/2.0/">CC-BY-SA</a>',
      id: 'sea'
    }).addTo(mymap);


    var popup = L.popup();

    function onMapClick(e) {
      console.log(e);
      popup
        .setLatLng(e.latlng)
        .setContent(e.latlng.toString().replace('LatLng', '').replace('(', '').replace(')', ''))
        .openOn(mymap);
    }

    mymap.on('click', onMapClick);

  </script>
</body>
</html>

Java code to download tiles:

/* (C) 2016 Krop sp. z o.o.   www.krop.pl */
package pl.krop.download.osm.tiles;

import com.google.common.collect.Lists;
import java.io.File;
import java.io.FileNotFoundException;
import java.io.IOException;
import java.net.URL;
import java.util.Collections;
import java.util.LinkedList;
import java.util.List;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import org.apache.commons.io.FileUtils;

/**
 *
 * @author Grzegorz Krugły <gk@krop.pl>
 */
public class DownloadOsmTiles {
    private static final String SEA_TILES_DIR = "/work/westernrose/000tiles/sea/";
    private static final String OSM_TILES_URL = "http://localhost/osm/%s";
    private static final String OSM_TILES_DIR = "/work/westernrose/000tiles/osm/";
    
    /**
     * @param args the command line arguments
     */
    public static void main(String[] args) throws IOException {
        new DownloadOsmTiles().go(args);
    }

    private void go2(String[] args) throws IOException {
        List<String> seaTiles = findTiles(SEA_TILES_DIR);
        Collections.sort(seaTiles);
        
        List<List<String>> partitions = Lists.partition(seaTiles, seaTiles.size() / 8);
        
        for (List<String> partition : partitions) {
            System.out.println("Created thread");
            Thread t = new Thread() {
                @Override
                public void run() {
                    try {
                        download(partition);
                    } catch (IOException ex) {
                        ex.printStackTrace();
                    }
                }
            };
            t.start();
        }
    }

    private void download(List<String> seaTiles) throws IOException {
        String path;
        String url;
        String dest;
        File destFile;
        int size = seaTiles.size();
        double percent;
        for (int i = 0; i < size; i++) {
            String seaTile = seaTiles.get(i);
            path = seaTile.replace(SEA_TILES_DIR, "");
            url = String.format(OSM_TILES_URL, path);
            dest = OSM_TILES_DIR + path;
            
            try {
                destFile = new File(dest);
                percent = 100.0 * i / size;
                if (!destFile.exists()) {
                    FileUtils.copyURLToFile(new URL(url), destFile);
                    System.out.println(String.format("%.2f%%  --  Copied %s to %s", percent, url, dest));
                } else {
//                    System.out.println(String.format("%.2f%%  --  Exists: %s", percent, dest));
                }
            } catch (FileNotFoundException ex) {
                System.out.println(String.format("!!!!!!!!!!! Doesn't exist: %s", url));
            }
        }
    }

    private List<String> findTiles(String dir) throws IOException {
        File[] files = new File(dir).listFiles();
        List<String> list = new LinkedList<>();
        for (File file : files) {
            if (".".equals(file.getCanonicalPath()) || "..".equals(file.getCanonicalPath())) continue;
            if (file.isDirectory()) {
                list.addAll(findTiles(file.getCanonicalPath()));
            } else {
                list.add(file.getCanonicalPath());
            }
        }
        return list;
    }

    private void go(String[] args) throws IOException {
        ExecutorService executorService = Executors.newFixedThreadPool(8);
        
        for (int x = 265; x <= 298; x++) {
            for (int y = 146; y <= 166; y++) {
                final int xx = x;
                final int yy = y;
                
                executorService.execute(new Runnable() {
                    @Override
                    public void run() {
                        try {
                            osmDownload(xx, yy);
                        } catch (IOException ex) {
                            ex.printStackTrace();
                        }
                    }
                });
            }
        }

        executorService.shutdown();
    }

    private void osmDownload(int tx, int ty) throws IOException {
        String path;
        String url;
        String dest;
        File destFile;
        
        for (int zoom = 0; zoom <= 14; zoom++) {
            int x1 = (int) (tx * Math.pow(2.0, zoom - 9));
            int y1 = (int) (ty * Math.pow(2.0, zoom - 9));
            int x2 = x1 + (int) Math.pow(2.0, zoom - 9) - 1;
            int y2 = y1 + (int) Math.pow(2.0, zoom - 9) - 1;
            if (x2 < x1) x2 = x1;
            if (y2 < y1) y2 = y1;

            for (int x = x1; x <= x2; x++) {
                for (int y = y1; y <= y2; y++) {
                    path = String.format("%d/%d/%d.png", zoom, x, y);
                    
                    url = String.format(OSM_TILES_URL, path);
                    dest = OSM_TILES_DIR + path;

                    try {
                        destFile = new File(dest);
                        if (!destFile.exists()) {
                            FileUtils.copyURLToFile(new URL(url), destFile);
                            System.out.println(String.format("Copied %s to %s", url, dest));
                        } else {
//                            System.out.println(String.format("Exists: %s", dest));
                        }
                    } catch (FileNotFoundException ex) {
                        System.out.println(String.format("!!!!!!!!!!! Doesn't exist: %s", url));
                    }
                }
            }
        }
    }
}

Offline

#44 2016-03-16 13:36:00

gshegosh
Member
From: Szczecin, Poland
Registered: 2016-03-12
Posts: 18
Website

Re: OpenSeaMap symbols

To be more clear -- packages installed via PPA mentioned at https://switch2osm.org/serving-tiles/bu … -packages/ serve tiles from localhost and my Java code "downloads" them from there. For PPA packages to serve tiles, I had to do step "Stylesheet configuration" from https://switch2osm.org/serving-tiles/ma … ver-14-04/ and change /etc/renderd.conf to point to custom-built OSMBright.xml (XML= under [default])

When building OSMBright.xml, their advice to apt-get install node-carto was wrong for me because I have newer node.js than the one in repositories. If it bites you, too, just install node-carto using npm instead of apt-get.

Also be advised that my Java code assumes 8-core CPU (or 4-core with HT) so if you don't have an Intel i7 it might make the CPU hot as it'll run 8 threads simultanously (or change "8" to something different at lines 36 and 97).

Last edited by gshegosh (2016-03-16 13:41:52)

Offline

#45 2016-03-16 13:50:22

lepipasd
Member
Registered: 2016-02-16
Posts: 36

Re: OpenSeaMap symbols

I have also followed the switch2osm tutorial for ubuntu 14.04 and managed to produce a base map with openstreetmap-carto style. I have also used leaflet in order to display the base map in the browser.

Thanks for the information and the code

"You can download whole project from http://krop.pl/pliki/download-osm-tiles.tar.gz - to build it, use maven (command is mvn clean install)"

I understand the part of editing the code but I am not sure I know how to run your code afterwards...

any suggestions? Just run your code as I would run any java project? The idea is that after editing download-osm-tiles, one should run your code and the

openseamap tiles will be "downloaded" to the directory specified during editing? Should I run  download-osm-tiles or just build it and then run your code which will

do all the magic?

Last edited by lepipasd (2016-03-16 14:50:46)

Offline

#46 2016-03-16 15:45:30

lepipasd
Member
Registered: 2016-02-16
Posts: 36

Re: OpenSeaMap symbols

check also this post about render_list

http://comments.gmane.org/gmane.comp.gi … evel/27665

I am a little bit confused... Please help me to make it clear

Renderd.conf uses mapnik library according to an .xml style and produces tiles which are then served to the client via mod_tile.

We have already produce tiles from openseamap renderer. So why not store them under /var/lib/mod_tile/ ? In that way mod_tile will serve them (?) to the client?

The rendering process has already be done as it concerns seamarks. It is like having all of them cached, isn't it?

Why the need of the "downloading" process through your custom java code?

Last edited by lepipasd (2016-03-16 16:07:22)

Offline

#47 2016-03-16 20:08:48

lepipasd
Member
Registered: 2016-02-16
Posts: 36

Re: OpenSeaMap symbols

gshegosh,

I have followed this guide from switch2osm (manually building a tile server)

https://switch2osm.org/serving-tiles/ma … ver-14-04/

now I am reading your guide, there is some more info ...

I run tilegen for about 7 hours and still running!!

version: Intel(R) Core(TM) i5-5200U CPU @ 2.20GHz
4GB RAM

How much time did it take on your machine? CPU? RAM?

Last edited by lepipasd (2016-03-16 21:45:47)

Offline

#48 2016-03-16 22:06:44

gshegosh
Member
From: Szczecin, Poland
Registered: 2016-03-12
Posts: 18
Website

Re: OpenSeaMap symbols

It took about 3 hours to generate OpenSeaMap tiles and about 4 hours to render OpenStreetMap ones. I have 4th gen i7 with 16GB RAM and a fast SSD drive (I/O is most important for database and it was the most busy component).

But I only rendered Baltic Sea -- the PBF file was less than 1GB.


I didn't want to use all this infrastructure to serve tiles dynamically. My use case is that I serve these maps from Raspberry Pi so there's no computing power to render tiles. I've prerendered them all, dropped them as static files in a folder and serve them with nginx. So, no mod_tile et all on my target.

For sure you can just use mod_tile, renderd and mapnik to serve them on demand. It'll save you quite a lot of disk space, but it's not that performant.

OpenSeaMap is a separate, transparent layer and you have it rendered statically. I have no idea how to tell mod_tile to serve it. I don't really see the need. Just configure Apache to serve OSeaMap tiles as static file, using Location directive.

Offline

#49 2016-03-16 22:09:11

gshegosh
Member
From: Szczecin, Poland
Registered: 2016-03-12
Posts: 18
Website

Re: OpenSeaMap symbols

Also, make sure new tiles are still rendered and showing up. After it was all rendered it still looped in the tilegen, waiting for another next.osm or something like that. I just pressed ctrl-c when I noticed all tiles are rendered already.

Offline

#50 2016-03-16 23:34:51

lepipasd
Member
Registered: 2016-02-16
Posts: 36

Re: OpenSeaMap symbols

For the OpenSeaMap tiles did you use tiles.openseamap.org/seamark/world.osm or the area of your map only?

Offline

Board footer

Powered by FluxBB