Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generating maps issues in mapsforge-map-writer 0.3.0 #275

Closed
mapsforge opened this issue Nov 6, 2014 · 4 comments
Closed

Generating maps issues in mapsforge-map-writer 0.3.0 #275

mapsforge opened this issue Nov 6, 2014 · 4 comments

Comments

@mapsforge
Copy link
Owner

From menion.asamm on March 24, 2012 12:06:13

What steps will reproduce the problem? 1. cut arkhangel_sk (part of Russia) area by polygon from planet file
2. start generating map file

3. even after 10hours, process not yet completed What is the expected output? What do you see instead? generated map What version of the product are you using? On what operating system? new mapsforge-map-writer-0.3.0.jar Please provide any additional information below. additional infos:

  • previous version of map-writter, took less then one hour to generate (used type=hd param)
  • current version stopped after 10 hours (whole time CPU on 100%!), last message "written 70.0% of sub file for zoom 2" ... so writter worked but probably without error (used type=ram param). Tested quickly with type=hd, but after around a hour, no progress.
  • order that started conversion:
    osmosis --read-pbf file=/.../arkhangel_sk.osm.pbf --mapfile-writer file=/.../arkhangel_sk.osm.map type=ram -v
  • if you find useful, poly, osm and pbf file can be downloaded from here http://www.locusmap.eu/_temp/arkhangel_sk_error.zip - this happen to more then this map (actually around 10 discovered). All are "close" to north pole (mainly Canada, some parts of Russia, north of Japan etc...)

also one more issue

  • previous version 0.2.4 used without problem parameter "thread-pool-size=4", but now it do not work with this. Maybe this affect so much speed of generating on Quad-core CPU that it takes more then 10 hours insteadof one (as before)?

feel free to request additional info. I'm of course willing to help, but because I'm not familiar with system of mapsforge-map-writer, I'm firstly writing here

Original issue: http://code.google.com/p/mapsforge/issues/detail?id=275

@mapsforge
Copy link
Owner Author

From jjbross@gmail.com on March 25, 2012 03:12:38

Two quick answers:

  1. be patient, mapsforge-map-writer-0.3.0 may take quite a while for specific regions
    or
  2. update to mapsforge-map-writer-0.3.1-SNAPSHOT, it uses caching and thread pooling

Longer answer:

Since version 0.3.0, we switched back to the Java Topology Suite (JTS) library for all computational geometry tasks. Before that, we had our own, specialized and faster implementations which served most purposes. JTS allows us to accomplish more complex operations at the expense of some performance. But it is not only JTS alone, it is rather the mere fact that we now process more complex polygons which we discarded before.

Some regions have a lot of large, complex polygons (e.g. Sweden, Finland, Russia, Romania, Belarus) and in the current implementation of the writer these large polygons have to be created and processed many times. In version 0.3.1-SNAPSHOT we have added a cache for JTS geometries to relieve that situation. For the earlier mentioned regions cache hit rates of 35% to 50% are observed. We also added a thread pool so that expensive JTS operations can be processed in parallel. Currently the parameter thread-pool-size is not available, the thread pool size corresponds to the number of available processors.

I tested 0.3.1-SNAPSHOT with the supplied arkhangel_sk.osm.pbf and it took less than 10 minutes on a quad core machine (type=ram).

Please let us know if that also worked for you or if you experience any other problems with the updated writer.

Status: Accepted

@mapsforge
Copy link
Owner Author

From menion.asamm on March 25, 2012 03:19:18

Hello,
I'm patient don't worry. This wasn't unpatient report. I'll be very glad to help as much as possible.

You pointed on exactly regions I had to remove from generating process. Belarus took 12 hours, Finland, Sweden etc ... I'll try to update to SNAPSHOT version and will report back soon. Thank you!!

@mapsforge
Copy link
Owner Author

From menion.asamm on March 27, 2012 03:14:11

I'm back .. have to say that very good solution. Some maps still take quite a lot of time, mainly Honshu island of Japan anyway around 14 hours isn't big problem for me, because I'm generating new maps once a month. But just to let you know ...

So thank you very much. Generating is slower, anyway also countries that I wasn't able to generate before, are now completed, so perfect! This is more important

As I wrote before, if you'll want to test generating on some maps, let me know. I can measure some times with different versions or ... whatever ;)

@mapsforge
Copy link
Owner Author

From thilo.mu...@gmail.com on April 01, 2012 04:28:58

I am closing this issue now as it seems to be solved by the improvements in the latest version of the map-writer.

Status: Fixed
Labels: -Type-Defect Type-Enhancement

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant