A new way to make maps with OpenStreetMap • Hacker News Board

Re: A new way to make maps with OpenStreetMap

#31

Post by jleedev » Fri, Apr 23, 2021, 10:57 PM UTC

Tomte wrote: post #2

> Existing formats like MBTiles are based on SQLite. This limits the ability to self-host maps to those confident running a server in production. Is that comical, or is there a deeper meaning I don't understand?

For comparison, Cloud Optimized GeoTIFF (COG) is specifically designed to be readable with HTTP Range requests, so the single file can be placed on S3 and be immediately usable.

As others pointed out, the MVTs inside can be unpacked for static serving. But a SQLite database is not itself usable in this way.

Re: A new way to make maps with OpenStreetMap

#32

Post by MrMartian » Fri, Apr 23, 2021, 11:00 PM UTC

Tomte wrote: post #2

> Existing formats like MBTiles are based on SQLite. This limits the ability to self-host maps to those confident running a server in production. Is that comical, or is there a deeper meaning I don't understand?

So I fee like it actually missed the mark as to why it's so impactful. When you get to global scale tiling with building accuracy, you literally end up with hundreds of millions of tiles. The upload time / cost per upload STACKS. However, the tilesets combined usually sits between 100GB and 200GB. ONE upload of ~150GB seems is faster and more cost efficient via S3.

Re: A new way to make maps with OpenStreetMap

#33

Post by ed25519FUUU » Fri, Apr 23, 2021, 11:14 PM UTC

What’s the easiest way to self-host OSM data to use like a offline google maps?

Re: A new way to make maps with OpenStreetMap

#34

Post by brunoqc » Fri, Apr 23, 2021, 11:22 PM UTC

kennydude wrote: post #8

brunoqc wrote: post #6

> Pricing > coming soon...

At least it's open source and self hostable so if it's too expensive for you, you can host it yourself

Ah fair enough.

Re: A new way to make maps with OpenStreetMap

#35

Post by tppiotrowski » Fri, Apr 23, 2021, 11:25 PM UTC

TurkishPoptart wrote: post #28

tppiotrowski wrote: post #20

Super excited for your project. I have been using map tiles from Mapbox, MapTiler, OSM Buildings for my project and am coming to the point that I need to merge elevation data with building height data in order to cast realistic shadows across the earth. Unfortunately elevation data and building data come from two different tile sets from two different tile providers and it's a lot of download/processing overhead to m…

What kind of project can this tool be used for?

I think the author would explain best. Personally, I'm working on casting live earth shadows on a tiled map (link in HN profile) and lack an off-the-shelf tile set that contains all the data I need and in some cases the available tiles I use contain more data than I want. Also, the tiles cost $$$ for each tile load.

So long-term I am looking into generating and hosting my own map tiles.

Re: A new way to make maps with OpenStreetMap

#36

Post by d-yakovlev » Fri, Apr 23, 2021, 11:30 PM UTC

gorbypark wrote: post #26

xrd wrote: post #17

Earlier quoted context omitted.

Is this a simple process? Can you share a link to how this is done, I'm interested. I always assumed it was fairly complicated.

I’ve used the utility tippecanoe with success to generate vector tiles from geojson sources and am hosting them statically. It’s been overall pretty easy. The only gotcha I can remember is I needed to pass the no compression flag since mapbox gl can’t read compressed tiles (and the file host will gzip everything anyways).

What issues have you run into with gzipped tiles? I store a lot of gzipped individual tiles for use in Mapbox GL maps on S3 with no problem - the only necessary change was making sure that S3 was emitting `content-encoding` headers.

Re: A new way to make maps with OpenStreetMap

#37

Post by jhgb » Fri, Apr 23, 2021, 11:33 PM UTC

tppiotrowski wrote: post #20

Super excited for your project. I have been using map tiles from Mapbox, MapTiler, OSM Buildings for my project and am coming to the point that I need to merge elevation data with building height data in order to cast realistic shadows across the earth. Unfortunately elevation data and building data come from two different tile sets from two different tile providers and it's a lot of download/processing overhead to m…

If you need to create your own tiles, I'm not sure how this changes anything. I can't seem to find how one would go about generating one's own tiles for this platform in their documentation. Did you find something to that effect in it?

Re: A new way to make maps with OpenStreetMap

#38

Post by bdon » Fri, Apr 23, 2021, 11:45 PM UTC

jhgb wrote: post #37

tppiotrowski wrote: post #20

Super excited for your project. I have been using map tiles from Mapbox, MapTiler, OSM Buildings for my project and am coming to the point that I need to merge elevation data with building height data in order to cast realistic shadows across the earth. Unfortunately elevation data and building data come from two different tile sets from two different tile providers and it's a lot of download/processing overhead to m…

If you need to create your own tiles, I'm not sure how this changes anything. I can't seem to find how one would go about generating one's own tiles for this platform in their documentation. Did you find something to that effect in it?

Author here. You should be able to first download OpenStreetMap tiles in PMTiles format (http://github.com/protomaps/pmtiles) and then write out a new augmented tileset in the same format. It's a use case I want to have first-class support for, but the tooling doesn't exist yet.

Re: A new way to make maps with OpenStreetMap

#39

Post by bdon » Fri, Apr 23, 2021, 11:50 PM UTC

c0nsumer wrote: post #5

This is really interesting to me. I do a lot of mapping of natural surface single track trails (for hiking and mountain biking) along the way be sure to properly name things, add intersection markers, and create relations (routes and superroutes) to document the systems. The biggest downside thus far is a good way of displaying the data. mtb.waymarkedtrails.org gets close, but I'd really like something that displays…

Author of OP here. Is this information that you're interested in contributing, and is appropriate for, OpenStreetMap?

The end goal is to build a system that lets you customize the tileset to include any node, way, relation, or tag in OSM. This gets quite tricky when considering automated cartographic generalization (how such features will appear when zoomed out) but it's one of my next steps.

Re: A new way to make maps with OpenStreetMap

#40

Post by bdon » Fri, Apr 23, 2021, 11:56 PM UTC

incanus77 wrote: post #13

Tomte wrote: post #2

> Existing formats like MBTiles are based on SQLite. This limits the ability to self-host maps to those confident running a server in production. Is that comical, or is there a deeper meaning I don't understand?

Originally, MBTiles was designed for mobile offline. Then, Mapbox started serving directly from the SQLite for web & other clients, creating the Node.js SQLite bindings in the process. Eventually, they had an unpacker for MBTiles uploads that would stick the tiles in CDN. So, really no more complex than old school static content hosting. Source: I created MBTiles ;-)

Author of OP here, I use the tile unpacker as well! I didn't point this out in the OP, but I found the ergonomics of uploading tiles to S3 etc get bad once you're over a couple tens of thousands of tiles, as each upload has some overhead. I've tested PMTiles on S3 successfully with hundreds of millions of tiles.

Deduplication is also important to save space on redundant ocean/earth tiles; MBTiles can deduplicate via views, PMTiles deduplicates by reference, but unpacked Z/X/Y tiles on S3 can't.