Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A new way to make maps with OpenStreetMap

protomaps.com

Post reply on HN
Print view View original

91–100 of 102 posts

Re: A new way to make maps with OpenStreetMap

#91

Post by jamesrr39 »

Really interesting project/product! It's really nice to see new ways of rendering OSM data, and alternative map services.

I am developing another way to run your own map server (without mapnik!) from a low-end computer (https://github.com/jamesrr39/ownmap-app in case you're interested), but a little bit different to you in that I'm currently focusing on rendering raster tiles. I'm really happy I saw this thread, however, I hadn't heard of the OSM Express, MBTiles or PMTiles formats before and was/am rolling my own format, so it's really interesting looking at some ideas from these.

Thanks again for sharing your project!

Re: A new way to make maps with OpenStreetMap

#92

Post by c0nsumer »

ourguile wrote: post #11
c0nsumer wrote: post #5

This is really interesting to me. I do a lot of mapping of natural surface single track trails (for hiking and mountain biking) along the way be sure to properly name things, add intersection markers, and create relations (routes and superroutes) to document the systems. The biggest downside thus far is a good way of displaying the data. mtb.waymarkedtrails.org gets close, but I'd really like something that displays…

Have you used GaiaGPS at all? I use it along with ridewithgps for looking at trail conditions and mapping points of interest on my rides and hikes. You can import your own custom map layers as well. The developers are fairly consistently adding new content as well, might not hurt to reach out.

I have! It's great, but it doesn't show route relations for paths.

Re: A new way to make maps with OpenStreetMap

#93

Post by motiejus »

bdon wrote: post #86
bdon wrote: post #85

Earlier quoted context omitted.

Yes, I'll look into this. Can you email me at brandon@protomaps.com or make an issue at http://github.com/protomaps/protomaps.js ? I may need to make a get a sourcemap-enabled build online as well.

I think I've fixed this now, and it exposed some other problems in my code and build process... can you verify everything works? Thanks for reporting!

It worked! Thanks for a swift turnaround.

Re: A new way to make maps with OpenStreetMap

#94

Post by orra »

On the off chance the author is still hanging around here:

Do you mean it when you say on the blog that OSMX is faster than a relational database? e.g. faster than PostGIS.

And I presume you mean because of the up front cost of conversion?

Re: A new way to make maps with OpenStreetMap

#95

Post by iandanforth »

TheTaytay wrote: post #68
iandanforth wrote: post #50

Earlier quoted context omitted.

Towns across the country that meet our criteria overlayed with climate change maps. The existing layers around satellite and topo views are also super helpful to get an idea of if the area has good tree cover and interesting landscapes.

When we were city and house shopping, I kept wanted to be able to query houses for things that the major providers don’t provide query terms for. Even proximity to parks was difficult to discern or searching by walkscore, crime rates, etc, was difficult. Each house has that data, and you can search for some of it visually, but as a developer, you just want to run queries...You seem like the sort of person to crack th…

I am, but you really need to be at business scale to get data access, for an individual doing a 1-off search the economics don't make sense and the incentives of data providers arn't aligned.

That said there is a company (nextburb.com) who's CEO I've been in regular contact with who is trying to dramatically scale what kind of searches people can do. I've been talking to them about our needs and use-cases and they've been quite responsive.

Re: A new way to make maps with OpenStreetMap

#96

Post by bdon »

orra wrote: post #94

On the off chance the author is still hanging around here: Do you mean it when you say on the blog that OSMX is faster than a relational database? e.g. faster than PostGIS. And I presume you mean because of the up front cost of conversion?

I gave a talk on this subject early on in this project, you can watch it here: https://2019.stateofthemap.us/program/sun/osm-express-a-spat...

I personally avoid blanket statements about a certain design being faster than relational databases; for this project as a whole I try to take a "systems programming" approach; I make things faster by doing less work, in this case because I am designing for one specific use case and no more.

For OSMX vs. PostGIS, this entails:

1. PostGIS is a client-server database, OSMX is embedded (in-process). PostGIS must be written defensively for cases like multiple writers; this isn't important for a system that occasionally has a single writer.

2. PostGIS indexes must be built for arbitrary query plans, OSMX only has two kinds of access patterns: by primary key and by S2 cell. S2 cell indexing is an approximation, but will always return more data than you asked for and never less. This is an acceptable downside.

3. PostGIS must treat geodata as OGC Simple Features like points, lines and polygons, this involves lossy conversion from the OSM data model, which is topological.

4. OSMX is built on LMDB, which is an outstanding and underrated piece of database tech (I think the author posts on HN). It's paired with a message format that does not require deserialization (CapnProto) so certain operations common in working with OSM data, such as accessing the coordinates for an OSM node, do not need to malloc for each node. OSM data can be many times larger than RAM; memory-mapping the entire database means you don't need to write your own caching layer; spatial access patterns exhibit high locality since cells are arranged on space-filling curves.

Re: A new way to make maps with OpenStreetMap

#97

Post by bdon »

dvdkon wrote: post #90

In the past I made a map of train lines in and around Prague, which required quite a complex stack, since the information I wanted was in relations. How does Protomaps deal with relations?

It can deal with relations more flexibly than PostGIS-based workflows, because the tiles are generated from a near lossless representation of OSM instead of having to convert to OGC Simple Features first.

Can you email me at brandon@protomaps.com with what specific tags and relations you are interested in? I want to support transit layers as part of basemaps and ideally this is an option when generating a PMTiles archive.

Re: A new way to make maps with OpenStreetMap

#98

Post by orra »

bdon wrote: post #96
orra wrote: post #94

On the off chance the author is still hanging around here: Do you mean it when you say on the blog that OSMX is faster than a relational database? e.g. faster than PostGIS. And I presume you mean because of the up front cost of conversion?

I gave a talk on this subject early on in this project, you can watch it here: https://2019.stateofthemap.us/program/sun/osm-express-a-spat... I personally avoid blanket statements about a certain design being faster than relational databases; for this project as a whole I try to take a "systems programming" approach; I make things faster by doing less work, in this case because I am designing for one specific use ca…

Thank you for the detailed answer.

I have tinkered with PostGIS before, and it seemed great. Nonetheless, you make several good points about how a special purpose tool/format can be faster by avoiding certain work.

As for LMDB, I haven't used it, but I have heard about it previously. A full-text search server called MeiliSearch uses it internally.

---

Separately, I think your broader project is really cool. Simplifying the whole end to end process is incredibly useful. I have previously thought "I would love to host a regional map, but I have no idea how".

Re: A new way to make maps with OpenStreetMap

#99

Post by alwayshumans »

worace wrote: post #30
alwayshumans wrote: post #16

You can already host vector tiles within S3 that work with Mapbox and other libraries. Not really sure what problem this is trying to solve.

The typical way to build and distribute Mapbox vector tiles has been by packing them into a sqlite database with individual rows for each Z/X/Y quad tree coordinate. This is what tools like tippecanoe typically produce. The problem with this is it still requires a running server process colocated with that sqlite db in order to service requests for individual tiles, like what you'll receive from a client-side mapping…

In a typical example isn't this what route 53 solves?

Re: A new way to make maps with OpenStreetMap

#100

Post by hutch120 »

@bdon Awesome! The geospatial industry is in need of simpler tooling for specific jobs such as generating tilesets.

Currently complex tools such as QGIS and GDAL do everything, but the learning curve is very steep.

An addon you might consider a feature to drop an image (and adjust it to fit) over the OSM data to generate a XYZ tilesets directly on the customer AWS S3 (or equivalent storage).

This might be a good way to monetise your offering... I know my customers would buy it.

Post reply on HN