Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenPlanetData – Free Daily Planet OSM PBF and GOL Indexed Snapshots

openplanetdata.com

Post reply on HN
Print view View original

11–15 of 15 posts

Re: OpenPlanetData – Free Daily Planet OSM PBF and GOL Indexed Snapshots

#11

Post by rustc »

jauntywundrkind wrote: post #9

> Snapshots are proudly hosted on Cloudflare R2, a storage service optimized for fast, global scalable access to large datasets. Little curious to hear how this goes for everyone. R2's promise of infinite egress feels huge. But there's some grumbles in HN threads here & there that folks tend to get forced into more expensive contracts after they actually start using R2. Folks downloading ~90GB data sets daily sounds……

It would be interesting to see how much you can push this. At $0.36/million requests, this would be $0.36 for 90PB of data. Serving 90PB from AWS S3 would cost over $5M if my calculations are correct. Has anyone tried serving petabytes of data from R2?

Re: OpenPlanetData – Free Daily Planet OSM PBF and GOL Indexed Snapshots

#12

Post by Aachen »

morgenkaffee wrote: post #6

Well done. Every conpany with maps otherwise needs their own daily diff server. The challenge tho is running it over years reliably especially if something in upstream OSM breaks.

How else were you going to do it than ingesting the new data constantly, if you're a company who needs regular updates? You need a server, either to pull the whole planet file over and over or to pull the trickle of minute by minute diffs. Might as well opt for diffs

Re: OpenPlanetData – Free Daily Planet OSM PBF and GOL Indexed Snapshots

#13

Post by jt_b »

rustc wrote: post #11
jauntywundrkind wrote: post #9

> Snapshots are proudly hosted on Cloudflare R2, a storage service optimized for fast, global scalable access to large datasets. Little curious to hear how this goes for everyone. R2's promise of infinite egress feels huge. But there's some grumbles in HN threads here & there that folks tend to get forced into more expensive contracts after they actually start using R2. Folks downloading ~90GB data sets daily sounds……

It would be interesting to see how much you can push this. At $0.36/million requests, this would be $0.36 for 90PB of data. Serving 90PB from AWS S3 would cost over $5M if my calculations are correct. Has anyone tried serving petabytes of data from R2?

Not sure about petabytes, but pretty sure Brandon Liu of Protomaps is hosting all the daily protomaps extracts via R2. Not sure what the egress usage is.

Re: OpenPlanetData – Free Daily Planet OSM PBF and GOL Indexed Snapshots

#15

Post by linsomniac »

pbsurf wrote: post #5

This is great! I recently built a simple project to generate and serve vector tiles on demand from a GOL file: https://github.com/styluslabs/geodesk-tiles

Sounds interesting and built up fine, but I had ChatGPT build a leaflet viewer pointing at localhost:8080/tiles/z/x/y and I'm just getting 404s on http://localhost:8080/tiles/6/12/24. A few words in the README about "what to do next" or "how to tell if it's working" would be appreciated, though I realize your target audience is probably more familiar with maps than I am.
Post reply on HN