Live data from Hacker News

Show HN: OpenFreeMap – Open-Source Map Hosting

openfreemap.org

101–110 of 243 posts

Re: Show HN: OpenFreeMap – Open-Source Map Hosting

#102

Looks great love the free tiles and no limits or API keys. Question, why use btrfs?

So a full planet OSM extract is about 300 million binary files, in total about 90 GB. The most popular ways to store it: - MBTiles - an SQLite file, each file is a row in a table, you need a server to serve it. - PMTiles - a single file, optimised for serverless usage. - Extract them into a directory, which in practice should be on a partition image. This is the approach I choose. I tested ext4 and Btrfs and I choose…

I apologize if this is something stupid, or answered elsewhere, but is there a good way to trim Planet down to a smaller area?

I've been making some custom map stuff for mountain bike trails and I'd really like to move to self-hosted vector tiles for all layers, but too much of what I find says to start with Planet.osm when all I really need is a State (in the US) or even a few-miles-wide area.

(My goal is to basically snapshot OSM data, generate tiles, and use that until I decide to do another snapshot down the line, so the underlying data doesn't change. And limit it to a small area because that's all I need.)

Examples of maps I've done this way, and want to improve, are: https://trailmaps.app/ramba/ and https://trailmaps.app/dte/

Re: Show HN: OpenFreeMap – Open-Source Map Hosting

#105
post #98

Earlier quoted context omitted.

Oh, there is another pal who's streaming making his own maps, fairly entertaining: https://youtube.com/playlist?list=PL980gcR1LE3L8RoIMSNBFfw4d...

Thanks ! I've thought about making an "education" project but it takes time, and after 9 months of working on it I'm now convinced it's possible to build a basic Gmaps alternative in a few years. Business model still missing though ! Edit : just watched some minutes, he's rebuilding low-level components, like route finding. Impressive stuff but not quite user-friendly.

> not quite user-friendly

Yes, it does not seem like he makes a business or a product for a wide audience. It's more about exploring the unknown, teaching programming and being entertaining.

Although I adore such ambitious projects that seem insurmountable at the first glance.

Re: Show HN: OpenFreeMap – Open-Source Map Hosting

#106

Looks great love the free tiles and no limits or API keys. Question, why use btrfs?

So a full planet OSM extract is about 300 million binary files, in total about 90 GB. The most popular ways to store it: - MBTiles - an SQLite file, each file is a row in a table, you need a server to serve it. - PMTiles - a single file, optimised for serverless usage. - Extract them into a directory, which in practice should be on a partition image. This is the approach I choose. I tested ext4 and Btrfs and I choose…

This sounds like a perfect application for EROFS[1]. While it comes from an embedded systems background, it has seen some usage in container use cases and is moving towards a general "mountable tar" application. It would also avoid the tedium you have to go through in shrink_btrfs.py because you can just generate the image out of a tree.

I wanted to give repackaging the btrfs image a shot but the download was pretty slow - I assume your server is getting HN-hugged a bit so I didn't want to make it worse and stopped the download.

[1] https://erofs.docs.kernel.org/en/latest/index.html

Re: Show HN: OpenFreeMap – Open-Source Map Hosting

#107
This is really cool, I'll definitely look at this if I need to embed a map in the future. Are there any libraries that lets you plot geographic data on top of this in Python, with Matplotlib or similar?

Slightly off topic, but how come there doesn't seem to be any open projects using the Overture maps data?

> Overture is a data-centric map project, not a community of individual map editors. Therefore, Overture is intended to be complementary to OSM. We combine OSM with other sources to produce new open map data sets. Overture data will be available for use by the OpenStreetMap community under compatible open data licenses. Overture members are encouraged to contribute to OSM directly.

Sounds like it would be a good data source, or am I missing something?

https://overturemaps.org/about/faq/

Re: Show HN: OpenFreeMap – Open-Source Map Hosting

#108

Earlier quoted context omitted.

So a full planet OSM extract is about 300 million binary files, in total about 90 GB. The most popular ways to store it: - MBTiles - an SQLite file, each file is a row in a table, you need a server to serve it. - PMTiles - a single file, optimised for serverless usage. - Extract them into a directory, which in practice should be on a partition image. This is the approach I choose. I tested ext4 and Btrfs and I choose…

Do you think btfs would work well for old school pre rendered raster tiles on disk?

Yes, I think it would. Have a look at the extract_mbtiles script: https://github.com/hyperknot/openfreemap/blob/main/modules/t...

Re: Show HN: OpenFreeMap – Open-Source Map Hosting

#109
post #88

Related : I'm building https://github.com/laem/cartes , an alternative to Google Maps. On top of a custom protomaps tileset and a few other MapTiler options (such as satellite and hiking), it packs a search engine, a basic OSM place UI, transit calculators (walk, bike with profiles, transit, car), small features like ruler and favorites, transit maps, photos of places, place search by categories, and the French open…

Wow, now that's an ambitious project! I wish you a lot of success with it!

Re: Show HN: OpenFreeMap – Open-Source Map Hosting

#110
post #56

Hard links are to deduplicate identical tiles, right? How much does that save and how close is the tile with most links (empty one(s)?) to the filesystem limits? Inquiring minds want to know...

Yes, exactly, it's to save space. For example there is one such tile for "ocean", which covers 70% of earth. In total, there are 271 million hard links. So out of 300 million files 271 are hard links! The file system limit is 64k hard-links for the same file, so I have to handle the case when it's reached and then start a new file for the next 64k.

> The file system limit is 64k hard-links for the same file

I had never heard that, so I went sniffing around and it seems to be ext4 specific[1] but I wasn't able to easily get the limits for ZFS (or xfs, etc), so depending on how much glucose one wished to spend it may be better to use a different FS than all that renaming work around

1: https://en.wikipedia.org/wiki/Hard_link#Limitations

Post reply on HN