71–80 of 169 posts
I wonder if Github Pages supports HTTP range. Obviously putting a multi-GB blob on there would probably be abuse, but a small blob limited to a small region with limited detail would be really nice for having a map to accompany a blog post or something.
It does, but there is a limit of something like 500 MB for single files.
Earlier quoted context omitted.
Pmtiles are the vector counterpart to Cloud Optimized GEOTIFF (COG), allowing for efficient usage of mapping info from clients when the server supports a http range request. Previous iterations of this on the vector side have either been a ton of small files(pbf) or a large file that needed a front end to serve the individual tiles (mbtiles). At this point, you can take an OSM dump and convert it to a country level b…
Generating country level basemaps from scratch is a great approach! I've made an even easier path which is to just extract the relevant tiles from a daily build: https://docs.protomaps.com/guide/getting-started This can take seconds to minutes for small-medium areas and doesn't need a powerful computer at all. The drawbacks are that the build is only daily, and the lower-zoom tiles aren't clipped so include extra inf…
Earlier quoted context omitted.
It does, but there is a limit of something like 500 MB for single files.
It would be cool to have an automatic tool to generate a blob of any size and it would include as much detail as possible within your given blob size limit...
pmtiles extract INPUT.pmtiles OUTPUT.pmtiles --bbox=BBOX --maxzoom=Z --dry-run
--dry-run will output the total size of the final archive without downloading any tiles, so you can adjust the maxzoom from the default of 15 (for basemaps) until it fits under your limit
Earlier quoted context omitted.
Your best bet for this is something like OsmAnd or organic maps, which download countries worth of osm data and make it available offline. The amount of map data for any reasonable area is going to be too large for a PWA, though it would technically be possible to do with existing map viewers and this format. (Eg maplibre or leaflet)
Thanks. I'm OK with a single layer just showing the outlines of coastlines and regions, including the names of regions. I think a suitable simplified world map could fit all this in SVG. edit: Initially underestimated so did some calculations below Key Data Points for Estimate of SVG Size - Radius of Earth: 6371 km - Surface Area of Earth: 510e6 km^2 - Approximate Number of Countries: 250 - Land Area of Earth: 29% of…
116KB - 5MB for country borders
16MB - 52MB for ~50K city/county level borders based on geoBoundaries
The range of sizes depends on how much custom compression/simplification you put into it. The source files are about 10x bigger, but that's already pretty small.
Topojson might be even smaller though.
Check the repo for details /selfplug https://github.com/SmilyOrg/tinygpkg-data
If anyone else was curious, their sample OSM derived PMTile file (protomaps-basemap-opensource-20230408.pmtiles) is 103GB. https://docs.protomaps.com/basemaps/downloads
It's 107 GB now. It would be great to be able to download only diffs and apply them to a daily, weekly, monthly baseline.
Earlier quoted context omitted.
Your best bet for this is something like OsmAnd or organic maps, which download countries worth of osm data and make it available offline. The amount of map data for any reasonable area is going to be too large for a PWA, though it would technically be possible to do with existing map viewers and this format. (Eg maplibre or leaflet)
Thanks. I'm OK with a single layer just showing the outlines of coastlines and regions, including the names of regions. I think a suitable simplified world map could fit all this in SVG. edit: Initially underestimated so did some calculations below Key Data Points for Estimate of SVG Size - Radius of Earth: 6371 km - Surface Area of Earth: 510e6 km^2 - Approximate Number of Countries: 250 - Land Area of Earth: 29% of…
FWIW, I've got an official 900MB geojson of borders from a large unnamed multinational NGO type that's particular about their mapping.
At z=4, or 600m resolution, it's a 2.5MB pmtiles, at z=10 (10m resolution) it's 83M.
That's just the ADM0 borders, adding local borders would obviously increase the size.
Earlier quoted context omitted.
It's 107 GB now. It would be great to be able to download only diffs and apply them to a daily, weekly, monthly baseline.
I wonder if the compressed Hilbert ordering inside the tiles make them hard to diff efficiently.
For me the most interesting part is “PMTiles”: > PMTiles is a single-file archive format for pyramids of tiled data. A PMTiles archive can be hosted on a storage platform like S3, and enables low-cost, zero-maintenance map applications. > PMTiles is a general format for tiled data addressed by Z/X/Y coordinates. This can be cartographic basemap vector tiles, remote sensing observations, JPEG images, or more. > PMTile…
We were taking daily CSV files, turning them into vector tiles, syncing them with S3, and displaying in mapbox-gl map. The biggest cost was S3 file operations. We had to aggressively cluster to reduce our cost down and got to a point where the tiles numbered ~75,000. At the frequency we wanted to update this was costing $500/mo in S3 ops.
Now we process the files into a single PMTiles file, deploy that one file and cut the costs down to pennies.
For a map that we make public, it was incredibly enabling. You can checkout the Canadian hotspot data for all of 2023 if you're curious: https://lens.pathandfocus.com
I've been an avid Mapbox user since getting alpha access to Tilemill. From my perspective, this is the most important contribution to the open mapping space since the introduction of vector tiles and Mapbox GL / Map GL. Mapbox in my opinion left the door open in how they approached tile baking with MTS. While powerful it was way to confusing and expensive. While you could always bake tiles with Tippecanoe you still h…
It's kinda sad how we need open source alternatives to MapBox, the formerly open source company.
Mapbox is a venture-backed company with a SaaS business model, and has never been open source in total - it used to be open core with a FOSS frontend and proprietary backend. This SaaS model is absolutely the best way to fund huge companies and give investors a return. Mapbox has also done the bulk of innovation in open source web mapping over the past 10 years - the Protomaps project uses MapLibre (fork of Mapbox GL 1.0) and the MVT tile format. Both required teams of full-time developers - easily tens of millions in salaries and stock - and they have given away version 1.0 for free as a gift, even though 2.0 is not open source.
The ideal software economy is one in which innovators capture a good portion of the wealth they create. This is why it's important for Protomaps to focus on use cases underserved by SaaS, instead of just being a cheaper map API. The sibling comment on wildfire mapping https://news.ycombinator.com/item?id=37989059 is a good example of the applications I want the project to support.
For me the most interesting part is “PMTiles”: > PMTiles is a single-file archive format for pyramids of tiled data. A PMTiles archive can be hosted on a storage platform like S3, and enables low-cost, zero-maintenance map applications. > PMTiles is a general format for tiled data addressed by Z/X/Y coordinates. This can be cartographic basemap vector tiles, remote sensing observations, JPEG images, or more. > PMTile…