211–220 of 243 posts
Very nice! Curious that you describe the OSM Bright style as "abandoned by their upstream project". I see edits 4 months old ( https://github.com/openmaptiles/osm-bright-gl-style ), but I only looked superficially. Is it really abandoned?
Earlier quoted context omitted.
Thank you for sharing the story! I hope OpenFreeMap can contribute either with the MBTiles / Btrfs dumps, or with the public instance.
Thanks for offering the service! The intensity of map tile pricing is always a little baffling to me. For example, with mapbox this month we (got linked to on The Verge and) exceeded the free tier on the 22nd. Two days later and we have a $40 bill and counting, with a week left in the month. We're a free, open source project and can't handle dynamic pricing like this. My only feedback on your tiles is the lack of POI…
For POIs, they are present in the data, it's just that the styles might not have the matching icons for them. But you can always make them into a circle, etc.
Having said that, working on the styles is a big and never ending effort, so if someone would like to help with matching the POIs then contributors welcome.
Earlier quoted context omitted.
You both specify the filesystem to be Btrfs. Is there any advantage in this case against ZFS, ext4, XFS... or is it just a practical choice?
Yes, small files can fit in the metadata, which makes a super big difference when you have 300 million files of 405 bytes each. Also, inode handling is way better compared to ext4.
The nearest thing (and it's not really really similar, just related) that ZFS has is special VDEVs.
When you have an array of disks (usually "slow", like regular HDDs) you can attach to it another array of disks (usually very fast, like NVMe) where you can store metadata (file information) and optionally small files up to a size that you can define.
So for example you have a 50TB (or 500, who knows) array of SATA disks, and a small but superfast array of NVMe drives. Lets say 128GB. Or 512, or 1TB depending on you want to do.
File metadata is saved there, so doing a find, ls, tree... operation is now very fast. And if you save, for example, all files smaller than 32KB there (it will depend on your needs, also) all the small file operations will be way faster.
Earlier quoted context omitted.
As a hobby historian (not really haha) I'm interested in old streets and names particularly. Especially here in Europe / Germany, where entire quarters where bombed to rubbles newer streets do not match neither namely nor spatially. I loved how you could "move back in time" in street view. I think that has been killed too? There is a lovely Twitter/X account for Detroit tho: https://x.com/DetroitStreetVu
You can still go back in time on Street View. There's a "See more dates" link next to the address info in the top left (at least on the desktop Maps interface) that when you click it opens a film strip of different dated captures for that location. Here's the 2007 capture of 1 Embarcadero (outside the Ferry Building in San Francisco) for example: https://maps.app.goo.gl/ArrucFgus9uMvdSaA
It's just that in Europe the data is much more sparse to begin with. In many places Google didn't go multiple times, so there just can't be older data. On some streets however I see that if the data was really old (before ~2010/2012) Google probably decided that the quality was so bad – well, it really is – that they do not make it available. So even if there were multiple passes it is not a must that this feature comes available.
Earlier quoted context omitted.
As long as it stays with one server that’s fine. But what do you do in the hypothetical scenario that Facebook decides to use your tile server? Nginx is fast, but I think that’s asking too much of it. Of course that problem hasn’t happened yet, but it’s what I’d be worried about.
I benchmarked and nginx can host 30 Gbps on cold cache, on actual real world load. (I played back a log of 1 million real world requests). So using nginx in a 1 Gbps server is definitely not asking too much of it, or I might not have understood your comment clearly. Why would Facebook decide to use my tile server? When a company's business depends on having reliable map tiles, they either self host or have an Enterpr…
It’s interesting to know that the limiting factor is bandwidth and not nginx/disk speed though. I’ll admit I don’t see 30Gbit of map tiles happening any time soon.
Earlier quoted context omitted.
I benchmarked and nginx can host 30 Gbps on cold cache, on actual real world load. (I played back a log of 1 million real world requests). So using nginx in a 1 Gbps server is definitely not asking too much of it, or I might not have understood your comment clearly. Why would Facebook decide to use my tile server? When a company's business depends on having reliable map tiles, they either self host or have an Enterpr…
I just used it as an example of some large service. The point is that I expect someone will take advantage. It’s interesting to know that the limiting factor is bandwidth and not nginx/disk speed though. I’ll admit I don’t see 30Gbit of map tiles happening any time soon.
Earlier quoted context omitted.
The latency for small files and ranges of large files is pretty similar on most storage platforms, but there are some exceptions like Cloudflare R2. The main reason PMTiles is one file and not two or more files is that it enables atomic updates in-place (which every mature storage platform supports) as well as ETag content change detection in downstream caches. All of the server and serverless implementations at http…
Now I'm curious, what causes the latency for range requests with R2?
Also, I'm pretty sure range requests are going to be difficult to cache. That implies going to origin every request which is bad.
Earlier quoted context omitted.
At Hetzner, the bandwidth is unlimited in theory. In practice, there are stories of terminating accounts of those who used too much. I don't think this project will ever make $2k per month. TBH, I'd be happy it it ever reaches $200 per month. I have to be very careful about what hosting providers to use and CDNs are out of the question.
If I were running fastly, I'd donate bandwidth so that you would implement turn-key, bring-your-api-key fastly support to your open source setup! As for other cdns - I remember https://neocities.org/ built their own cdn - but I couldn't immediately find a blog post about the cdn, or something related in their GitHub (unless every node runs part of it, which would make sense, I guess).
They could use you as a PoC/demo for their serverless stuff. Same with cloudflare.
It's worth it to ask. I can send you the contact for our fastly rep if you'd like.