Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenFreeMap survived 100k requests per second

blog.hyperknot.com

Post reply on HN
Print view View original

61–70 of 140 posts

Re: OpenFreeMap survived 100k requests per second

#61

Post by charcircuit »

eszed wrote: post #45
charcircuit wrote: post #20

Earlier quoted context omitted.

>You are so entitled That's how agreements work. If someone says they will sell a hamburger for $5, and another person pays $5 for a hamburger, then they are entitled to a hamburger. >On a free service. It's up to the owner to price the service. Being overwhelmed by traffic when there are no limits is not a problem limited only to free services.

Sure, and if you bulk-order 5k hamburgers the restaurant will honor the price, but they'll also tell you "we're going to need some notice to handle that much product". Perfect analogy, really. This guy handled the situation perfectly, imo.

Except in this case the restauraut would have been able to handle the 5k orders if they didn't arbitrarily have their workers work with their hands tied behind their back. And instead of untieing their workers and appreciating the realization they were accidently bottlenecking themselves they blame the nearby event who caused a spike in foot traffic.

Publicly attacking your users instead of celebrating their success and your new learnings is not what I would call handling it perfectly. I think going for a halo effect strategy where you celebrate how people are using your platform to accomplish their goals will help people understand how what is being done is valuable and want people to adopt it or financially support it. On the other hand attacking people who use your platform publicly can make people apprehensive in using it fearing that they will be criticized too.

Re: OpenFreeMap survived 100k requests per second

#62

Post by perching_aix »

hoppp wrote: post #60
internetter wrote: post #50

Earlier quoted context omitted.

The tiles need to be rendered. Yes frequent tiles can be cached but you already have a cache… it’s Cloudflare. Theoretically you could port the tileserver to Cloudflare pages but then you’d need to… port it… and it probably wouldn’t be cheaper

It would cost a lot. Hetzner is hardware and them you can hammer it, free bandwidth. You get a very good server for cheap. cloudflare would be pay per request, a hefty sum if ddos happens

Did a quick cost calc (with the help of gpt5, so might be wrong) when I read their comment about Pages not being suitable for this many files.

They say they're receiving $500/mo in donos and that it's currently just enough to cover their infra costs. Given 300 million 70 KB files, R2 + high cache hit ratio would work out to about $300 in storage-months + request costs, or $600/mo with Cache Reserve and then they'd always hit cache if I understand the project right: meaning the costs shouldn't blow up beyond that, and that request count would essentially just not matter.

Re: OpenFreeMap survived 100k requests per second

#63

Post by hoppp »

perching_aix wrote: post #62
hoppp wrote: post #60

Earlier quoted context omitted.

It would cost a lot. Hetzner is hardware and them you can hammer it, free bandwidth. You get a very good server for cheap. cloudflare would be pay per request, a hefty sum if ddos happens

Did a quick cost calc (with the help of gpt5, so might be wrong) when I read their comment about Pages not being suitable for this many files. They say they're receiving $500/mo in donos and that it's currently just enough to cover their infra costs. Given 300 million 70 KB files, R2 + high cache hit ratio would work out to about $300 in storage-months + request costs, or $600/mo with Cache Reserve and then they'd al…

Yea but the cost is not a fixed monthly sum and things can go wrong as we can see from the blog post. An accident could bankrupt the dev.

A dedicated server will always cost the same so you always know how much you pay.

It will cost 40 Euro/month to have 6 cores/12 threads,64gb of ram and 1Tb of ssd.

Dirt cheap compared to any other alternative

Re: OpenFreeMap survived 100k requests per second

#64

Post by Sesse__ »

hyperknot wrote: post #31
rtaylorgarlock wrote: post #25

Is it always/only 'laziness' (derogatory, i know) when caching isn't implemented by a site like wplace.live ? Why wouldn't they save openfreemap all the traffic when a caching server on their side presumably could serve tiles almost as fast or faster than openfreemap?

We are talking about an insane amount of data here. It was 56 Gbit/s (or 56 x 1 Gbit servers 100% saturated!). This is not something a "caching server" could handle. We are talking on the order of CDN networks, like Cloudflare, to be able to handle this.

> We are talking about an insane amount of data here. It was 56 Gbit/s. This is not something a "caching server" could handle.

You are not talking about an insane amount of data if it's 56 Gbit/s. Of course a caching server could handle that.

Source: Has written servers that saturated 40gig (with TLS) on an old quadcore.

Re: OpenFreeMap survived 100k requests per second

#65

Post by hyperknot »

Sesse__ wrote: post #64
hyperknot wrote: post #31

Earlier quoted context omitted.

We are talking about an insane amount of data here. It was 56 Gbit/s (or 56 x 1 Gbit servers 100% saturated!). This is not something a "caching server" could handle. We are talking on the order of CDN networks, like Cloudflare, to be able to handle this.

> We are talking about an insane amount of data here. It was 56 Gbit/s. This is not something a "caching server" could handle. You are not talking about an insane amount of data if it's 56 Gbit/s. Of course a caching server could handle that. Source: Has written servers that saturated 40gig (with TLS) on an old quadcore.

OK, technically there might exist such server, I guess Netflix and friends are using those. But we are talking about a community supported, free service here. Hetzner servers are my only options, because of their unmetered bandwidth.

Re: OpenFreeMap survived 100k requests per second

#66

Post by cuu508 »

perching_aix wrote: post #62
hoppp wrote: post #60

Earlier quoted context omitted.

It would cost a lot. Hetzner is hardware and them you can hammer it, free bandwidth. You get a very good server for cheap. cloudflare would be pay per request, a hefty sum if ddos happens

Did a quick cost calc (with the help of gpt5, so might be wrong) when I read their comment about Pages not being suitable for this many files. They say they're receiving $500/mo in donos and that it's currently just enough to cover their infra costs. Given 300 million 70 KB files, R2 + high cache hit ratio would work out to about $300 in storage-months + request costs, or $600/mo with Cache Reserve and then they'd al…

Factor in that you also need resources to generate and upload the tiles weekly.

Re: OpenFreeMap survived 100k requests per second

#67

Post by ohdeargodno »

internetter wrote: post #50
perching_aix wrote: post #49

Haven't worked with Cloudflare yet first hand, and I'm not familiar with web map tech. But if the site really is pretty much just serving lots of static files, why is Hetzner in the loop? Wouldn't fully migrating to Cloudflare Pages be possible?

The tiles need to be rendered. Yes frequent tiles can be cached but you already have a cache… it’s Cloudflare. Theoretically you could port the tileserver to Cloudflare pages but then you’d need to… port it… and it probably wouldn’t be cheaper

Noone renders tiles on servers anymore. The vast majority of services have moved on to sending out vector tiles.

Re: OpenFreeMap survived 100k requests per second

#68

Post by TylerE »

andai wrote: post #53

From the screenshot I wanted to say, couldn't this be done on a single VPS? Seemed over engineered to me. Then I realized the silly pixels are on top of a map of the entire earth. Dang! I'm curious what the peak req/s is like. I think it might be just barely within the range supported by benchmark-friendly web servers. Unless there's some kind of order of magnitude slowdowns due to the nature of the application. Edit…

"€20/month from Hetzner" is great until you actually need it to be up and working when you need it.

Re: OpenFreeMap survived 100k requests per second

#70

Post by Sesse__ »

hyperknot wrote: post #65
Sesse__ wrote: post #64

Earlier quoted context omitted.

> We are talking about an insane amount of data here. It was 56 Gbit/s. This is not something a "caching server" could handle. You are not talking about an insane amount of data if it's 56 Gbit/s. Of course a caching server could handle that. Source: Has written servers that saturated 40gig (with TLS) on an old quadcore.

OK, technically there might exist such server, I guess Netflix and friends are using those. But we are talking about a community supported, free service here. Hetzner servers are my only options, because of their unmetered bandwidth.

It really depends on the size of the active set. If it fits into RAM of whatever server you are using, then it's not a problem at all, even with completely off-the-shelf hardware and software. Slap two 40gig NICs in it, install Varnish or whatever and you're good to go. (This is, of course, assuming that you have someone willing to pay for the bandwidth out to your users!)

If you need to go to disk to serve large parts of it, it's a different beast. But then again, Netflix was doing 800gig already three years ago (in large part from disk) and they are handicapping themselves by choosing an OS where they need to do significant amounts of the scaling work themselves.

Post reply on HN