Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenFreeMap survived 100k requests per second

blog.hyperknot.com

Post reply on HN
Print view View original

31–40 of 140 posts

Re: OpenFreeMap survived 100k requests per second

#31

Post by hyperknot »

rtaylorgarlock wrote: post #25

Is it always/only 'laziness' (derogatory, i know) when caching isn't implemented by a site like wplace.live ? Why wouldn't they save openfreemap all the traffic when a caching server on their side presumably could serve tiles almost as fast or faster than openfreemap?

We are talking about an insane amount of data here. It was 56 Gbit/s (or 56 x 1 Gbit servers 100% saturated!). This is not something a "caching server" could handle. We are talking on the order of CDN networks, like Cloudflare, to be able to handle this.

Re: OpenFreeMap survived 100k requests per second

#32

Post by Karliss »

LoganDark wrote: post #7

> I believe what is happening is that those images are being drawn by some script-kiddies. Oh absolutely not. I've seen so many autistic people literally just nolifing and also collaborating on huge arts on wplace. It is absolutely not just script kiddies. > 3 billion requests / 2 million users is an average of 1,500 req/user. A normal user might make 10-20 requests when loading a map, so these are extremely high, sc…

Just scrolled around a little bit 2-3minutes with network monitor open. That already resulted in 500requests, 5MB transferred (after filtering by vector tile data). Not sure how many of those got cached by browser with no actual requests, cached by browser exchanging only headers or cached by cloudflare. I am guessing that the typical 10-20 requests/user case is for embedded map fragment like those commonly found in contact page where most users don't scroll at all or at most slightly zoom out to better see rest of city.

Re: OpenFreeMap survived 100k requests per second

#34

Post by wyager »

hyperknot wrote: post #31
rtaylorgarlock wrote: post #25

Is it always/only 'laziness' (derogatory, i know) when caching isn't implemented by a site like wplace.live ? Why wouldn't they save openfreemap all the traffic when a caching server on their side presumably could serve tiles almost as fast or faster than openfreemap?

We are talking about an insane amount of data here. It was 56 Gbit/s (or 56 x 1 Gbit servers 100% saturated!). This is not something a "caching server" could handle. We are talking on the order of CDN networks, like Cloudflare, to be able to handle this.

> or 56 x 1 Gbit servers 100% saturated

Presumably a caching server would be 10GbE, 40GbE, or 100GbE

56Gbit/sec of pre-generated data is definitely something that you can handle from 1 or 2 decent servers, assuming each request doesn't generate a huge number of random disk reads or something

Re: OpenFreeMap survived 100k requests per second

#35

Post by thunderfork »

markerz wrote: post #30
rtaylorgarlock wrote: post #25

Is it always/only 'laziness' (derogatory, i know) when caching isn't implemented by a site like wplace.live ? Why wouldn't they save openfreemap all the traffic when a caching server on their side presumably could serve tiles almost as fast or faster than openfreemap?

It looks like a fun website, not a for-profit website. The expectations and focus of fun websites is more to just get it working than to handle the scale. It sounds like their user base exploded overnight, doubling every 14 hours or so. It also sounds like it’s other a solo dev or a small group based on the maintainers wording.

[dead]

Re: OpenFreeMap survived 100k requests per second

#37

Post by ndriscoll »

willsmith72 wrote: post #33

so 96% availability = "survived" now? but interesting write-up. If I were a consumer of OpenFreeMap, I would be concerned that such an availability drop was only detected by user reports

If I were a consumer of a free service from someone who will not take your money to offer support or an SLA (i.e. is not trying to run a business), I would assume there's little to no monitoring at all.

Re: OpenFreeMap survived 100k requests per second

#38

Post by perching_aix »

charcircuit wrote: post #20
columb wrote: post #15

Earlier quoted context omitted.

You are so entitled... Because of you most nice things have "no limits but...". Not cool stress testing someone's infrastructure. Not cool. The author of this post is more than understanding, tried to fix it and offered a solution even after blocking them. On a free service. Show us what you have done.

>You are so entitled That's how agreements work. If someone says they will sell a hamburger for $5, and another person pays $5 for a hamburger, then they are entitled to a hamburger. >On a free service. It's up to the owner to price the service. Being overwhelmed by traffic when there are no limits is not a problem limited only to free services.

> Do you offer support and SLA guarantees?

>

> At the moment, I don’t offer SLA guarantees or personalized support.

From the website.

Re: OpenFreeMap survived 100k requests per second

#39

Post by timmg »

willsmith72 wrote: post #33

so 96% availability = "survived" now? but interesting write-up. If I were a consumer of OpenFreeMap, I would be concerned that such an availability drop was only detected by user reports

96% during a unique event. I think you would typically consider long term in a stat like that.

Assuming it was close to 100% the rest of the year, that works out to 99.97% over 12 months.

Re: OpenFreeMap survived 100k requests per second

#40

Post by ndriscoll »

hyperknot wrote: post #31
rtaylorgarlock wrote: post #25

Is it always/only 'laziness' (derogatory, i know) when caching isn't implemented by a site like wplace.live ? Why wouldn't they save openfreemap all the traffic when a caching server on their side presumably could serve tiles almost as fast or faster than openfreemap?

We are talking about an insane amount of data here. It was 56 Gbit/s (or 56 x 1 Gbit servers 100% saturated!). This is not something a "caching server" could handle. We are talking on the order of CDN networks, like Cloudflare, to be able to handle this.

I'd be somewhat surprised if nginx couldn't saturate a 10Gbit link with an n150 serving static files, so I'd expect 6x $200 minipcs to handle it. I'd think the expensive part would be the hosting/connection.
Post reply on HN