Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenFreeMap survived 100k requests per second

blog.hyperknot.com

Post reply on HN
Print view View original

111–120 of 140 posts

Re: OpenFreeMap survived 100k requests per second

#111

Post by [deleted] »

charcircuit wrote: post #22
010101010101 wrote: post #18

Earlier quoted context omitted.

Do you expect him just to let the service remain broken or to scale up to infinite cost to himself on this volunteer project? He worked with the project author to find a solution that works for both and does not degrade service for every other user, under literally no obligation to do anything at all. This isn’t Anthropic deciding to throttle users paying hundreds of dollars a month for a subscription. Constructive c…

We are talking about hosting a fixed amount of static files. This should be a solved problem. This is nothing like running large AI models for people.

[deleted]

Re: OpenFreeMap survived 100k requests per second

#112

Post by eitland »

slacktivism123 wrote: post #92
motorest wrote: post #87

Earlier quoted context omitted.

> "€20/month from Hetzner" is great until you actually need it to be up and working when you need it. I managed a few Hetzner cloud instances, and some report perfect uptime for over a year. The ones that don't, I was the root cause. What exactly leads you to make this sort of claim? Do you actually have any data or are you just running your mouth off?

>What exactly leads you to make this sort of claim? https://news.ycombinator.com/item?id=29651993 https://news.ycombinator.com/item?id=42365295 https://news.ycombinator.com/item?id=44038591 >are you just running your mouth off? Don't be snarky. Edit out swipes. https://news.ycombinator.com/newsguidelines.html

I’ve just checked all three, and it appears to be rather more nuanced than you make it out to be.

Re: OpenFreeMap survived 100k requests per second

#113

Post by Aurornis »

Starlevel004 wrote: post #57

> I believe what is happening is that those images are being drawn by some script-kiddies. If I understand correctly, the website limited everyone to 1 pixel per 30 seconds, so I guess everyone was just scripting Puppeteer/Chromium to start a new browser, click a pixel, and close the browser, possibly with IP address rotation, but maybe that wasn't even needed. I think you perhaps underestimate just how big of a thin…

> I think you perhaps underestimate just how big of a thing this became basically overnight.

They don’t need to estimate because in the article they talked to the site and got their traffic numbers: An estimated 2 million users.

That’s 1500 requests per user, which implies a lot of scripting is going on.

Re: OpenFreeMap survived 100k requests per second

#114

Post by cube00 »

It's really surprising that no CDNs or cloud storage providers offer even the single PMTiles file in some sort of shared library customers can use.

I guess they'd all rather their customers each upload the 120GB file and then charge them all individually.

If they're crafty they'll have their storage configured so there's only one actual copy on the underlying infra so every other shadow copy is pure profit.

Re: OpenFreeMap survived 100k requests per second

#115

Post by zahlman »

Starlevel004 wrote: post #57

> I believe what is happening is that those images are being drawn by some script-kiddies. If I understand correctly, the website limited everyone to 1 pixel per 30 seconds, so I guess everyone was just scripting Puppeteer/Chromium to start a new browser, click a pixel, and close the browser, possibly with IP address rotation, but maybe that wasn't even needed. I think you perhaps underestimate just how big of a thin…

> I think you perhaps underestimate just how big of a thing this became basically overnight. I mentioned a drawing over my house to a few people and literally everyone instantly knew what I meant without even saying the website.

On the other hand, this is the first I've heard of this thing.

Re: OpenFreeMap survived 100k requests per second

#116

Post by Aurornis »

ivanjermakov wrote: post #94

> Nice idea, interesting project, next time please contact me before. I understand that my popular service might bring your less popular one to the halt, but please configure it on your end so I know _programmatically_ what its capabilities are. I host no API without rate-limiting. Additionally, clearly listing usage limits might be a good idea.

> I understand that my popular service might bring your less popular one to the halt, but please configure it on your end so I know _programmatically_ what its capabilities are.

Quite entitled expectations for someone using a free and open service to underpin their project.

The requests were coming from distributed clients, not a central API gateway that could respond to rate limiting requests

> I host no API without rate-limiting. Additionally, clearly listing usage limits might be a good idea.

Again, this wasn’t a central, well-behaved client hitting the API from a couple of IPs or with a known API key.

They calculate that per every 1 user of the wlive.place website, they were getting 1500 requests. This implies a lot of botting and scripting.

This is basically load testing the web site at DDoS scale.

Re: OpenFreeMap survived 100k requests per second

#118

Post by yifanl »

Starlevel004 wrote: post #57

> I believe what is happening is that those images are being drawn by some script-kiddies. If I understand correctly, the website limited everyone to 1 pixel per 30 seconds, so I guess everyone was just scripting Puppeteer/Chromium to start a new browser, click a pixel, and close the browser, possibly with IP address rotation, but maybe that wasn't even needed. I think you perhaps underestimate just how big of a thin…

They have the user count from the dev, 2 million daily users shouldn't be generating billions of requests unless a good portion of them are botting.

Re: OpenFreeMap survived 100k requests per second

#119

Post by ivanjermakov »

Aurornis wrote: post #116
ivanjermakov wrote: post #94

> Nice idea, interesting project, next time please contact me before. I understand that my popular service might bring your less popular one to the halt, but please configure it on your end so I know _programmatically_ what its capabilities are. I host no API without rate-limiting. Additionally, clearly listing usage limits might be a good idea.

> I understand that my popular service might bring your less popular one to the halt, but please configure it on your end so I know _programmatically_ what its capabilities are. Quite entitled expectations for someone using a free and open service to underpin their project. The requests were coming from distributed clients, not a central API gateway that could respond to rate limiting requests > I host no API without…

Ugh, then I agree. This way it's indistinguishable from DDoS attack.

Re: OpenFreeMap survived 100k requests per second

#120

Post by radu_floricica »

toast0 wrote: post #43
rtaylorgarlock wrote: post #25

Is it always/only 'laziness' (derogatory, i know) when caching isn't implemented by a site like wplace.live ? Why wouldn't they save openfreemap all the traffic when a caching server on their side presumably could serve tiles almost as fast or faster than openfreemap?

Why should they when openfreemap is behind a CDN and their home page says things like: > Using our public instance is completely free: there are no limits on the number of map views or requests. There’s no registration, no user database, no API keys, and no cookies. We aim to cover the running costs of our public instance through donations. > Is commercial usage allowed? > Yes. IMHO, reading this and then just using…

Politeness? Also speed.

But it depends on the project architecture. If the tiles are needed only client-side, then there's really little reason to cache things on _my_ server. That would imply I'm better at caching openstreetmap tiles than... openstreetmap. Plus you're just making the system needlessly more complicated.

And there's little reason for openstreet map to be upset, since it's not like _I_ am making 2 million requests - there are 2 million separate users of osm+. It's aligned to their incentive of having more people use osm and derived works. All is well.

Post reply on HN