Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Let us serve you, but don't bring us down

blog.archive.org

Post reply on HN
Print view View original

251–255 of 255 posts

Re: Let us serve you, but don't bring us down

#251

Post by coldpie »

xenotize wrote: post #250
coldpie wrote: post #237

Earlier quoted context omitted.

So the answer is it's just speculation and has never been seen in the wild :)

Kind of a snarky response. Obviously this has been seen in the wild. If you created an intrusion detection system to look for suspicious requests, I think one occurring over and over and at a regular interval would clearly be seen as malicious and not a genuine user.

> Obviously this has been seen in the wild.

Can you provide an actual example? I see it come up a lot in these conversations, but I'm really skeptical that anyone actually does this analysis. It seems like rate analysis (either requests or bandwidth) would achieve the same result in a far simpler manner, so I suspect that is what actually happens.

Re: Let us serve you, but don't bring us down

#252

Post by account42 »

emporas wrote: post #111
sph wrote: post #107

Earlier quoted context omitted.

Proof-of-work was originally introduced by Adam Back's hashcash, to fight email spam. Bitcoin is an evolution of that idea applied to digital cash. https://en.wikipedia.org//wiki/Hashcash

Yes, DDoS attacks are spam over the http protocol. Spam is spam over the imap protocol. Overwhelming a server with too many download requests, is not spam but it has the same effect. Calling the police every ten minutes because the door sounds like someone tries to break in, is spam.

> Spam is spam over the imap protocol.

IMAP isn't used for sending email. You're thinking of SMTP.

Re: Let us serve you, but don't bring us down

#253

Post by account42 »

abbe98 wrote: post #167
yyyk wrote: post #65

Archive.org is a bit of a special case, you need to call them repeatedly to archive a website. They do have a rate limit there, it's pretty aggressive* to the point you could trip it by manually using the site. They must have forgotten to limit the OCR files download. * If they had a better API (a simple non-synchronous API would be enough, one where we could send a list of URLs would be even better), one could have…

I believe you can upload WRAC files to IA and ask them to index the content. Saves them the need to do the archiving and you won't be rate limited on their end.

So they just trust you that your archives are not manipulated?

Re: Let us serve you, but don't bring us down

#255

Post by daoman »

nektro wrote: post #241
thayne wrote: post #208

Earlier quoted context omitted.

It isn't that hard to set up naive rate limiting per ip address. It's a few lines in haproxy, and there is documentation on how to do it. There are a couple of problems that make it more complicated though. The first is that with NATs, you can have a lot of users behind a single IP address, which can result in legitimate requests getting blocked. The second is that, while it can help against a DoS, it doesn't help th…

ipv6 would solve these issues because they remove the need for NATs

And create a new problem: banning by IPv6 is a nightmare https://adam-p.ca/blog/2022/02/ipv6-rate-limiting/
Post reply on HN