Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An Alternative Approach to Rate Limiting

medium.com

Post reply on HN
Print view View original

21–30 of 73 posts

Re: An Alternative Approach to Rate Limiting

#21

Post by philsnow »

daliwali wrote: post #20
philsnow wrote: post #15

Earlier quoted context omitted.

depending on the resources being thrown at you, just trying to limit concurrent connections per address could help (there's the question about how to keep information about how many connections a given address has opened distributed to your load balancing layer and consistent, or at least consistent enough to mostly do the right thing most of the time) maybe that doesn't help with ipv6, though. you'd run out of memor…

More sophisticated ways to reject connections rely on heuristics and may result in denying legitimate requests. I don't have all the answers, I'm just suggesting a way that doesn't consider all requests equal. Every CORS pre-flight request uses the OPTIONS method. It is also used to advertise which methods are available on a route.

> Every CORS pre-flight request uses the OPTIONS method.

https://stackoverflow.com/questions/8685678/cors-how-do-pref...

Neat, thanks :) didn't know that

Re: An Alternative Approach to Rate Limiting

#22

Post by dvt »

joaodlf wrote: post #16

It's a nice post with a lot of detail and nice imagery... With that said, how would a simple, slightly modified, exponential backoff work any worse?

Literally was about to post exactly this (I was thinking Fibonacci). The article's solutions seems like way too much work for something that shouldn't be half as complicated.

Re: An Alternative Approach to Rate Limiting

#23

Post by cakoose »

joaodlf wrote: post #16

It's a nice post with a lot of detail and nice imagery... With that said, how would a simple, slightly modified, exponential backoff work any worse?

What? These are different things.

This article is about the server deciding which requests to reject. Exponential backoff is a strategy clients use deciding when to retry after their request is rejected. (Plus, the article is about malicious clients; they're not going to follow your preferred backoff strategy.)

More concretely, how would exponential backoff ensure that you don't allow more than 10 requests/second per user?

Re: An Alternative Approach to Rate Limiting

#24

Post by antirez »

Animats wrote: post #17
daliwali wrote: post #5

I think rate limiting is the wrong idea. Say for example, a client wants to re-fetch everything that it has cached, it may send a burst of requests in a short amount of time, and some of those requests may be wrongly rejected due to rate limits. This is what happens when a browser refreshes a page for example. A better approach I think is delaying request handling based on resource allocation. If one client is dispro…

This is why I've suggested, and used, fair queuing for rate limiting. Any source can make lots of requests, but they queue up by IP address. (In practice, you have queues organized by hashed IP address, which you service round-robin. That's how routers do fair queuing.) If one queue gets very long, then you start sending 429 errors for new adds to that queue. Otherwise, the requests from the heavy load source just ta…

Neat idea, but if you hash the same as the spammer?

Re: An Alternative Approach to Rate Limiting

#25

Post by kbenson »

jdmichal wrote: post #12
daliwali wrote: post #5

I think rate limiting is the wrong idea. Say for example, a client wants to re-fetch everything that it has cached, it may send a burst of requests in a short amount of time, and some of those requests may be wrongly rejected due to rate limits. This is what happens when a browser refreshes a page for example. A better approach I think is delaying request handling based on resource allocation. If one client is dispro…

Once you start getting into quality-of-service, basic rate limiting like the discussed is not enough. I think Stripe did a better job of covering such concerns in their rate limiter post. They specifically talk about being lenient to "bursty" traffic, as that was a legitimate use-case for clients. https://stripe.com/blog/rate-limiters https://news.ycombinator.com/item?id=13997029

I particularly liked a comment about a fairly elegant way to rate limit with a caching proxy[1] in that HN post. When your main application returns a rate limit reached request, cache that in your proxy for a certain amount of time, and once it's timed out of the cache, they'll hit your main app again.

1: https://news.ycombinator.com/item?id=13998081

Re: An Alternative Approach to Rate Limiting

#26

Post by irgeek »

In response, we implemented a shadow ban: On the surface, the attackers continued to receive a 200 HTTP response code, but behind the scenes we simply stopped sending document invitations after they exceeded the rate limit.

And right there they broke the service for legitimate users. Totally unacceptable collateral damage IMHO.

Re: An Alternative Approach to Rate Limiting

#27

Post by [deleted] »

antirez wrote: post #24
Animats wrote: post #17

Earlier quoted context omitted.

This is why I've suggested, and used, fair queuing for rate limiting. Any source can make lots of requests, but they queue up by IP address. (In practice, you have queues organized by hashed IP address, which you service round-robin. That's how routers do fair queuing.) If one queue gets very long, then you start sending 429 errors for new adds to that queue. Otherwise, the requests from the heavy load source just ta…

Neat idea, but if you hash the same as the spammer?

[deleted]

Re: An Alternative Approach to Rate Limiting

#28

Post by [deleted] »

antirez wrote: post #24
Animats wrote: post #17

Earlier quoted context omitted.

This is why I've suggested, and used, fair queuing for rate limiting. Any source can make lots of requests, but they queue up by IP address. (In practice, you have queues organized by hashed IP address, which you service round-robin. That's how routers do fair queuing.) If one queue gets very long, then you start sending 429 errors for new adds to that queue. Otherwise, the requests from the heavy load source just ta…

Neat idea, but if you hash the same as the spammer?

[deleted]

Re: An Alternative Approach to Rate Limiting

#29

Post by matt_wulfeck »

irgeek wrote: post #26

In response, we implemented a shadow ban: On the surface, the attackers continued to receive a 200 HTTP response code, but behind the scenes we simply stopped sending document invitations after they exceeded the rate limit. And right there they broke the service for legitimate users. Totally unacceptable collateral damage IMHO.

Agreed. This would be so absolutely frustrating to me and very easy to circumvent for nefarious people.

Re: An Alternative Approach to Rate Limiting

#30

Post by markwaldron »

timothycrosley wrote: post #4

I would think if you have a consumer application that can't handle double what is set as the rate limit during a very small corner case (start and end of the the minute barrier) you have bigger problems. As you're still effectively enforcing your rate limit over time with that approach. This just sounds like micro-optimization at its worst.

Had we not discovered the attack, we could have faced a huge surge in our delivery costs and a decline in our email sender reputation.

This isn't just about the application being able to handle double the amount of load, it about keeping costs down and preventing someone from drastically increasing your bill. I work at a fintech company, and many of the 3rd party providers we use have a non-negligible cost per API call. You wouldn't want to wake up one morning and find that someone triggered one of those calls thousands of times while you were asleep.

Post reply on HN