An Alternative Approach to Rate Limiting
medium.com
An Alternative Approach to Rate Limiting
1–10 of 73 posts
Re: An Alternative Approach to Rate Limiting
#2Sounds like a lot of work to avoid writing 20 lines of Lua.
Re: An Alternative Approach to Rate Limiting
#3Sounds like a lot of work to avoid writing 20 lines of Lua.
agreed. I was a bit leery of diving into Lua as I was building http://ratelim.it but it really expands Redis's capabilities dramatically and was easy enough to add.
My apps all write the lua into redis and store the hash when they boot up. Duplicative, but means everybody is on the same page and it's easy to store the lua in the main codebase.
Re: An Alternative Approach to Rate Limiting
#4I would think if you have a consumer application that can't handle double what is set as the rate limit during a very small corner case (start and end of the the minute barrier) you have bigger problems. As you're still effectively enforcing your rate limit over time with that approach. This just sounds like micro-optimization at its worst.
Re: An Alternative Approach to Rate Limiting
#5I think rate limiting is the wrong idea. Say for example, a client wants to re-fetch everything that it has cached, it may send a burst of requests in a short amount of time, and some of those requests may be wrongly rejected due to rate limits. This is what happens when a browser refreshes a page for example.
A better approach I think is delaying request handling based on resource allocation. If one client is disproportionately using up more time than others, then processing that clients' request will be queued up to process later, while well behaving clients will get their requests handled quickly. I think this is a more realistic approach and imposes less arbitrary restrictions.
Re: An Alternative Approach to Rate Limiting
#6I think rate limiting is the wrong idea. Say for example, a client wants to re-fetch everything that it has cached, it may send a burst of requests in a short amount of time, and some of those requests may be wrongly rejected due to rate limits. This is what happens when a browser refreshes a page for example. A better approach I think is delaying request handling based on resource allocation. If one client is dispro…
Those concepts are not mutually exclusive and are almost universally used in tandem.
Queues and backpressure to properly serve bursty traffic and rate limiting to protect against bad actors.
Re: An Alternative Approach to Rate Limiting
#7I think rate limiting is the wrong idea. Say for example, a client wants to re-fetch everything that it has cached, it may send a burst of requests in a short amount of time, and some of those requests may be wrongly rejected due to rate limits. This is what happens when a browser refreshes a page for example. A better approach I think is delaying request handling based on resource allocation. If one client is dispro…
Where do you queue those requests ? If you do it anywhere under your own control, you will accumulate memory and open connections.
If you issue a 429, the client knows it needs to wait and retry, and you've pushed the backpressure all the way past the demarcation line of your own infrastructure.
Re: An Alternative Approach to Rate Limiting
#8I think rate limiting is the wrong idea. Say for example, a client wants to re-fetch everything that it has cached, it may send a burst of requests in a short amount of time, and some of those requests may be wrongly rejected due to rate limits. This is what happens when a browser refreshes a page for example. A better approach I think is delaying request handling based on resource allocation. If one client is dispro…
Where do you queue those requests ? If you do it anywhere under your own control, you will accumulate memory and open connections. If you issue a 429, the client knows it needs to wait and retry, and you've pushed the backpressure all the way past the demarcation line of your own infrastructure.
One could limit concurrent connections per address. The idea is that an OPTIONS request which doesn't take much resources at all could be treated with a different weight than a POST which costs the server time to process.
Re: An Alternative Approach to Rate Limiting
#9This is why I like HackerNews comments. As a primarily front-end dev building a SaaS, I'd already bookmarked this post and was planning implementing it. But it seems like the comments here are pointing me in a better direction.
Re: An Alternative Approach to Rate Limiting
#10I would think if you have a consumer application that can't handle double what is set as the rate limit during a very small corner case (start and end of the the minute barrier) you have bigger problems. As you're still effectively enforcing your rate limit over time with that approach. This just sounds like micro-optimization at its worst.
Yeah, I was also thinking how meaningful ~20MB of memory use really would be in this context. Or how badly would racy token bucket perform in the real world. Still, enjoyed the read.