How to Design a Scalable Rate Limiting Algorithm
1–10 of 36 posts
Re: How to Design a Scalable Rate Limiting Algorithm
#2Excellent article! One question though:
> A better approach is to use a “set-then-get” mindset, relying on atomic operators that implement locks in a very performant fashion, allowing you to quickly increment and check counter values without letting the atomic operations get in the way.
Can you elaborate on this? Why is it more performant? And what are the trade-offs vs. get-then-set?
Re: How to Design a Scalable Rate Limiting Algorithm
#3Excellent article! One question though: > A better approach is to use a “set-then-get” mindset, relying on atomic operators that implement locks in a very performant fashion, allowing you to quickly increment and check counter values without letting the atomic operations get in the way. Can you elaborate on this? Why is it more performant? And what are the trade-offs vs. get-then-set?
Was passing by and saw this, a better explanation and in-depth analysis can be found here:
> https://blog.figma.com/an-alternative-approach-to-rate-limit...
Re: How to Design a Scalable Rate Limiting Algorithm
#4For a different view on the topic watch "Stop Rate Limiting! Capacity Management Done Right" https://www.youtube.com/watch?v=m64SWl9bfvk
The basic premise is not do do req/s limiting but rather concurrency limiting which results in req/s limiting by itself.
Concurrency limiting is rather simple and doesn't require a lot of code complexity.
Re: How to Design a Scalable Rate Limiting Algorithm
#5Disregarding the article completely, I'll share my opinion on Kong because we use quite a bit at my workplace. We use an old version (0.9.x as of right now). The things I shared below might not be true anymore in new versions but I can't tell and are regarding the plugin system.
IMO, the idea of taking things like authentication, rate limiting, etc in a proxy is a wonderful idea (https://2tjosk2rxzc21medji3nfn1g-wpengine.netdna-ssl.com/wp-...). So in theory, the approach Kong takes is wonderful, but I think the implementation is not so much.
Kong is layer over nginx and uses lua as a scripting language to add all sorts of stuff. Quickly, you reach the limit of the plugins capability so you think, well, I will write my own plugins. Well, to me it was a very unpleasing experience. I found that the thing was hard to test and hard to maintain. Maybe my criticism is more lua than Kong but since Kong relies heavily on lua, there is not much I can do.
There is also magic happening. You declare a schema.lua files to configure the store to hold your data. Then automatically you've got a DAO interface available with a bunch of method on it to work with the store. You don't know what methods are available or what arguments should be passed into these functions.
Anyway, this is my take after spending quite a few hours working on home made plugins in lua for Kong.
In the end, I'm glad Kong is open source and it's a great piece of software. It helped us reduce our applications complexity but make sure you don't start to shift to much logic into it because the plugin system can be hard to work with.
Re: How to Design a Scalable Rate Limiting Algorithm
#6This may be how Kong does it but it's not really 'high performance'. The right way to do rate limiting is to limit by IP using a counting Bloom Filter or Cuckoo filter along with random samples. When you hit a false positive then you have a second normal rate limiter to 'mop up' IPs that are over the first limiter.
This doesn't give you a hard exact limit but gets the job done storing far less state. You also need to bucket by IP sub-ranges in IPV6 to stop people crap flooding you with tons of unique IP's
Re: How to Design a Scalable Rate Limiting Algorithm
#7Disregarding the article completely, I'll share my opinion on Kong because we use quite a bit at my workplace. We use an old version (0.9.x as of right now). The things I shared below might not be true anymore in new versions but I can't tell and are regarding the plugin system. IMO, the idea of taking things like authentication, rate limiting, etc in a proxy is a wonderful idea ( https://2tjosk2rxzc21medji3nfn1g-wpe…
Have you found any viable alternatives to Kong?
Re: How to Design a Scalable Rate Limiting Algorithm
#8Disregarding the article completely, I'll share my opinion on Kong because we use quite a bit at my workplace. We use an old version (0.9.x as of right now). The things I shared below might not be true anymore in new versions but I can't tell and are regarding the plugin system. IMO, the idea of taking things like authentication, rate limiting, etc in a proxy is a wonderful idea ( https://2tjosk2rxzc21medji3nfn1g-wpe…
Marco here, Kong's CTO. We hear you, we are aware that plugin development is not as easy as we wish, and we are planning to rollout a few improvements next year to make it mainstream, including:
- A built-in Lua plugin API that abstracts away the underlying complexity of NGINX and OpenResty. Among other things the plugin API will make it easy to interact with the lifecycle of request/response objects.
- An official testing framework for plugins (unit and integration tests).
- A new DAO to get rid of some magic and make the overall system more robust and extensible (with an in-memory DAO implementation for example).
- Support for remote gRPC plugins to support plugin development in any gRPC supported languages.
- And finally supporting plugin installation/versioning within Kong itself using the HTTP Admin API to install/uninstall plugins cluster-wide on any Kong node (as opposed to installing plugins on the file system).
NGINX and Lua (on top of LuaJIT http://luajit.org/) were chosen for their reliability and performance, the next step for Kong is to focus more on making the overall Kong ecosystem bigger therefore simpler plugin development is a very high priority item in our roadmap.
Re: How to Design a Scalable Rate Limiting Algorithm
#9For a different view on the topic watch "Stop Rate Limiting! Capacity Management Done Right" https://www.youtube.com/watch?v=m64SWl9bfvk The basic premise is not do do req/s limiting but rather concurrency limiting which results in req/s limiting by itself. Concurrency limiting is rather simple and doesn't require a lot of code complexity.
This video was very interesting, thanks for sharing!
Re: How to Design a Scalable Rate Limiting Algorithm
#10Disregarding the article completely, I'll share my opinion on Kong because we use quite a bit at my workplace. We use an old version (0.9.x as of right now). The things I shared below might not be true anymore in new versions but I can't tell and are regarding the plugin system. IMO, the idea of taking things like authentication, rate limiting, etc in a proxy is a wonderful idea ( https://2tjosk2rxzc21medji3nfn1g-wpe…
Have you found any viable alternatives to Kong?
So far we're still using Kong. Planning an upgrade soon. But we try to avoid shifting to much domain logic in the form of Kong plugins.