Live data from Hacker News

How to Design a Scalable Rate Limiting Algorithm

konghq.com

11–20 of 36 posts

Re: How to Design a Scalable Rate Limiting Algorithm

#11

Earlier quoted context omitted.

Have you found any viable alternatives to Kong?

So far we're still using Kong. Planning an upgrade soon. But we try to avoid shifting to much domain logic in the form of Kong plugins.

If I may ask, what kind of functionality have you been trying to implement in a Kong plugin?

Re: How to Design a Scalable Rate Limiting Algorithm

#12

For a different view on the topic watch "Stop Rate Limiting! Capacity Management Done Right" https://www.youtube.com/watch?v=m64SWl9bfvk The basic premise is not do do req/s limiting but rather concurrency limiting which results in req/s limiting by itself. Concurrency limiting is rather simple and doesn't require a lot of code complexity.

Wow! That was an incredible talk/demonstration. Thanks for that!

Re: How to Design a Scalable Rate Limiting Algorithm

#13
One thing that RateLim.it uses to its advantage is calculating the “nextPossiblePass” for each limit. This allows clients to cache forever that something is over the limit until time X and not have to make another request.

In the bursty case this let’s clients effectively short circuit and protect the system.

Disclaimer: I work on https://www.ratelim.it/documentation/basic_rate_limits

Re: How to Design a Scalable Rate Limiting Algorithm

#14

For a different view on the topic watch "Stop Rate Limiting! Capacity Management Done Right" https://www.youtube.com/watch?v=m64SWl9bfvk The basic premise is not do do req/s limiting but rather concurrency limiting which results in req/s limiting by itself. Concurrency limiting is rather simple and doesn't require a lot of code complexity.

The problem here is, it depends on WHY you are rate limiting. If it's just to balance so you don't overload your backends, then absolutely this is a great way to do it.

If however you are trying to limit client(s) because the service is an authentication gateway for instance, then you want to limit user/pass requests to X number then concurrency limiting isn't a good way to do that.

So you may need both, depending on your use-cases, so it's not a one-size fits all solution.

Re: How to Design a Scalable Rate Limiting Algorithm

#16
I have been using Kong since 2016 on various APIs and I have been impressed by the continuous increase of performance after every release. I wish upgrades were easier but I know they are working on it (the "migrations" between each major version), but overall it's a fast and pluggable layer for any API inside or outside the firewall, especially if you are running containers.

On this note for those of you who haven't noticed it yet, they have released an Alpine version of their Docker image, but it's still not the default one. I would actually recommend using it to further reduce the size of Kong containers: https://konghq.com/blog/kong-alpine-docker/

Re: How to Design a Scalable Rate Limiting Algorithm

#17
post #11

Earlier quoted context omitted.

So far we're still using Kong. Planning an upgrade soon. But we try to avoid shifting to much domain logic in the form of Kong plugins.

If I may ask, what kind of functionality have you been trying to implement in a Kong plugin?

As you know, there are many plugins to do authentication in Kong. We started with jwt, then a coworker decided that we needed a more flexible approach so he basically forked the jwt plugin to add stuff for our needs. It quickly became confusing and hard to maintain.

When we tried to introduce a new feature (tokens similar to what Github offers with personal token, that is a revocable token with a given set of permissions), we had a rough time.

In the end, the decision to fork the plugin was maybe not the good one and the decision to bring the token feature into this plugin were maybe not the right one. But still, working in the plugin code was unpleasant in my opinion.

By the way, keep up the good work, it's a solid piece of software!

Re: How to Design a Scalable Rate Limiting Algorithm

#18

Disregarding the article completely, I'll share my opinion on Kong because we use quite a bit at my workplace. We use an old version (0.9.x as of right now). The things I shared below might not be true anymore in new versions but I can't tell and are regarding the plugin system. IMO, the idea of taking things like authentication, rate limiting, etc in a proxy is a wonderful idea ( https://2tjosk2rxzc21medji3nfn1g-wpe…

Have you found any viable alternatives to Kong?

Not OP, but we are using Tyk (tyk.io) at my workplace. It's definitely more user friendly than Kong, but we've had other issues with it. Having not used Kong extensively but knowing what it is, I'll go out on a limb and say Kong is likely more performant.

Re: How to Design a Scalable Rate Limiting Algorithm

#19
post #14

For a different view on the topic watch "Stop Rate Limiting! Capacity Management Done Right" https://www.youtube.com/watch?v=m64SWl9bfvk The basic premise is not do do req/s limiting but rather concurrency limiting which results in req/s limiting by itself. Concurrency limiting is rather simple and doesn't require a lot of code complexity.

The problem here is, it depends on WHY you are rate limiting. If it's just to balance so you don't overload your backends, then absolutely this is a great way to do it. If however you are trying to limit client(s) because the service is an authentication gateway for instance, then you want to limit user/pass requests to X number then concurrency limiting isn't a good way to do that. So you may need both , depending o…

What is the use case for your second example? I am not following.

Wouldn't that just be rate limiting by client IP though?

Re: How to Design a Scalable Rate Limiting Algorithm

#20

For a different view on the topic watch "Stop Rate Limiting! Capacity Management Done Right" https://www.youtube.com/watch?v=m64SWl9bfvk The basic premise is not do do req/s limiting but rather concurrency limiting which results in req/s limiting by itself. Concurrency limiting is rather simple and doesn't require a lot of code complexity.

He mentioned that nginx using lua managed the requests but I didn't see the code for that. Is that available anywhere?
Post reply on HN