Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How to Design a Scalable Rate Limiting Algorithm

konghq.com

Post reply on HN
Print view View original

31–36 of 36 posts

Re: How to Design a Scalable Rate Limiting Algorithm

#31

Post by hirstys »

bob_roboto wrote: post #30
johnbrodie wrote: post #18

Earlier quoted context omitted.

Not OP, but we are using Tyk (tyk.io) at my workplace. It's definitely more user friendly than Kong, but we've had other issues with it. Having not used Kong extensively but knowing what it is, I'll go out on a limb and say Kong is likely more performant.

Used Tyk with a similar experience. Generally very happy with it but the documentation was, at the time, a bit challenging.

Hey Bob - you'll be pleased to hear Tyk took that feedback on-board. There is now very comprehensive documentation and user-guides. And true to the Open Source approach, anyone can contribute to them, the community have been great in working to improve them.

Re: How to Design a Scalable Rate Limiting Algorithm

#32

Post by dmaumenee »

I use Kong EE for a client, I have read Kong EE documentation carefully and made a lot of tests. The current implementation (0.29) have this behavior (that not meet our needs).

1) All incoming requests are take into account, including those which have been rejected (with 429 error). If the consumer exceed his limit during many consecutive time windows, all the requests of the consecutive time windows will be rejected (with 429 error).

2) For a windows size of 1 second, the computed weight of the previous windows is always 100%. If the limit was reach during the previous second, all requests made in the current windows will be rejected (with 429 errors).

Re: How to Design a Scalable Rate Limiting Algorithm

#33

Post by dmaumenee »

The performance degradation of doing strict rate limit relying on Kong Cluster data store (Postgres or Cassandra) is amplified by the lake of database connection pool management. Each incoming http request induce the creation of a new database connection, this process is very expensive especially with Cassandra when authentication is enabled.

Re: How to Design a Scalable Rate Limiting Algorithm

#34

Post by heavenlyblue »

dboreham wrote: post #29
bogomipz wrote: post #19

Earlier quoted context omitted.

What is the use case for your second example? I am not following. Wouldn't that just be rate limiting by client IP though?

>Wouldn't that just be rate limiting by client IP though? This won't defeat an adversary with a botnet.

But this is not something you deal with a rate-limiter. By your logic, then a sufficiently large botnet would dwarf any live traffic whatsoever, implying you’d probably want to filter them on network level instead.

Re: How to Design a Scalable Rate Limiting Algorithm

#35

Post by jbramble81 »

coopr wrote: post #26
kylequest wrote: post #21

Earlier quoted context omitted.

Actually it should be the opposite. Definitely do your own benchmark, but take a look at the latest numbers from Tyk and compare it with the latest numbers from Kong to get a general idea.

This analysis was done by BBVA comparing Tyk and Kong performance https://www.bbva.com/en/api-gateways-kong-vs-tyk/ - disclosure: I work for Kong, but Kong (the company) had no involvement with the benchmarking in that article.

Those BBVA benchmarks look very very wrong. BBVA achieved 3822 requests per second with a 16Gb server running Tyk.

I used a 16Gb Digital Ocean box, and a separate box to run benchmarks from.

Tyk CE with keyless api.

docker run --rm williamyeh/wrk -t4 -c100 -d1m --latency --timeout 2s http://10.131.45.89:8080/tykbench/get Running 1m test @ http://10.131.45.89:8080/tykbench/get 4 threads and 100 connections Thread Stats Avg Stdev Max +/- Stdev Latency 5.79ms 6.97ms 206.76ms 87.28% Req/Sec 6.35k 0.93k 10.89k 74.33% Latency Distribution 50% 3.19ms 75% 6.28ms 90% 15.93ms 99% 31.14ms 1516573 requests in 1.00m, 185.13MB read Requests/sec: 25244.99 Transfer/sec: 3.08MB

As per above results, I got 25,244 requests per second. 90% of requests were served with sub 15ms latency.

Another thing - you can freely scale with Tyk CE as much as you want at no-cost. I currently run 3 gateways in prod and it doesn't cost a penny. The BBVA article states that you need to pay Tyk to scale - simply not true:

>However, if we want to deploy a significantly larger number of microservices, or simply ensure high-tolerance to failure, we need to scale up the number of gateways. In the case of Kong, all it takes is adding a new node and connecting it to the database. With Tyk we can do the same, although it will require us to pay for the service. This could be a determining factor when choosing what API Gateway to use.

I went through process picking API gateway about 3 months ago. Tyk is not without it's warts, but has far more out-of-box features baked in (inc distributed rate limiting) than Kong. And if custom plugins are required, I can choose between lua, python, js or anything that supports gRPC. With Kong - stuck with Lua.... no thank you.

I guess my point here is that if Tyk not performant enough for you, just add another gateway. And if I need custom functionality - I don't need to learn Lua.

Post reply on HN