41–48 of 48 posts
No mention of CGNAT which caused me many problems at a previous role?
Does CGNAT do rate limiting? If so then is there some documentation I can lookup.
My favourite algorithm is generic cell rate algorithm (GCRA). It works like token bucket in this post. The implementation is dead simple and requires no background tasks and needs very minimal state. Instead of storing the current number of tokens, you instead store when the bucket will be full. If you take a token from the bucket, you increment the timestamp accordingly by 1/rps. The only complication is it the fill…
Redis/valkey? Postgres? DynamoDB?
Earlier quoted context omitted.
Does CGNAT do rate limiting? If so then is there some documentation I can lookup.
I'm pretty sure GP means: all those users have egress from a finite number of IPv4 and thus if rate limiting is done by IP those behind the NAT are going to have a real bad time. It's true of all NAT setups, but the affected audience size for GCNAT could be outrageous
Something I’ve long wondered is why you never hear about rate limiting algorithms that are based on the cost to serve the request or algorithms that dynamically learn the capacity of the system and give everyone a fair share. In the field of router buffer management, there are algorithms like Stochastic Fair Blue, which does the latter, but is somewhat hard to apply to HTTP because you’d have to define a success/fail…
I mentioned this in another place on this thread, but a simple AIMD algorithm paired with a token bucket is surprisingly effective at dynamically adjusting to available capacity, even across a fleet of services not sharing state other than the contended resource. Pretty easy to pair AIMD with token bucket (eg https://github.com/webriots/rate )
Stochastic Fair Blue shares that problem, so I think you might find the solution it uses interesting: it rotates hashes on fixed intervals by re-seeding to ensure that if a responsive flow collides with a non-responsive flow that's being rate-limited, it is at least guaranteed not to do it for very long.
This leads to a problem similar to the problem with Fixed Window Counter rate limiting where it would basically forget the rate limit history every interval. To solve this, two queues are fed input data at a time: the router enforces one of them, while just feeding the other data and ignoring its output in order to warm up its buckets.
I imagine if I tried to make use of the library you linked and didn't absolutely need per-user granularity, I'd do something similar by concatenating the user's identifier with a seed value for each of the two queues and rotating at regular intervals.
Excellent dataviz. Related tangent, "HPBN" (High-Performance Browser Networking) is a great book that includes related concepts. https://hpbn.co/
I think SSE needs expansion with http 3.
> Follow Sagyam's Blog's journey > By following, you'll have instant access to our new posts in your feed. > Continue with Google > More options As soon as I see this in a blog, I quit tab. Why do authors do this to themselves?
Sorry about that that is my blogging platform Hashnode. It was lesser of four evils: - Medium which paywalls the article and forces you to sign up just to read. - Substack has same problem, it's great for funneling people to your paid newsletter but there is a sign up banner as soon the page loads. - Build your own and miss out on the social aspect and there's no proof if the numbers are real.
Earlier quoted context omitted.
I mentioned this in another place on this thread, but a simple AIMD algorithm paired with a token bucket is surprisingly effective at dynamically adjusting to available capacity, even across a fleet of services not sharing state other than the contended resource. Pretty easy to pair AIMD with token bucket (eg https://github.com/webriots/rate )
I notice that library self-identifies a problem of generally keeping a fixed-size number of buckets and addressing them with hashing, which leads to great memory usage but introduces a risk of collision. Stochastic Fair Blue shares that problem, so I think you might find the solution it uses interesting: it rotates hashes on fixed intervals by re-seeding to ensure that if a responsive flow collides with a non-respons…