Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An Interactive Guide to Rate Limiting

blog.sagyamthapa.com.np

Post reply on HN
Print view View original

21–30 of 48 posts

Re: An Interactive Guide to Rate Limiting

#21

Post by sagyam »

mtlynch wrote: post #18

This seems like someone used AI to generate the article and examples without much review. It's all bullet points, and it repeatedly uses "Working:" as a heading, which doesn't make any sense to me. The site defaults to dark mode for me (I'm assuming based on my system preferences), but all the examples are unusable with dark mode. The examples all seem to be within fixed-width divs that cut off part of the content un…

- I like bullet points, they are easy to read.

- "Working" I wanted to keep things consistent.

- Content getting cut was a limitation of iframe. Most blogging platform don't allows you to embed another page. This was best I could do given the limitation.

- I do use AI to bounce ideas, but a lot of effort went into getting the apps working as intended.

Re: An Interactive Guide to Rate Limiting

#23

Post by tra3 »

I tried to explain the benefits of circuit breakers and adaptive concurrency to improve the performance of our distributed monolith, but I failed. I tried to visualize it using step by step packet diagrams but failed. This is hard stuff to understand.

Great visualization tools. Next time I have to explain it, I'll reach for these.

Re: An Interactive Guide to Rate Limiting

#25

Post by wonnage »

hxtk wrote: post #12

Something I’ve long wondered is why you never hear about rate limiting algorithms that are based on the cost to serve the request or algorithms that dynamically learn the capacity of the system and give everyone a fair share. In the field of router buffer management, there are algorithms like Stochastic Fair Blue, which does the latter, but is somewhat hard to apply to HTTP because you’d have to define a success/fail…

Envoy has a latency-based adaptive concurrency feature: https://www.envoyproxy.io/docs/envoy/latest/configuration/ht...

Netflix has a blog post for their implementation: https://netflixtechblog.medium.com/performance-under-load-3e...

Re: An Interactive Guide to Rate Limiting

#26

Post by remus »

hxtk wrote: post #12

Something I’ve long wondered is why you never hear about rate limiting algorithms that are based on the cost to serve the request or algorithms that dynamically learn the capacity of the system and give everyone a fair share. In the field of router buffer management, there are algorithms like Stochastic Fair Blue, which does the latter, but is somewhat hard to apply to HTTP because you’d have to define a success/fail…

My assumption would be that it is a complexity thing. As a consumer of the service having a rate limit that is easy to understand and write retry logic for is a big plus. If the criteria is "x requests per 5 minute window" and I start getting rate limit errors it's very clear what back off behaviour I need to implement. If the criteria is CPU usage of my requests, as a consumer it's hard for me to reason about how much CPU a given request is going to take so my retry logic is going to be fairly dumb.

Re: An Interactive Guide to Rate Limiting

#27

Post by jsw »

hxtk wrote: post #12

Something I’ve long wondered is why you never hear about rate limiting algorithms that are based on the cost to serve the request or algorithms that dynamically learn the capacity of the system and give everyone a fair share. In the field of router buffer management, there are algorithms like Stochastic Fair Blue, which does the latter, but is somewhat hard to apply to HTTP because you’d have to define a success/fail…

I mentioned this in another place on this thread, but a simple AIMD algorithm paired with a token bucket is surprisingly effective at dynamically adjusting to available capacity, even across a fleet of services not sharing state other than the contended resource.

Pretty easy to pair AIMD with token bucket (eg https://github.com/webriots/rate)

Re: An Interactive Guide to Rate Limiting

#29

Post by mtlynch »

sagyam wrote: post #21
mtlynch wrote: post #18

This seems like someone used AI to generate the article and examples without much review. It's all bullet points, and it repeatedly uses "Working:" as a heading, which doesn't make any sense to me. The site defaults to dark mode for me (I'm assuming based on my system preferences), but all the examples are unusable with dark mode. The examples all seem to be within fixed-width divs that cut off part of the content un…

- I like bullet points, they are easy to read. - "Working" I wanted to keep things consistent. - Content getting cut was a limitation of iframe. Most blogging platform don't allows you to embed another page. This was best I could do given the limitation. - I do use AI to bounce ideas, but a lot of effort went into getting the apps working as intended.

Why "Working?" It's unclear what that means.

Is it supposed to say, "How it works"?

Post reply on HN