Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Retries – An interactive study of request retry methods

encore.dev

Post reply on HN
Print view View original

21–30 of 56 posts

Re: Retries – An interactive study of request retry methods

#21

Post by usrbinbash »

rewmie wrote: post #16
usrbinbash wrote: post #14

This is the client side of things. And I think this is a great resource that everyone who writes clients for anything, should see. But there is an additional piece of info everyone who writes clients needs to see: And that's what people like me, who implement backend services, may do if clients ignore such wisdom. Because: I'm not gonna let bad clients break my service. What that means in practice: Clients are given…

> This is the client side of things. The article is about making requests, and strategies to implement when the request fails. By definition, these are clients. Was there any ambiguity? > But there is an additional piece of info everyone who writes clients needs to see: And that's what people like me, who implement backend services, may do if clients ignore such wisdom. I don't think this is the obscure detail you ar…

> Was there any ambiguity?

Did I say there was?

> I don't think this is the obscure detail you are making it out to be

Where did I call this detail "obscure"?

My post is meant as a light-hearted, humorous note pointing out one of the many reasons why it is in general a good idea for clients to implement the principles outlined in the article.

Re: Retries – An interactive study of request retry methods

#22

Post by samwho »

usrbinbash wrote: post #21
rewmie wrote: post #16

Earlier quoted context omitted.

> This is the client side of things. The article is about making requests, and strategies to implement when the request fails. By definition, these are clients. Was there any ambiguity? > But there is an additional piece of info everyone who writes clients needs to see: And that's what people like me, who implement backend services, may do if clients ignore such wisdom. I don't think this is the obscure detail you ar…

> Was there any ambiguity? Did I say there was? > I don't think this is the obscure detail you are making it out to be Where did I call this detail "obscure"? My post is meant as a light-hearted, humorous note pointing out one of the many reasons why it is in general a good idea for clients to implement the principles outlined in the article.

Throttling, tarpitting, and circuit-breakers are something I'd love to visualise in future, too. Throttling on its own is such a massive topic!

Re: Retries – An interactive study of request retry methods

#23

Post by cowsandmilk »

samwho wrote: post #10
lclarkmichalek wrote: post #9

This still isn't what I'd call "safe". Retries are amazing at supporting clients in handling temporary issues, but horrible for helping them deal with consistently overloaded servers. While jitter & exponential backoff help with the timing, they don't reduce the overall load sent to the service. The next step is usually local circuit breakers. The two easiest to implement are terminating the request if the error rate…

Totally! Thanks for bringing those up. I tried to keep the scope specifically on retries and client-side mitigation. There's a whole bunch of cool stuff to visualise on the server-side, and I'm hoping to get to it in the future.

Your response makes it sound like you think circuit breakers are server side and not related to retries. They are not; they are a client-side mitigation that are a critical part of a mature retry library.

Re: Retries – An interactive study of request retry methods

#24

Post by sesm »

Summary of the article: use exponential backoff + jitter for retry intervals.

What author didn’t mention: sometimes you want to add jitter to delay the first request too, if the request happens immediately after some event from server (like server waking up). If you don’t do this, you may crash the server, and if your exponential backoff counter is not global you can even put server into cyclic restart.

Re: Retries – An interactive study of request retry methods

#25

Post by whenlambo »

sesm wrote: post #24

Summary of the article: use exponential backoff + jitter for retry intervals. What author didn’t mention: sometimes you want to add jitter to delay the first request too, if the request happens immediately after some event from server (like server waking up). If you don’t do this, you may crash the server, and if your exponential backoff counter is not global you can even put server into cyclic restart.

If you can crash the server with an improperly timed request, then you have a much bigger problem than client-side stuff.

Re: Retries – An interactive study of request retry methods

#26

Post by andenacitelli »

whenlambo wrote: post #25
sesm wrote: post #24

Summary of the article: use exponential backoff + jitter for retry intervals. What author didn’t mention: sometimes you want to add jitter to delay the first request too, if the request happens immediately after some event from server (like server waking up). If you don’t do this, you may crash the server, and if your exponential backoff counter is not global you can even put server into cyclic restart.

If you can crash the server with an improperly timed request, then you have a much bigger problem than client-side stuff.

Yes. Worst that should happen is getting a 404 or something. A crash due to requesting a piece of data that has not yet been created is poor design.

Re: Retries – An interactive study of request retry methods

#27

Post by samwho »

whenlambo wrote: post #25
sesm wrote: post #24

Summary of the article: use exponential backoff + jitter for retry intervals. What author didn’t mention: sometimes you want to add jitter to delay the first request too, if the request happens immediately after some event from server (like server waking up). If you don’t do this, you may crash the server, and if your exponential backoff counter is not global you can even put server into cyclic restart.

If you can crash the server with an improperly timed request, then you have a much bigger problem than client-side stuff.

I think what they mean is something that would cause client to do something at the same time (could be all sorts, some synchronised crash, aligning timers to clock-time, etc.). If the requests aren't user-driven then yes, you likely would want to include some jitter in the first request too.

Funnily, you'll notice that some of the visualisations have the clients staggering their first request. It's exactly for this reason. I wanted the visualisations to be as deterministic as possible while still feeling somewhat realistic. This staggering was a bit of a compromise.

Not sure what is meant by "if your exponential backoff counter is not global", though. Would love to know more about that.

Re: Retries – An interactive study of request retry methods

#28

Post by [deleted] »

samwho wrote: post #27
whenlambo wrote: post #25

Earlier quoted context omitted.

If you can crash the server with an improperly timed request, then you have a much bigger problem than client-side stuff.

I think what they mean is something that would cause client to do something at the same time (could be all sorts, some synchronised crash, aligning timers to clock-time, etc.). If the requests aren't user-driven then yes, you likely would want to include some jitter in the first request too. Funnily, you'll notice that some of the visualisations have the clients staggering their first request. It's exactly for this r…

[deleted]

Re: Retries – An interactive study of request retry methods

#29

Post by korm »

cowsandmilk wrote: post #23
samwho wrote: post #10

Earlier quoted context omitted.

Totally! Thanks for bringing those up. I tried to keep the scope specifically on retries and client-side mitigation. There's a whole bunch of cool stuff to visualise on the server-side, and I'm hoping to get to it in the future.

Your response makes it sound like you think circuit breakers are server side and not related to retries. They are not; they are a client-side mitigation that are a critical part of a mature retry library.

The client can track its own error rate to the service, but it would need information from a server to get the overall health of the service, which is what the author probably means. Furthermore the load balancer can add a Retry-After header to have more control over the client's retries.

Re: Retries – An interactive study of request retry methods

#30

Post by marcosdumay »

fiddlerwoaroof wrote: post #19
samwho wrote: post #12

Earlier quoted context omitted.

You know, I hadn't actually considered mentioning it. Another commenter brought it up, too. It's so second nature I forgot about it entirely. I'll look about giving it a nod in the text, thank you for the feedback. :)

Exponential retries can effectively have a maximum number of requests if the gap between retries gets long enough quickly enough. In practice, the user will refresh or close the page if things look broken for too long.

Oh, please don't do that.

Unbounded exponential backoff is an horrible experience, and improves basically nothing.

If it makes sense to completely fail the request, do it before the waiting becomes noticeable. If it's something that can't just fail, set a maximum waiting time and add jitter.

Post reply on HN