Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Retries – An interactive study of request retry methods

encore.dev

Post reply on HN
Print view View original

41–50 of 56 posts

Re: Retries – An interactive study of request retry methods

#41

Post by spockz »

lclarkmichalek wrote: post #9

This still isn't what I'd call "safe". Retries are amazing at supporting clients in handling temporary issues, but horrible for helping them deal with consistently overloaded servers. While jitter & exponential backoff help with the timing, they don't reduce the overall load sent to the service. The next step is usually local circuit breakers. The two easiest to implement are terminating the request if the error rate…

Finagle fixes this with Retry Budgets: https://finagle.github.io/blog/2016/02/08/retry-budgets/

Re: Retries – An interactive study of request retry methods

#42

Post by __turbobrew__ »

whenlambo wrote: post #25
sesm wrote: post #24

Summary of the article: use exponential backoff + jitter for retry intervals. What author didn’t mention: sometimes you want to add jitter to delay the first request too, if the request happens immediately after some event from server (like server waking up). If you don’t do this, you may crash the server, and if your exponential backoff counter is not global you can even put server into cyclic restart.

If you can crash the server with an improperly timed request, then you have a much bigger problem than client-side stuff.

One example is if a datacenter loses power and then all the hosts get turned on at the same time they can all send requests at the same time and crash a server.

Re: Retries – An interactive study of request retry methods

#45

Post by samwho »

Probiotic6081 wrote: post #44

Exponential backoff doesn't apply for successful requests right? The simulation doesn'T reflect that i think. peace

It doesn’t apply to successful requests, that’s right.

The simulation retries failed requests using various retry strategies, and then after a successful request will wait a configured amount before sending the next request.

Re: Retries – An interactive study of request retry methods

#46

Post by fiddlerwoaroof »

marcosdumay wrote: post #30
fiddlerwoaroof wrote: post #19

Earlier quoted context omitted.

Exponential retries can effectively have a maximum number of requests if the gap between retries gets long enough quickly enough. In practice, the user will refresh or close the page if things look broken for too long.

Oh, please don't do that. Unbounded exponential backoff is an horrible experience, and improves basically nothing. If it makes sense to completely fail the request, do it before the waiting becomes noticeable. If it's something that can't just fail, set a maximum waiting time and add jitter.

I think decoupling retry logic from the “there’s something wrong” UI ends up being a better experience than tieing the UI state to the details of network retries. (For one thing, it gives you a chance to fix the “everything is broken” UI without any action on the user’s part.

Re: Retries – An interactive study of request retry methods

#48

Post by codebeaker »

samwho wrote: post #13
codebeaker wrote: post #11

Earlier quoted context omitted.

What technology did you use for the animations? I've a bunch of itches I'd like to scratch that would be improved by having some canvas animated explainers or UI but I never clicked with anything. D3 back in the day. A rudimentary look in the source code showed a element but I'm not up to date enough with web standards to guess where to look for that in your JS bundle to guess at the framework!

It uses PixiJS ( https://pixijs.com/ ) for the 2D rendering and GSAP3 ( https://gsap.com/ ) for the animation. The blocks are custom HTMl elements ( https://developer.mozilla.org/en-US/docs/Web/API/Web_compone... ) which I use to encapsulate the logic. I've been thinking about creating a separate repo to house the source code of posts I've finished so people can see it. I don't like all the bundling and minification…

Thank you so much for the detailed reply. pixijs looks amazing, and gsap looks pretty approachable!

Re: Retries – An interactive study of request retry methods

#49

Post by rewmie »

tyingq wrote: post #20

This is one of those things that sort of exposes our industry maturity versus other engineering that's been around longer. You would think by now that the various frameworks for remote calls would have standardized down to include the best practice retry patterns, with standard names, setting ranges, etc. But we mostly still roll our own for most languages/frameworks. And that's full of footguns around DNS caching, w…

> You would think by now that the various frameworks for remote calls would have standardized down to include the best practice retry patterns, with standard names, setting ranges, etc.

There is a school of thought that argues that the best retry pattern is no retry at all, and just get the client to fail and handle that state.

One of the driving arguments is that retries are a lazy way to try to move faults from the client onto the server, and in the process cause more harm (i.e., DDoS).

Sometimes complex means wrong, and all these retry strategies are getting progressively more complex at the expense of hammering servers with traffic way beyond the volume it's designed to handle. How is that a decent tradeoff?

Re: Retries – An interactive study of request retry methods

#50

Post by pixel8account »

rewmie wrote: post #49
tyingq wrote: post #20

This is one of those things that sort of exposes our industry maturity versus other engineering that's been around longer. You would think by now that the various frameworks for remote calls would have standardized down to include the best practice retry patterns, with standard names, setting ranges, etc. But we mostly still roll our own for most languages/frameworks. And that's full of footguns around DNS caching, w…

> You would think by now that the various frameworks for remote calls would have standardized down to include the best practice retry patterns, with standard names, setting ranges, etc. There is a school of thought that argues that the best retry pattern is no retry at all, and just get the client to fail and handle that state. One of the driving arguments is that retries are a lazy way to try to move faults from the…

I disagree. I think the trade-off is very reasonable. At some point you need to retry (even if the trigger is user manually pressing F5 in the browser/clicking a button again/running a program again). Because they actually have some goal to accomplish.

Some failures really are random, let's say 0.1% of requests fail. For a sufficiently complex backend/operation, one user request can easily generate 100 internal requests that can fail. If you don't retry, this adds up to a non-negliglible chance that a whole user facing operation fails and all 100 requests have to be retried - you actually increased the number of requests that had to be made! As an extreme example, imagine that during training ChatGPT one request failed, and whole training has to be started from scratch because we don't do retries.

Post reply on HN