Live data from Hacker News

Retries – An interactive study of request retry methods

encore.dev

51–56 of 56 posts

Re: Retries – An interactive study of request retry methods

#51
post #49

Earlier quoted context omitted.

> You would think by now that the various frameworks for remote calls would have standardized down to include the best practice retry patterns, with standard names, setting ranges, etc. There is a school of thought that argues that the best retry pattern is no retry at all, and just get the client to fail and handle that state. One of the driving arguments is that retries are a lazy way to try to move faults from the…

I disagree. I think the trade-off is very reasonable. At some point you need to retry (even if the trigger is user manually pressing F5 in the browser/clicking a button again/running a program again). Because they actually have some goal to accomplish. Some failures really are random, let's say 0.1% of requests fail. For a sufficiently complex backend/operation, one user request can easily generate 100 internal reque…

> I disagree. I think the trade-off is very reasonable. At some point you need to retry (even if the trigger is user manually pressing F5 in the browser/clicking a button again/running a program again). Because they actually have some goal to accomplish.

I don't think your belief holds water if you think about your example. The goal of a retry from a client standpoint is to introduce an acceptable delay in order to pretend the original request was successful. This strategy is only valid if the number of retries are enough to not penalize perceived performance or the normal operational state of a service. Consequently, all retry strategies involve sending multiple requests per second. The link to Retry Budgets posted in this discussion explicitly mentions "a minimum of 10 retries per second."

A user pressing F5 will never generate this volume of requests.

> Some failures really are random, let's say 0.1% of requests fail.

That's why failing fast and not retry is the best strategy for most if not all applications. Retry strategies introduce high levels of complexity to a task that only rarely happens, and in the rare case that it happens it can be trivially fixed by the user triggering a refresh.

If it's an applications that already outputs a high volume of requests, once your first request fails then it will simply post again a request as part of their happy path.

Some developers like retries because they use it to patch their broken code path to pretend that they do not have to deal with scenarios where a network is not 100% reliable. They onboard a retry library, they update their requests to transparently appear to be a single request, and they proceed as if their application doesn't have a failure mode. Except it does, but now they also decide to tradeoff their wishful thinking with higher risk of causing a cascading DDoS attack on their own infrastructure.

Re: Retries – An interactive study of request retry methods

#52
post #49

Earlier quoted context omitted.

> You would think by now that the various frameworks for remote calls would have standardized down to include the best practice retry patterns, with standard names, setting ranges, etc. There is a school of thought that argues that the best retry pattern is no retry at all, and just get the client to fail and handle that state. One of the driving arguments is that retries are a lazy way to try to move faults from the…

I disagree. I think the trade-off is very reasonable. At some point you need to retry (even if the trigger is user manually pressing F5 in the browser/clicking a button again/running a program again). Because they actually have some goal to accomplish. Some failures really are random, let's say 0.1% of requests fail. For a sufficiently complex backend/operation, one user request can easily generate 100 internal reque…

[deleted]

Re: Retries – An interactive study of request retry methods

#53
post #16

Earlier quoted context omitted.

> This is the client side of things. The article is about making requests, and strategies to implement when the request fails. By definition, these are clients. Was there any ambiguity? > But there is an additional piece of info everyone who writes clients needs to see: And that's what people like me, who implement backend services, may do if clients ignore such wisdom. I don't think this is the obscure detail you ar…

> Was there any ambiguity? Did I say there was? > I don't think this is the obscure detail you are making it out to be Where did I call this detail "obscure"? My post is meant as a light-hearted, humorous note pointing out one of the many reasons why it is in general a good idea for clients to implement the principles outlined in the article.

> Did I say there was?

Yes. You explicitly wrote in your comment "This is the client side of things", as if there was any ambiguity in where requests came from.

> Where did I call this detail "obscure"?

You explicitly wrote that "(...) there is an additional piece of info everyone who writes clients needs to see" on what people like you "who implement backend services, may do if clients ignore such wisdom", as if somehow this was obscure, arcane and secret knowledge that no team whatsoever working on backend services with client teams ever dared share with the outside world.

> (...) why it is in general a good idea for clients to implement the principles outlined in the article.

I don't think there is a single developer out there working on client/server projects that aren't aware of the need to handle request failures and be mindful of service level agreements, specially when dealing with retries.

Your post reads like addressing life guards to let them know that there is an additional piece of info everyone who works as a lifeguard needs to see: that the water is wet.

Re: Retries – An interactive study of request retry methods

#54
post #51

Earlier quoted context omitted.

I disagree. I think the trade-off is very reasonable. At some point you need to retry (even if the trigger is user manually pressing F5 in the browser/clicking a button again/running a program again). Because they actually have some goal to accomplish. Some failures really are random, let's say 0.1% of requests fail. For a sufficiently complex backend/operation, one user request can easily generate 100 internal reque…

> I disagree. I think the trade-off is very reasonable. At some point you need to retry (even if the trigger is user manually pressing F5 in the browser/clicking a button again/running a program again). Because they actually have some goal to accomplish. I don't think your belief holds water if you think about your example. The goal of a retry from a client standpoint is to introduce an acceptable delay in order to p…

> That's why failing fast and not retry is the best strategy for most if not all applications.

I think it's more complex than this. You also have to lump timeouts, caching and failure behavior into the conversation. And there are also situations where you absolutely need some amount of retries. Say, for example, you want seamless failover between backends...you're expecting some failures and don't want or need to expose those to your end users. Or, maybe the "end user" isn't a person. Like, for example, finalizing a financial transaction from a queue.

Re: Retries – An interactive study of request retry methods

#56
post #2

Thanks for sharing! I’m the author of this post, and happy to answer any questions :)

Thanks for this -- it's really great!

One thing I noticed is that the post is very first-principles right up to where it reaches exponential backoff. At that point, it quickly jumps to "and here's exponential backoff and here's some good parameters". But I've worked on a lot of systems that got those wrong. In both directions: too-short caps that were insufficient for the underlying system to recover and too-long caps so that even when the servers _did_ recover, clients weren't even going to try again for way too long (e.g., 2 days). It'd be neat to have another section or two exploring those tradeoffs.

I really want one of these visual explorations for the idea of margin. Concretely: it's common to have systems at, say, 88% CPU utilization that appear to be working great. Then you ramp them up to like 92% and start seeing latency bubbles of multiple seconds or even tens of seconds. We tend to think of that idle time as waste, but it's essential for surviving transient blips in load. I increasingly feel like this concept is really fundamental and ought to be taught in like high school because it applies so many places (e.g., emergency funds, in the realm of personal finance).

Post reply on HN