Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Idempotency is easy until the second request is different

blog.dochia.dev

Post reply on HN
Print view View original

171–180 of 244 posts

Re: Idempotency is easy until the second request is different

#171

Post by jmalicki »

pdonis wrote: post #109
stickfigure wrote: post #106

Earlier quoted context omitted.

Retries will only receive 409 if the original request was successful. If the original request failed, the server performs the operation as normal on the second request. It doesn't replay failures. The whole point of the idempotence mechanism is so you can make a reliable distributed system. If the first try fails, the client doesn't know if it succeeded or not, so the client should try again later ("at-least-once").…

What if the original request is still being processed when the retry comes in? That doesn't fall into either of your categories: the request isn't successful, but it hasn't failed either.

Your database should not allow both commits to happen - one should get rolled back.

If it processed 99% of the request and the final bookkeeping failed because of a duplicate, that's still a failed request.

Arguably this should be the primary way you check for idempotent requests - you shouldn't have a separate check for existence, you should have the insert/update fail atomically.

This is the same thing you see on filesystems for TOCTOU security holes - the right way is to atomically access and modify once, and you only know the request was already processed because that fails.

Re: Idempotency is easy until the second request is different

#172

Post by whoamii »

stickfigure wrote: post #149
whoamii wrote: post #138

Earlier quoted context omitted.

The topic is about idempotency though, and what you are describing is not idempotency. You are describing something different, and arguing “but it accomplishes a similar thing.” It appears you have come to the same conclusion as the article that building idempotency isn’t trivial.

It absolutely is idempotent behavior of the system. The goal is "make an idempotent payment", and this approach ("return an error for duplicates") was standard for financial APIs before Stripe. It still dominates for ecommerce APIs. It isn't complicated, though I can see how if your entire experience with financial APIs is Stripe, you might not be aware of how simple it is. Because Stripe's approach, while mildly mor…

You seem to believe idempotency is a fancy word for avoiding duplicates, which is why we seem to be having different conversations.

Re: Idempotency is easy until the second request is different

#173

Post by hn_throwaway_99 »

saltcured wrote: post #168
hn_throwaway_99 wrote: post #143

Earlier quoted context omitted.

To be honest, I liked your original response about returning a 409 - it's not something I'd done before and I like how it keeps things simpler. But your follow up responses here are making me rethink. Now you have to have all these special cases where the original request is still in process. I think or assertion of "99% are simple POST operations" is bullshit. For the times where idempotency is hard and really matte…

Idempotence already requires some client thinking if you want to conform to HTTP specs. I.e. idempotent DELETE with proper protocol behavior requires that one request see the 200 OK or 204 No Content and the other sees 404 Not Found, because the delete has already happened. It would be misleading to say 200 OK to both, because that answer means the resource was there when the request arrived. Honestly, the whole HTTP…

> The cleanest mapping in the spirit of HTTP would be that you do multiple round trips. A POST creates a new idempotence context, a bit like "start a transaction". The new URI is the key for coordinating state change and allowing restart/recovery.

I think that gave me "Enterprise Java Beans PTSD". I.e. an over-engineered solution that adds complexity for both the client and server in the name of some sort of "protocol purity".

People bolted on idempotent semantics onto HTTP because it wasn't provided natively by the protocol, so I don't think it makes sense to go through some hoop-jumping gymnastics for the sake of conforming to a spec that doesn't describe the necessary semantics in the first place.

Re: Idempotency is easy until the second request is different

#175

Post by jeremyjh »

stickfigure wrote: post #89

This is all way too much. If you see a duplicate idempotency key, skip the replay and always return 409. This becomes a client problem. Clients already need to help enforce idempotent contracts; "check for conflict response" is not an onerous imposition. I've built multiple ecommerce APIs with this approach and they work great. No heroic measures required. You can often satisfy this contract with a unique constraint;…

I really like that article, thank you for sharing it again. I wish I had read it a decade ago, or even the first time you submitted it - I had to learn some of these things the hard way.

I agree with you - I don't think Stripe has made the right choices here and its unfortunate that it has inspired so many other people to make the same poor choices. I don't agree that their system is as sound as always returning 409s. I think having a short window where you return response bodies is fine, but after that they should still be sending 409s. If no one will ever actually resend a request after 24 hours how is it not fine to send 409s when they do? They've chosen to implement the more expensive choice and then not back it with the cheapest one.

Re: Idempotency is easy until the second request is different

#176

Post by mainde »

stingraycharles wrote: post #122
mainde wrote: post #81

Earlier quoted context omitted.

IMHO it's more: fix problems, or at least mitigate them, regardless whose problem it is. I've been in this situation, a clientside bug meant that different requests arrived with the same idempotency key. In my case, updating the client would have taken weeks, in the best case scenario. Updating the backend to check for a matching request body would have taken minutes, maybe hours. It took me a surprising amount of ar…

> In my case, updating the client would have taken weeks, in the best case scenario. Updating the backend to check for a matching request body would have taken minutes, maybe hours. Then at least admit you’re just hacking quickly fixes, creating technical debt, and not fixing the actual problem. I agree with your point that business interest is most important, I disagree that it’s the technically most appropriate sol…

Eh, idk, I wouldn't classify these fixes as hacks nor as technical debt. It's labels that only work from a partial perspective. IMHO a solution that expects perfect compliance is not really complete, it's not good enough to put all the burden on the client, idempotency keys are part of the solution, but not the solution. So, in this sense I would say it is a technical problem.

Re: Idempotency is easy until the second request is different

#178

Post by scott_w »

pdonis wrote: post #109
stickfigure wrote: post #106

Earlier quoted context omitted.

Retries will only receive 409 if the original request was successful. If the original request failed, the server performs the operation as normal on the second request. It doesn't replay failures. The whole point of the idempotence mechanism is so you can make a reliable distributed system. If the first try fails, the client doesn't know if it succeeded or not, so the client should try again later ("at-least-once").…

What if the original request is still being processed when the retry comes in? That doesn't fall into either of your categories: the request isn't successful, but it hasn't failed either.

That’s because everyone here thinking about payments completely incorrectly. They’re not atomic and your server shouldn’t pretend they are.

You need to store the payment state at each relevant step and process it asynchronously. If requests time out, you check the status of it using the key you store (with the processor) to see if it was even received.

It’s not perfect, some processors will 500 while processing the payment (Braintree), so you still need reconciliation on the backend.

Re: Idempotency is easy until the second request is different

#179

Post by scott_w »

jmalicki wrote: post #171
pdonis wrote: post #109

Earlier quoted context omitted.

What if the original request is still being processed when the retry comes in? That doesn't fall into either of your categories: the request isn't successful, but it hasn't failed either.

Your database should not allow both commits to happen - one should get rolled back. If it processed 99% of the request and the final bookkeeping failed because of a duplicate, that's still a failed request. Arguably this should be the primary way you check for idempotent requests - you shouldn't have a separate check for existence, you should have the insert/update fail atomically. This is the same thing you see on f…

Not in the payments world. If you’re 99% done but only the bookkeeping failed, then it’s likely that money is changing hands and you need to deal with that fact. Payments are not an atomic infrastructure and you cannot magic that into reality.

Re: Idempotency is easy until the second request is different

#180

Post by jmalicki »

yladiz wrote: post #150
chii wrote: post #142

Earlier quoted context omitted.

In my opinion, the idea of idempotency is to accept both requests, but only one is actioned (and the requester is non-the-wiser about which). Otherwise, you're just recreating database transactions - something that doesn't need to be named idempotency. And you haven't considered multiple servers in your scenario - what if two requests meant to be idempotent with each other arrived at different servers?

How would the requester get notified if it doesn’t know which request succeeded? Is it listening for events? And at the sake of repeating the above commenter, you solve the multiple server by serializing somewhere, because you ultimately need a lock on something. You can also perform the operation in both places and then reconcile the state later but that’s a lot more complex.

The requester doesn't want to know which request succeeded, because they are duplicates and one is a retry!

When you are using TCP, and you send the same data twice because of a delayed ack, you likewise don't care if the ACK is for the first time or the second time you sent the data. You just know the other side got the data, and that's all you care about.

Post reply on HN