Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Idempotency is easy until the second request is different

blog.dochia.dev

Post reply on HN
Print view View original

181–190 of 244 posts

Re: Idempotency is easy until the second request is different

#181

Post by [deleted] »

jeremyjh wrote: post #175
stickfigure wrote: post #89

This is all way too much. If you see a duplicate idempotency key, skip the replay and always return 409. This becomes a client problem. Clients already need to help enforce idempotent contracts; "check for conflict response" is not an onerous imposition. I've built multiple ecommerce APIs with this approach and they work great. No heroic measures required. You can often satisfy this contract with a unique constraint;…

I really like that article, thank you for sharing it again. I wish I had read it a decade ago, or even the first time you submitted it - I had to learn some of these things the hard way. I agree with you - I don't think Stripe has made the right choices here and its unfortunate that it has inspired so many other people to make the same poor choices. I don't agree that their system is as sound as always returning 409s…

[deleted]

Re: Idempotency is easy until the second request is different

#182

Post by [deleted] »

pdonis wrote: post #156
stickfigure wrote: post #146

Earlier quoted context omitted.

> special cases where the original request is still in process This isn't a special case, and it's the same problem if you want to replay the original response on conflict. If the original request isn't complete, what are you going to replay ?

> If the original request isn't complete, what are you going to replay? Who says you have to replay? If you get a second request with the same idempotency key, and the original request is still in process, why not just send the client a response that says so?

[deleted]

Re: Idempotency is easy until the second request is different

#183

Post by pdonis »

graemefawcett wrote: post #170
pdonis wrote: post #155

Earlier quoted context omitted.

> I'm not assuming anything. Sure you are. You said: "Retries will only receive 409 if the original request was successful. If the original request failed, the server performs the operation as normal on the second request. It doesn't replay failures." I understand all that just fine; you don't need to keep trying to "reframe" it. But what you said that I just quoted above assumes, implicitly, that if you get a second…

The other cases are the original request is still in flight or never occurred. The former case was explained by the prior comment, one request is processed, the other is returned by 409. The system cares little for which is which and neither should the caller. The latter case is handled by clients retrying until a request is received, at which point one of the other three states takes over. Whether or not a prior req…

> The other cases are the original request is still in flight or never occurred.

I'm not sure what you mean by "in flight". The case I'm asking about is where the original request was received by the server and is being processed--and then a second request comes in with the same idempotency key. The original request has not succeeded, and has not failed--it's still in process. What response does the second request get? I do not see an answer to that question anywhere in this thread.

Re: Idempotency is easy until the second request is different

#184

Post by pdonis »

Groxx wrote: post #154
pdonis wrote: post #123

Earlier quoted context omitted.

> If the original request hasn't gotten a response yet, why is it sending a retry? Um, because connections over the Internet aren't 100% always on? Because packets can get lost? Because computers sometimes have to reboot? You're assuming that the client will always receive whatever response your server finally sends, and that the client will wait indefinitely to receive a response. Neither of those things are true. S…

You're asking questions that broadly summarize as: "what if we had an idempotency key [which must work concurrently to be useful], but it didn't work concurrently?" Idempotency keys are themselves the solution you're looking for. If they don't work concurrently, they aren't idempotency keys. Your response in races or duplicates doesn't inherently matter in that sense, pick whatever semantics make sense for your syste…

> You're asking questions that broadly summarize as

No, I'm asking one question, which doesn't seem to be summarized by your summary.

The situation is that your server has received two requests with the same idempotency key. For the first request, one of three things could be true: it could have succeeded, it could have failed, or it could still be in process.

The original post I responded to said what response the second request gets if the first request succeeded and if it failed. But it didn't say what response the second request gets if the first request is still in process on the server--so it hasn't succeeded and it hasn't failed. I do not see an answer to that anywhere in this thread.

Re: Idempotency is easy until the second request is different

#185

Post by stickfigure »

scott_w wrote: post #179
jmalicki wrote: post #171

Earlier quoted context omitted.

Your database should not allow both commits to happen - one should get rolled back. If it processed 99% of the request and the final bookkeeping failed because of a duplicate, that's still a failed request. Arguably this should be the primary way you check for idempotent requests - you shouldn't have a separate check for existence, you should have the insert/update fail atomically. This is the same thing you see on f…

Not in the payments world. If you’re 99% done but only the bookkeeping failed, then it’s likely that money is changing hands and you need to deal with that fact. Payments are not an atomic infrastructure and you cannot magic that into reality.

Payments are multistep but each of the steps needs to be atomic. The "create payment" operation must be transactional and the communication channel between you and the processor must be idempotent so you don't inadvertently create multiple payments.

The fact that payments have a settlement process is not relevant to this discussion.

Re: Idempotency is easy until the second request is different

#186

Post by Groxx »

pdonis wrote: post #184
Groxx wrote: post #154

Earlier quoted context omitted.

You're asking questions that broadly summarize as: "what if we had an idempotency key [which must work concurrently to be useful], but it didn't work concurrently?" Idempotency keys are themselves the solution you're looking for. If they don't work concurrently, they aren't idempotency keys. Your response in races or duplicates doesn't inherently matter in that sense, pick whatever semantics make sense for your syste…

> You're asking questions that broadly summarize as No, I'm asking one question, which doesn't seem to be summarized by your summary. The situation is that your server has received two requests with the same idempotency key. For the first request, one of three things could be true: it could have succeeded, it could have failed, or it could still be in process. The original post I responded to said what response the s…

If the first is still in progress and a second arrives, you can wait for it to finish (and then return whatever would be useful, e.g. you could replay the first request's response), or fail the second (409, 423, 429, there are a number of codes that could apply) to tell the caller to retry later.

So yeah, you can do basically anything that isn't inconsistent. Success, fail, delay, don't respond until timeout, all are valid as long as you don't double-apply. Most concurrent systems are like this in some way, because all successes can become errors, and all responses might never arrive. It has nothing really to do with idempotency.

Re: Idempotency is easy until the second request is different

#188

Post by reactordev »

stickfigure wrote: post #89

This is all way too much. If you see a duplicate idempotency key, skip the replay and always return 409. This becomes a client problem. Clients already need to help enforce idempotent contracts; "check for conflict response" is not an onerous imposition. I've built multiple ecommerce APIs with this approach and they work great. No heroic measures required. You can often satisfy this contract with a unique constraint;…

This is because there are countless tutorials and slop on the internet that says, no, instructs, you to handle idempotency on the client instead of where it belongs. Server-side.

Re: Idempotency is easy until the second request is different

#189

Post by stickfigure »

pdonis wrote: post #183
graemefawcett wrote: post #170

Earlier quoted context omitted.

The other cases are the original request is still in flight or never occurred. The former case was explained by the prior comment, one request is processed, the other is returned by 409. The system cares little for which is which and neither should the caller. The latter case is handled by clients retrying until a request is received, at which point one of the other three states takes over. Whether or not a prior req…

> The other cases are the original request is still in flight or never occurred. I'm not sure what you mean by "in flight". The case I'm asking about is where the original request was received by the server and is being processed--and then a second request comes in with the same idempotency key. The original request has not succeeded, and has not failed--it's still in process. What response does the second request ge…

The answer is "the same thing as every other concurrency conflict between two requests". In modern backend development this is most commonly handled by the database, and the practical result is that (from the client's perspective) the requests will block, and only one will "actually succeed".

Here's a typical example, assuming serializable isolation in a database that uses optimistic concurrency.

* Two simultaneous requests come in to create a payment.

* The requests provide an idempotency key that is expected to be unique (possibly scoped to a tenant).

* The first request starts a transaction and starts processing, everything looks good - no dups.

* The second request starts a transaction and starts processing, everything looks good - no dups.

* The first one commits and returns success.

* The second tries to commit, but a conflict is detected (the first txn committed first). Typically this causes the second transaction to retry.

* On retry, the second transaction detects the duplicate.

The only question here is what happens when the second transaction fails? The Stripe model is "look up the original response and hand that back to the client". An equally valid and much easier to implement solution is "return a response that tells the client that there was a conflict".

Both solutions offer "create payment" as an idempotent operation.

Re: Idempotency is easy until the second request is different

#190

Post by malux85 »

stickfigure wrote: post #89

This is all way too much. If you see a duplicate idempotency key, skip the replay and always return 409. This becomes a client problem. Clients already need to help enforce idempotent contracts; "check for conflict response" is not an onerous imposition. I've built multiple ecommerce APIs with this approach and they work great. No heroic measures required. You can often satisfy this contract with a unique constraint;…

Exactly - junior engineers haven't yet developed a good taste of "how much the system should try and deal with errors" - so they always go way overboard to guess and try and fix errors way too much. I agree with you - this is a client error - 409 and make them fix the bug on their side. Clean, reliable, simple, separation of responsibility.
Post reply on HN