Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Idempotency is easy until the second request is different

blog.dochia.dev

Post reply on HN
Print view View original

191–200 of 244 posts

Re: Idempotency is easy until the second request is different

#191

Post by jmalicki »

stickfigure wrote: post #185
scott_w wrote: post #179

Earlier quoted context omitted.

Not in the payments world. If you’re 99% done but only the bookkeeping failed, then it’s likely that money is changing hands and you need to deal with that fact. Payments are not an atomic infrastructure and you cannot magic that into reality.

Payments are multistep but each of the steps needs to be atomic. The "create payment" operation must be transactional and the communication channel between you and the processor must be idempotent so you don't inadvertently create multiple payments. The fact that payments have a settlement process is not relevant to this discussion.

And if there is part of the process that isn't idempotent, you want to make that surface area as small as possible, so that only failures at that one discrete step can cause issues.

Re: Idempotency is easy until the second request is different

#192

Post by turtlebits »

pdonis wrote: post #109
stickfigure wrote: post #106

Earlier quoted context omitted.

Retries will only receive 409 if the original request was successful. If the original request failed, the server performs the operation as normal on the second request. It doesn't replay failures. The whole point of the idempotence mechanism is so you can make a reliable distributed system. If the first try fails, the client doesn't know if it succeeded or not, so the client should try again later ("at-least-once").…

What if the original request is still being processed when the retry comes in? That doesn't fall into either of your categories: the request isn't successful, but it hasn't failed either.

You have a transaction row lock and you tell the subsequent requests to f off. I would go as far as returning a 429.

Devs are too scared to be nice (ie not return errors) to clients when they misbehave.

Re: Idempotency is easy until the second request is different

#193

Post by pdonis »

stickfigure wrote: post #189
pdonis wrote: post #183

Earlier quoted context omitted.

> The other cases are the original request is still in flight or never occurred. I'm not sure what you mean by "in flight". The case I'm asking about is where the original request was received by the server and is being processed--and then a second request comes in with the same idempotency key. The original request has not succeeded, and has not failed--it's still in process. What response does the second request ge…

The answer is "the same thing as every other concurrency conflict between two requests". In modern backend development this is most commonly handled by the database, and the practical result is that (from the client's perspective) the requests will block, and only one will "actually succeed". Here's a typical example, assuming serializable isolation in a database that uses optimistic concurrency. * Two simultaneous r…

> The second request starts a transaction and starts processing, everything looks good - no dups.

So when the second request comes in, even though it has the same idempotency key as the first request, the server doesn't check to see if there's already a request received with that idempotency key?

That would seem to defeat the whole purpose of idempotency keys.

> On retry, the second transaction detects the duplicate.

So at this point, the second request would return a 409 code (or something like that) to the client?

Re: Idempotency is easy until the second request is different

#194

Post by amiga386 »

mainde wrote: post #98
amiga386 wrote: post #82

Earlier quoted context omitted.

Yeah but don't let them reify it. Ideally you already send client version in requests (or have an API version prefix). Add the workaround only for legacy clients. Next client version must distinguish itself from predecessor and must not require the bodge to work.

Well.. it was ~6 years and ~10 billion payments ago, the clients have been fixed but the "hack" is still there, it has caused no harm as far as I can tell. Worst case scenario it's useless, best case scenario it prevents regressions. The issue with things that client must not do is that they might still do them, and users don't care whose fault it is. It's important to have auxilliary mechanisms to mitigate these.

That it may be there or not doesn't mean it "caused no harm". It sounds like yet another carbuncle added in haste and then never fixed properly, leading to 6 years of fear of touching it.

If it's truly intended, it needs to be part of the official spec, with a robust justification of why it's there at all. Neither server nor client ought to have unnecessary and undocumented things "just in case", because that breeds a culture of uncertainty.

If you fear client regressions, make it a mandatory part of the client's test suite. You control the client, right?

Re: Idempotency is easy until the second request is different

#195

Post by thaumasiotes »

yladiz wrote: post #165
thaumasiotes wrote: post #159

Earlier quoted context omitted.

> How would the requester get notified if it doesn’t know which request succeeded? By sending a third request and getting a response that reveals the state of the system.

Seems like more overhead than just getting a response from the initial request.

If you get a response from the initial request, then you do know whether it succeeded.

Re: Idempotency is easy until the second request is different

#196

Post by kelnos »

halestock wrote: post #100
stickfigure wrote: post #89

This is all way too much. If you see a duplicate idempotency key, skip the replay and always return 409. This becomes a client problem. Clients already need to help enforce idempotent contracts; "check for conflict response" is not an onerous imposition. I've built multiple ecommerce APIs with this approach and they work great. No heroic measures required. You can often satisfy this contract with a unique constraint;…

But that's not idempotent? If I'm a client and I don't know if the original request went through, getting a 409 on any subsequent requests tells me nothing about whether the original request was successful or not.

You provide in the response body a JSON blob or something that indicates a detailed error code, like DUPLICATE_IDEMPOTENCY_KEY, with some more information that points to the record ID of the corresponding transaction that the client can fetch.

Sure, a strict reading of "idempotence" might require that the response for subsequent requests be identical to the first, but for practical concerns, what matters is the API contract you define, document, and adhere to. The purpose of idempotence is to ensure that you don't end up with duplicate transactions. That's what actually matters. How that's represented in the protocol is an implementation detail.

Re: Idempotency is easy until the second request is different

#197

Post by kelnos »

pdonis wrote: post #109
stickfigure wrote: post #106

Earlier quoted context omitted.

Retries will only receive 409 if the original request was successful. If the original request failed, the server performs the operation as normal on the second request. It doesn't replay failures. The whole point of the idempotence mechanism is so you can make a reliable distributed system. If the first try fails, the client doesn't know if it succeeded or not, so the client should try again later ("at-least-once").…

What if the original request is still being processed when the retry comes in? That doesn't fall into either of your categories: the request isn't successful, but it hasn't failed either.

You still return 409, with error detail in the response body, because it is still true that the client sent multiple requests with the same idempotency key, regardless of the progress of the original request.

Regardless, I think your assumption about how the request/response cycle should be working is wrong. For this kind of API and transaction, the server should be returning a response immediately: 202 Accepted. The only thing the API server should be doing before returning is creating a row in a DB (with a "state" field with an initial value of "pending"), and pushing some work on a queue.

The server should not be sitting there with the HTTP request open, trying to complete the transaction, and only returning a response to the client when the transaction is finished or has encountered an error.

The client will have to learn about the progress of the state of the transaction outside of this initial request. There are many options here: polling, webhooks, a message queue like kinesis or kafka, etc.

Re: Idempotency is easy until the second request is different

#198

Post by kgwxd »

stickfigure wrote: post #89

This is all way too much. If you see a duplicate idempotency key, skip the replay and always return 409. This becomes a client problem. Clients already need to help enforce idempotent contracts; "check for conflict response" is not an onerous imposition. I've built multiple ecommerce APIs with this approach and they work great. No heroic measures required. You can often satisfy this contract with a unique constraint;…

> If you see a duplicate idempotency key

I'm no expert but an "idempotency key" already has some major smell to it.

Re: Idempotency is easy until the second request is different

#200

Post by zinkem »

ahoka wrote: post #30
zinkem wrote: post #3

Idempotency is easy if you don't use mutable state in your middleware. Auth, logging, and atomicity are all isolated concerns that should not affect the domain specific user contract with your API. How you handle unique keys is going to vary by domain and tolerance-- and its probably not going to be the same in every table. It's important to design a database schema that can work independently of your middleware laye…

So idempotency is easy if your service does not do anything useful?

Whether or not your service does something useful is up to you.

A database on it's own is enough for most business applications.

If you haven't seen this yet, you're just rent seeking.

Post reply on HN