Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Idempotency is easy until the second request is different

blog.dochia.dev

Post reply on HN
Print view View original

201–210 of 244 posts

Re: Idempotency is easy until the second request is different

#201

Post by zinkem »

mrkeen wrote: post #67
ahoka wrote: post #30

Earlier quoted context omitted.

So idempotency is easy if your service does not do anything useful?

It's just the horrible misapplication of the term 'stateless' to a wrapper around something very-much stateful. It's here to stay. (Though I do disagree with the original premise too. Putting on a 'stateless' boxing glove won't mean there's no difference between punching a guy once or twice)

> Putting on a 'stateless' boxing glove won't mean there's no difference between punching a guy once or twice

There are still side effects in the system, of course.

But what your database looks like afterwards is the important part.

Can you recover lost data, replay transactions, undo, etc etc?

Re: Idempotency is easy until the second request is different

#202

Post by stickfigure »

pdonis wrote: post #156
stickfigure wrote: post #146

Earlier quoted context omitted.

> special cases where the original request is still in process This isn't a special case, and it's the same problem if you want to replay the original response on conflict. If the original request isn't complete, what are you going to replay ?

> If the original request isn't complete, what are you going to replay? Who says you have to replay? If you get a second request with the same idempotency key, and the original request is still in process, why not just send the client a response that says so?

It's not a terrible question. It would be complicated to implement and isn't more useful than using the database's consistency model, so nobody does it that way.

Long running transactions create all sorts of problems, so transactions are generally expected to be short. The actual work behind "create payment" or "create order" is generally fairly trivial - more or less insert a row in a table. There's no good reason to make the API complicated... you either "win" at concurrency or you lose, and the difference is generally sub-millisecond. The only meaningful thing you need to communicate to the client is "you're done" (for both the win and lose cases) or "you need to try again" (for the "something unexpected went wrong" case).

Complicated workflows can certainly have multiple steps, with "fetch the current status" calls in between. But somewhere near the beginning of every complicated workflow there will be a call to "create workflow" and it will need to have sort of mechanism which allows clients to call it idempotently. Otherwise you end up with multiple starts.

I've literally received duplicate products in the mail because of this kind of problem. I've also sent multiple products in the mail because services I relied on didn't offer the necessary idempotency mechanisms.

Re: Idempotency is easy until the second request is different

#203

Post by stickfigure »

pdonis wrote: post #193
stickfigure wrote: post #189

Earlier quoted context omitted.

The answer is "the same thing as every other concurrency conflict between two requests". In modern backend development this is most commonly handled by the database, and the practical result is that (from the client's perspective) the requests will block, and only one will "actually succeed". Here's a typical example, assuming serializable isolation in a database that uses optimistic concurrency. * Two simultaneous r…

> The second request starts a transaction and starts processing, everything looks good - no dups. So when the second request comes in, even though it has the same idempotency key as the first request, the server doesn't check to see if there's already a request received with that idempotency key? That would seem to defeat the whole purpose of idempotency keys. > On retry, the second transaction detects the duplicate.…

There are a lot of ways to implement this, so I posted an example with one of the most common ways - a database which uses optimistic concurrency in serializable isolation level. Postgres is often configured this way, though it's not the only way it can be configured.

With optimistic concurrency models, collisions are only detected at commit time. Two transactions can simultaneously update the same data; each update will "succeed"; when they try to commit, only the first one will succeed. The second one will fail with a code that indicates a collision. Standard practice is to just retry the transaction.

In serializable isolation, every transaction sees the state of the database frozen in time at the start of the transaction. They don't see each other's writes (that would be "read committed"). So if you have two transactions simultaneously which do "check if value XYZ exists; if it doesn't exist, insert it" they will both run the insert. The collision will only be detected when the second transaction tries to commit.

There are many other ways to implement this, but this is a pretty common approach.

>> On retry, the second transaction detects the duplicate.

> So at this point, the second request would return a 409 code (or something like that) to the client?

Yes. Stripe's approach is not fundamentally different; they just lookup the original request and return that response body instead of returning an error. It's more work for the server side engineers (and has a bunch of complex but obscure failure modes) but all the underlying database behavior is the same.

Re: Idempotency is easy until the second request is different

#204

Post by stickfigure »

whoamii wrote: post #172
stickfigure wrote: post #149

Earlier quoted context omitted.

It absolutely is idempotent behavior of the system. The goal is "make an idempotent payment", and this approach ("return an error for duplicates") was standard for financial APIs before Stripe. It still dominates for ecommerce APIs. It isn't complicated, though I can see how if your entire experience with financial APIs is Stripe, you might not be aware of how simple it is. Because Stripe's approach, while mildly mor…

You seem to believe idempotency is a fancy word for avoiding duplicates, which is why we seem to be having different conversations.

Idempotence is a fancy word for being able to repeat an operation without changing the outcome.

You seem to think that it's important to make the specific bytes of an http request-response idempotent.

I think that it's important to make a business operation idempotent.

You're missing the forest for the trees.

Re: Idempotency is easy until the second request is different

#205

Post by bcrosby95 »

kgwxd wrote: post #198
stickfigure wrote: post #89

This is all way too much. If you see a duplicate idempotency key, skip the replay and always return 409. This becomes a client problem. Clients already need to help enforce idempotent contracts; "check for conflict response" is not an onerous imposition. I've built multiple ecommerce APIs with this approach and they work great. No heroic measures required. You can often satisfy this contract with a unique constraint;…

> If you see a duplicate idempotency key I'm no expert but an "idempotency key" already has some major smell to it.

Why? I've used plenty of systems that have an idempotency key. E.g. many payment processors will take an order id and won't let you charge an order id more than once. That's just an idempotency key by another name.

Re: Idempotency is easy until the second request is different

#206

Post by fuzzythinker »

stickfigure wrote: post #89

This is all way too much. If you see a duplicate idempotency key, skip the replay and always return 409. This becomes a client problem. Clients already need to help enforce idempotent contracts; "check for conflict response" is not an onerous imposition. I've built multiple ecommerce APIs with this approach and they work great. No heroic measures required. You can often satisfy this contract with a unique constraint;…

Thank you, like and agree with the rest of the rules

Re: Idempotency is easy until the second request is different

#207

Post by ahoka »

stickfigure wrote: post #203
pdonis wrote: post #193

Earlier quoted context omitted.

> The second request starts a transaction and starts processing, everything looks good - no dups. So when the second request comes in, even though it has the same idempotency key as the first request, the server doesn't check to see if there's already a request received with that idempotency key? That would seem to defeat the whole purpose of idempotency keys. > On retry, the second transaction detects the duplicate.…

There are a lot of ways to implement this, so I posted an example with one of the most common ways - a database which uses optimistic concurrency in serializable isolation level. Postgres is often configured this way, though it's not the only way it can be configured. With optimistic concurrency models, collisions are only detected at commit time. Two transactions can simultaneously update the same data; each update…

"a database which uses optimistic concurrency in serializable isolation level. Postgres is often configured this way, though it's not the only way it can be configured."

It's not the default (read committed is) and I never saw serializable being set in actual production systems. You can do it, but then you have to be able to retry all of your transactions, including read.

What if the task you do take 5 minutes? 30 minutes? 10 hours? Do you create long transaction, blocking all reads?

Re: Idempotency is easy until the second request is different

#208

Post by themafia »

xlii wrote: post #48

Don't fix other people problems. If idempotent key was seen then send back response. Clients intention is outside the scope. If contract says "idempotency on key" the idempotent response on key. If contract says "idempotent on body hash" then response on body hash (which might or might not include extra data). APIs are contracts. Not the pinky promise of "I'll do my best guess"

In this space the "client" may be a hardware terminal with firmware that's difficult if not impossible to update.

Re: Idempotency is easy until the second request is different

#209

Post by xlii »

mrkeen wrote: post #63
xlii wrote: post #52

Earlier quoted context omitted.

That's why you need to separate work from actual input. It's not about trying again but about making sure you get consistent state. Imagine request for payment. You made one and timeouted. Why did it timeout? Your network or payment service error? You don't know, so you can't decide between retry and not retry. Thus practice is: make request - ack request with status request id (idempotent, same request gives same st…

> You don't know, so you can't decide between retry and not retry I'm well aware that the first order went through, even though the dumb system fumbled the translation of the success message and gave me a 500 back. I do retry because I wanted the outcome. I'm not giving it a new key (firstly because I'm a user clicking a form, not choosing UUIDs for my shopping cart) but more importantly, if I did supply a second key…

> I do retry because I wanted the outcome. I'm not giving it a new key (firstly because I'm a user clicking a form, not choosing UUIDs for my shopping cart) but more importantly, if I did supply a second key, it's now my fault for ordering two copies.

Upon initial request I have you "URPAY1". If you never check URPAY1 for status, we'll callback you and expect the result. If neither check nor callback succeeded rollback actions are ran (this is contractual agreement on partnership level).

You can verify your status with URPAY1. You need to provide your status check with check ID (URPAY1) and an unique request ID. You will receive a timestamped response. You won't get different responses for same CheckID + RequestID because it's a activity log and also procedure check (e.g. grossly simplifying success at 23:59:58 might be something different than success at 00:00:05 - these times can vary depending on partner, continent, so it's not only midnight etc.) If at any point you didn't get response you can retry and you will always get the same response.

Didn't get URPAY1 for the first time? No problem try againt second time. You'll get the same URPAY1. No new effects needed.

In this design you, as requester are in full power. You can make the same request 100 times which will cause only 1 effect. If networking is lost, something will crash you're still guaranteed to have effect AT MOST once.

In case you're curious for the full flows and handling edge cases Stripe has great documentation regarding how process looks like from merchant's customer's side (as this is their business and you can integrate with them).

Re: Idempotency is easy until the second request is different

#210

Post by mrkeen »

stickfigure wrote: post #202
pdonis wrote: post #156

Earlier quoted context omitted.

> If the original request isn't complete, what are you going to replay? Who says you have to replay? If you get a second request with the same idempotency key, and the original request is still in process, why not just send the client a response that says so?

It's not a terrible question. It would be complicated to implement and isn't more useful than using the database's consistency model, so nobody does it that way. Long running transactions create all sorts of problems, so transactions are generally expected to be short. The actual work behind "create payment" or "create order" is generally fairly trivial - more or less insert a row in a table. There's no good reason t…

> The actual work behind "create payment" or "create order" is generally fairly trivial - more or less insert a row in a table.

It's generally insert a row in someone else's table, over the wire, 50ms+ away. They might not even be using an RDBMS.

Post reply on HN