Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Idempotency is easy until the second request is different

blog.dochia.dev

Post reply on HN
Print view View original

51–60 of 244 posts

Re: Idempotency is easy until the second request is different

#51

Post by asdfaoeu »

stavros wrote: post #2

Half of the mentioned issues are issues of atomicity, not idempotency. If I make a request, and the server crashes midway and doesn't send some crucial events, that's an issue whether or not I send a second request. From a cursory read, only the part up to "what if the second request comes while the first is running" is an idempotency problem, in which case all subsequent responses need to wait until the first one is…

Tbh the article seems to just be like "you can't solve idempotency with one idempotency-key header" and well like no shit.

Re: Idempotency is easy until the second request is different

#52

Post by xlii »

mrkeen wrote: post #13

The point of idempotency is safe retries. Systems are completely fallible, all the way down to the network cables. The user wants something + the system might fail = the user must be able to try again. If the system does not try again, but instead parrots the text of the previous failure, why bother? You didn't build reliability into the system, you built a deliberately stale cache.

That's why you need to separate work from actual input.

It's not about trying again but about making sure you get consistent state.

Imagine request for payment. You made one and timeouted. Why did it timeout? Your network or payment service error?

You don't know, so you can't decide between retry and not retry.

Thus practice is: make request - ack request with status request id (idempotent, same request gives same status id) - status checks might or might not be idempotent but they usually are - each request need to have unique id to validate if caller even tried to check (idenpotency requires state registration).

If you want to try again you give new key and that's it.

There might of course be bug in implementation (naive example: idempotency key is uint8) but proper implementation should scope keys so they don't clash. (Example implementation: idempotency keys are reusable after 48h).

If same calls result in different responses (doesn't matter if you saw it or not) then API isn't idempotent.

Re: Idempotency is easy until the second request is different

#53

Post by raffael_de »

mrkeen wrote: post #33
raffael_de wrote: post #23

Earlier quoted context omitted.

How and based on what is the idempotency key calculated which the clients sends with its request? In my double-purchase example above: when would the second purchase be requested with the same key or not?

1) Fresh UUID 2) Client's choice

now you are moving the core question to where the fresh UUID is calculated. a UUID is calculated or reused based on a process-defined decision.

Re: Idempotency is easy until the second request is different

#54

Post by mrkeen »

WilcoKruijer wrote: post #18

I really hate the POST verb for RESTish APIs because it cannot be idempotent without implementing an idempotency layer. Other verbs are naturally idempotent. Has anyone tried foregoing POST routes entirely? Theoretically you can let the client generate an ID and have it request a PUT route to create new entities. This would give you a tiny amount of extra complexity on the client, but make the server simpler as a tra…

In what sense is GET naturally idempotent?

The GET/POST split is the defence (even it's only advisory).

GET-only means every time you hit the back button during an order flow, you might double-order.

Re: Idempotency is easy until the second request is different

#56

Post by asdfaoeu »

syntex wrote: post #24

yes I always thought it's an easy thing. but I changed my mind recently when I had to deal with it. A lot little things you need to think of. For example. Client sends a request. The database is temporarily down. The server catches the exception and records the key status as FAILED. The client retries the request (as they should for a 500 error). The server sees the key exists with status FAILED and returns the error…

None of those are really unsolvable problems. I think though the issue it seems everyone in this thread is having is you can't wrap a non idempotent function to make it idempotent no matter how hard you try you have to design your system around it.

Re: Idempotency is easy until the second request is different

#57

Post by raffael_de »

harshitaneja wrote: post #46
raffael_de wrote: post #16

Idempotency means f(x) = f(f(x)).* Here x is interpreted as state and f an action acting on the state. State is in practice always subjected to side effects and concurrency. That's why if x is state then f can never be purely idempotent and the term has to be interpreted in a hand-wavy fashion which leads to confusions regarding attempts to handle that mismatch which again leads to rather meandering and confusing and…

Idempotence is a semantically overloaded term in computer science where in functional programming it refers to the same concept as mathematical idempotence it refers to any function leading to the same state in multiple calls as the first. And yes, in real machines we can't ever have true same states between multiple calls as system time, heat and other effects will differ but we define the state over the abstracted…

not just heat and system time. the context is about state handled by databases. database content can never be assumed to be identical between to identical operations involving it.

"delete record with id 123" is only idempotent if there is no chance that an operation like "create record with id 123" happened in between.

Re: Idempotency is easy until the second request is different

#58

Post by Hendrikto »

conradludgate wrote: post #49
Hendrikto wrote: post #41

Earlier quoted context omitted.

> That's why if x is state then f can never be purely idempotent That is simply not true. f could be, for example, “set x.variable to 7”, which is definitely idempotent.

There's no side effects in f here, so the statement does not apply

Parent said

> State is in practice always subjected to side effects and concurrency.

There was never any claim or assumption regarding f. Maybe the way you interpreted it is what they meant, but it is not what was stated.

Re: Idempotency is easy until the second request is different

#59

Post by javier2 »

i dont disagree with the problem, but this sort of Idempotency-Key header is kind of outsourcing the de-duplication to the client. If the client sends a different request with same Idempotency-Key header its the user's (client's) fault. Its also circumventing the fact that its the effect that should apply to give the same state, you could design the API itself to be idempotent wrt to some other property such as the transaction id. The designs I have seen using an explicit Idempotency-Key header has usually been added on after launch.
Post reply on HN