11–20 of 69 posts
Come here to learn more details and maybe find answers to my questions. Unfortunately, the article is very modest explaining how to work with idempotency. Missing key answers like what about two requests in flight, how to store and query requests status reliably. The article mention requests should be ACID, but the diagram has caching storage. Would redis/dynamodb work here?
I know it's just an introduction so perhaps it shouldn't be taken too literally, but the sequence diagram of how to make an order idempotent feels like instructions on how to create race conditions. You can't check for existence to guard against duplicate creation without either locking or another approach to making the Exists-or-Create step atomic. If you were to put in the sequence as described, you'd have somethin…
Presumably unique constraints could also solve for this, in a properly written transaction.
If your API is for example: "POST /orders/create {products:[{bananas:1}], shippingAddress:{},...}", then it's not clear how you would check uniqueness, because you'd want the same person to make the same order at different times by design.
To prevent double-submit, and make it idempotent, you need to add extra information.
One way to address this is generating transaction/sequence/correlation IDs at the very start of the process. So for example when you first request the order form, a transaction / correlation ID (called in the article an Idempotency key) is generated.
Then you would include that transaction ID in the order, and could prevent duplicate use of an order ID at multiple levels of the system including the database.
One design is to have a single-source-of-truth, and have at the (internal) API layer a state machine for a transaction. Once an order has been processed then it's impossible to make another order because there's no "make order" step from the state that an order is in having been processed. Again you need some kind of correlation ID to track which state machine needs loading.
That however is less scalable than other solutions including things like event-sourced architecture, which would also prevent duplicate orders through eventual consistency although there's less (or no) guarantee in many such systems that the order submitted "first" would win the race and not be superseded by the second. Such behaviour is fine if documented.
One of the things I think I did right in my career is working on payment and billing systems early on. It does teach you how to build at least semi-reliable software because people tend to get really mad when you screw with their money. It also is a good introduction to regulation and compliance. Idempotency was is one of the concepts you get taught on day one in that field (as kind of demonstrated by that being the…
I know it's just an introduction so perhaps it shouldn't be taken too literally, but the sequence diagram of how to make an order idempotent feels like instructions on how to create race conditions. You can't check for existence to guard against duplicate creation without either locking or another approach to making the Exists-or-Create step atomic. If you were to put in the sequence as described, you'd have somethin…
As I understand it, it's not checking for the existence of the resource but for the existence of the request. Assuming the requests are unique to a user/client and the user has a sequential conversation with the API server, the server would process the received requests in order and would reject the duplicate. However, if the requests of the same key (duplicates) are sent to any of several API servers, you have a pro…
Uhh thanks chatgpt.
pet peeve: idempotency is badly named. idempotent literally translated is ‘same power.’ two or more things having the same power doesn’t necessarily mean they produce same, single result. if that were the case we wouldn’t need the special handling for idempotent instructions/commands/actions to ensure duplicates are not created.
Reading this article is completely safe even if you already understand idempotency.
Idempotency is one mathematical concept that's pretty useful for software engineers to understand. Heck, might be more useful than BigO. - Useful to know when you work with APIs (as the article outlines). - Very useful when working with background jobs. You're not gonna have a good time if those aren't idempotent. - Good to know for interviews. I was asked to explain idempotency a handful of times, weird as that is.