Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Idempotence now prevents pain later

ericlathrop.com

Post reply on HN
Print view View original

21–30 of 133 posts

Re: Idempotence now prevents pain later

#21

Post by corty »

pdpi wrote: post #12
sdenton4 wrote: post #6

Earlier quoted context omitted.

Mathematically, it's x^2 = x, which implies x^n = x for all positive integers n. Nilpotence (x^2 = 0) is also very helpful some times: it's a process which is self-reversing. Like the discrete Fourier transform (if you set up the constants properly).

X^2 is a weird way to describe it. A function f is idempotent if f(f(x)) = f(x).

It is quite common in some fields. Operator application is written without parentheses, and functions are a kind of operator. Therefore:

f(x) = f(f(x)) = f f x = f^2 x = f x

And leaving out the x, because it is just a placeholder anyways:

f f = f^2 = f

And of course this means that

f^n = f because f^n-1 f = f^n-1 by induction.

Re: Idempotence now prevents pain later

#22

Post by firebaze »

We had one "special" team member who insisted on everything being idempotent.

This was his only leading principle. Result: absolute chaos - the code aspired to be idempotent, but due to idempotency he avoided thinking problems through and just created a mess of individual functions - each being idempotent, aside from the unavoidable bugs - which didn't form a coherent flow at all.

We did a major refactoring, threw out about all that code, rewrote everything in a logical manner. Now everything is still idempotent, but comprehensible.

TLDR: idempotency is the same snakeoil as the majority of guiding principles: alone, it doesn't help at all. There are lots of other factors to consider, which make the developer/architect role demanding (and fun).

Craftmanship at least, a sense for architecture (better) or understanding the whole picture of the requirements as a team of developers (best) is still required.

Re: Idempotence now prevents pain later

#23

Post by jayd16 »

brsg wrote: post #20

Idempotency is a pretty critical concept in system design, and I think most developers have run into issues related to it even if they aren't directly familiar with the term. To give another simple example as the OP - Suppose you have a product that relies on time series data. For demo purposes you might create a curated data set to present to clients, but the presenter doesn't want to show data from 2019 as the "mos…

I don't think B is technically idempotent either. Change still occurs but with minimal difference. You cannot cache the results and use them again next week.

An idempotent change would be to pass in the current time instead of checking system time. In this case, as long as the input is the same, the result is the same. You could use cached results, but most likely you want to use new inputs.

Re: Idempotence now prevents pain later

#24

Post by bob1029 »

I feel like immutability and a 100% leak-proof layer of domain methods which in turn manage domain mutations would ultimately bring more value than explicitly adding idempotence throughout.

If I have an idempotent method like "CreateCustomerRecord", this can cause a lot of pain for audit features and other aspects of the domain model if it is internally making determinations about whether to actually create or silently skip creation. For me, I would much rather that the method throw an exception if there is a duplicate business key than have it silently complete without taking any actual action. Exceptions indicating attempts at invalid state transitions can be extremely valuable if you have the discipline to create & use them properly.

Generally, seeking idempotence in otherwise mutable methods is a band-aid for when you have broken immutability rules and allowed things to leak out of the sacred garden of unit-tested state machines and other provably-correct code items.

If you should only conditionally execute some method, perhaps the solution is to investigate the caller(s) of the method, rather than attempt to infer the intent of all possible callers within the method itself.

Re: Idempotence now prevents pain later

#25

Post by elcomet »

sdenton4 wrote: post #6
omarhaneef wrote: post #4

I'm assuming a lot of people click on it to see what the word Idempotence means. From the article: "Idempotence is the property of a software that when run 1 or more times, it only has the effect of being run once." And the example is, instead of a chron job just running a process once a month or on some other schedule, it runs more frequently but checks if the change has already been made. (From the latin Idem which…

Mathematically, it's x^2 = x, which implies x^n = x for all positive integers n. Nilpotence (x^2 = 0) is also very helpful some times: it's a process which is self-reversing. Like the discrete Fourier transform (if you set up the constants properly).

self-reversing is not nilpotence.

In mathematics, a self-reversing function is called an involution, and it's f^2 (or f(f) ) = Id, the identity function.

Nilpotence is very different. It says that if you apply your function a certain number of times, you end up with zero no matter what the input is. For example, projection on x axis + 90 ° rotation of a vector is nilpotent.

Re: Idempotence now prevents pain later

#26

Post by elcomet »

Animats wrote: post #15
smiley1437 wrote: post #7

I had only ever encountered idempotence in the context of system management (Ansible, Puppet, Chef, etc) Article made me think it's actually applicable to other management as well

It's a basic property of HTTP "GET". Or it's supposed to be. "GET" is not supposed to change server site state. That's what "POST" is for. This matters if there's a cache in the middle, since caches tend to assume that GET requests are idempotent and can be served from cache. Cloudflare assumes that. POST requests have to go through to the real server.

Get is idempotent because it's the identity function. It does not change the state of the data. So it's a trivial case of idempotency.

A more interesting function is PUT (idempotent) vs POST (not idempotent)

Re: Idempotence now prevents pain later

#27

Post by omarhaneef »

throwawayboise wrote: post #16
omarhaneef wrote: post #4

I'm assuming a lot of people click on it to see what the word Idempotence means. From the article: "Idempotence is the property of a software that when run 1 or more times, it only has the effect of being run once." And the example is, instead of a chron job just running a process once a month or on some other schedule, it runs more frequently but checks if the change has already been made. (From the latin Idem which…

One of my first jobs was at an investment bank. They had a lot of programs that ran overnight, in a batch fashion. Everything had to be done before the markets opened the next morning. The term they used for idempotency was "free rerun." Being able to rerun any program with no special setup work was a high priority. The value in programs being a "free rerun" was that every so often the program would barf on a bad bit…

I hope this example makes it evident that one of the primary innovations of the last 30 years is defaulting to Latin terms so that they are taken more seriously in business and technology circles to acquire ... you know... gravitas.

Re: Idempotence now prevents pain later

#28

Post by eweise »

bob1029 wrote: post #24

I feel like immutability and a 100% leak-proof layer of domain methods which in turn manage domain mutations would ultimately bring more value than explicitly adding idempotence throughout. If I have an idempotent method like "CreateCustomerRecord", this can cause a lot of pain for audit features and other aspects of the domain model if it is internally making determinations about whether to actually create or silent…

You have the caller pass in a unique ID to the CreateCustomerRecord. Don't create a new customer record if you receive a duplicate ID.

Re: Idempotence now prevents pain later

#29

Post by jayd16 »

bob1029 wrote: post #24

I feel like immutability and a 100% leak-proof layer of domain methods which in turn manage domain mutations would ultimately bring more value than explicitly adding idempotence throughout. If I have an idempotent method like "CreateCustomerRecord", this can cause a lot of pain for audit features and other aspects of the domain model if it is internally making determinations about whether to actually create or silent…

This can create false negatives when a request must be retried due to network failure but actually succeeded because the failure was during the response.

Idempotency is great for "debouncing" requests. If you want to tell difference between identical requests that are different transactions, add a unique transaction id of some kind.

Re: Idempotence now prevents pain later

#30

Post by longhairedhippy »

firebaze wrote: post #22

We had one "special" team member who insisted on everything being idempotent. This was his only leading principle. Result: absolute chaos - the code aspired to be idempotent, but due to idempotency he avoided thinking problems through and just created a mess of individual functions - each being idempotent, aside from the unavoidable bugs - which didn't form a coherent flow at all. We did a major refactoring, threw ou…

I don't see this as any reflection on idempotency as principle (or other principles in general). Building systems poorly, without a plan, and no testing, will result in a bug-riddled mess, regardless of what pattern is being used.
Post reply on HN