31–40 of 133 posts
Idempotency is a pretty critical concept in system design, and I think most developers have run into issues related to it even if they aren't directly familiar with the term. To give another simple example as the OP - Suppose you have a product that relies on time series data. For demo purposes you might create a curated data set to present to clients, but the presenter doesn't want to show data from 2019 as the "mos…
We had one "special" team member who insisted on everything being idempotent. This was his only leading principle. Result: absolute chaos - the code aspired to be idempotent, but due to idempotency he avoided thinking problems through and just created a mess of individual functions - each being idempotent, aside from the unavoidable bugs - which didn't form a coherent flow at all. We did a major refactoring, threw ou…
Idempotency is a pretty critical concept in system design, and I think most developers have run into issues related to it even if they aren't directly familiar with the term. To give another simple example as the OP - Suppose you have a product that relies on time series data. For demo purposes you might create a curated data set to present to clients, but the presenter doesn't want to show data from 2019 as the "mos…
The idempotency I’ve seen is usually an unnecessary extra complexity.
> 2. Charge each of these accounts a fee
> 3. Setup a cron job to run this every hour
Note that if this job ever runs successfully, but takes more than an hour, you will double-count. Can easily happen if the box running these crons is overloaded. One fix is to automatically halt the job after 55 minutes, another would be to have the middle step be impotent, for each user you're doing the process on, ensure (ideally in a threadsafe manner) that they need the operation to be done still.
If you’re making or using an API where repeating would be bad, consider using idempotency tokens for those too. I believe Stripe supports them. The basic idea is the same: if you pass a token into them, they will guarantee that in a certain time frame, no other requests with that ID can be duplicated. This is useful when the network flakes during the response. Is it safe to retry?
Things get trickier when you combine network and database consistency measures; that’s when you get into locks and multi stage commit and etc. and it helps to know your database’s consistency model, since it’s often not as solid as you think! (In the past, even PostgreSQL had issues with providing serializable isolation.)
I'm assuming a lot of people click on it to see what the word Idempotence means. From the article: "Idempotence is the property of a software that when run 1 or more times, it only has the effect of being run once." And the example is, instead of a chron job just running a process once a month or on some other schedule, it runs more frequently but checks if the change has already been made. (From the latin Idem which…
> And the example is, instead of a chron job just running a process once a month or on some other schedule, it runs more frequently but checks if the change has already been made. As a property, I think it's even nicer if a script can literally fully run twice and for the outcome to be the same if it only ran once (so skipping the 'did I run before?' check). Even though this check is useful in general, if you can def…
So the goal of your functional core is to fully construct the email, and return it to the caller, who then has the choice to send the email, print it, write it to disk, etc.
The program you deploy looks like this
EmailSender().send_email(construct_email(args))
You can test by implementing a "safe" EmailSender interface, so that you're executing the same code that's in prod.
In general, if a job/function is mutating state deep in the syntax tree (i.e. sending emails in the middle of a batch job), I personally see that as a violation of the Single Responsibility Principle.
We had one "special" team member who insisted on everything being idempotent. This was his only leading principle. Result: absolute chaos - the code aspired to be idempotent, but due to idempotency he avoided thinking problems through and just created a mess of individual functions - each being idempotent, aside from the unavoidable bugs - which didn't form a coherent flow at all. We did a major refactoring, threw ou…
A bit of pragmatism goes a long way, like Python's odd
x = (some_tuple,)
. . .syntax amidst its generally clean approach.
Inflexibility itself is the bugaboo.
I'm assuming a lot of people click on it to see what the word Idempotence means. From the article: "Idempotence is the property of a software that when run 1 or more times, it only has the effect of being run once." And the example is, instead of a chron job just running a process once a month or on some other schedule, it runs more frequently but checks if the change has already been made. (From the latin Idem which…
One of my first jobs was at an investment bank. They had a lot of programs that ran overnight, in a batch fashion. Everything had to be done before the markets opened the next morning. The term they used for idempotency was "free rerun." Being able to rerun any program with no special setup work was a high priority. The value in programs being a "free rerun" was that every so often the program would barf on a bad bit…
And a lot of "secret" scripts that automated a big part of our job.