How to write idempotent Bash scripts
arslan.io
How to write idempotent Bash scripts
1–10 of 195 posts
Re: How to write idempotent Bash scripts
#2The problem with the idempotent trend in configuration management is that it’s all based on not tracking or knowing what the current state of something is. So reasoning about how these systems work is fundamentally impossible. It would be better to focus on systems where by we can always know the state and improve the tooling there.
Re: How to write idempotent Bash scripts
#3Why wouldn’t you use make? Side-effect tracking and cleanup is the job of a build tool.
Re: How to write idempotent Bash scripts
#4Many people who write scripts do not factor the environment they are run in and miss including traps to handle events like being terminated mid-way thru and cleaning up to a sanitized state. After all, creating a file, you can't presume their is enough free space, but people do in scripts all the time.
Then you have scripts that you want run once and if run again not have any more effect. FOr example a script to be run by ops on production servers, what happens if they accidently run it twice! An easy way to cover that is to have the script create a file, which you check at the start of the script and if present you exit. For this you can use the date/time, PID and script name for a unique file name, create in a /tmp directory that you cleanup weekly via sculker or whatever frequency you require. Handy way to handle scripts that you want run successfully once and never to be run again.
Re: How to write idempotent Bash scripts
#5The problem with the idempotent trend in configuration management is that it’s all based on not tracking or knowing what the current state of something is. So reasoning about how these systems work is fundamentally impossible. It would be better to focus on systems where by we can always know the state and improve the tooling there.
I don't see any conflict here. Idempotency is not a silver bullet and cannot fully replace state tracking, but it adds simplicity where appropriate.
Re: How to write idempotent Bash scripts
#6The problem with the idempotent trend in configuration management is that it’s all based on not tracking or knowing what the current state of something is. So reasoning about how these systems work is fundamentally impossible. It would be better to focus on systems where by we can always know the state and improve the tooling there.
On the contrary. With idempotence you know the state after an action. rm -f will always delete the file (if possible). ln -sfn will always work, even for a directory.
With the default behaviour of rm, ln -s, etc you know neither the state before, nor after.
Re: How to write idempotent Bash scripts
#7Why wouldn’t you use make? Side-effect tracking and cleanup is the job of a build tool.
Why would you use a build tool when you're not writing a build script?
Re: How to write idempotent Bash scripts
#8Re: How to write idempotent Bash scripts
#9Why wouldn’t you use make? Side-effect tracking and cleanup is the job of a build tool.
Why would you use a build tool when you're not writing a build script?
It's not necessarily a build tool - it's commonly used for that, but it's a useful DSL. rake in the Ruby world is used for all sorts of tasks for web apps and more, even though it was originally designed as the Ruby version of make. Look at npm - the NODE package manager, but it quickly grew to be used for more than server-side applications. A title is a description, not a constraint.
Re: How to write idempotent Bash scripts
#10The problem with the idempotent trend in configuration management is that it’s all based on not tracking or knowing what the current state of something is. So reasoning about how these systems work is fundamentally impossible. It would be better to focus on systems where by we can always know the state and improve the tooling there.
It's a pragmatic approach to this problem. Usually tracking state means that you write some version number or similar to a file, but than if for some reason that goes out of sync with the actual state, for example because you put it in /etc and someone accidentally restores an older version of it, hilarity ensues, because it's guaranteed the devs never expected this to happen and don't handle it in any way.
For the case of a setup script you'd need to update your state file after every step and in turn when running it again see what the file says and resume at whatever point it says. But then you risk running into above problem.
I try to write idempotent scripts whenever possible, combined with general sanity checks specific to whatever environment the script expects.