Great article, but a common scenario is missing:
I want to run a long running script, but not if another copy of it is already running.
How to write idempotent Bash scripts
41–50 of 195 posts
Re: How to write idempotent Bash scripts
#42Let's try to avoid using a hammer on anything that resembles a nail: https://github.com/valvesoftware/steam-for-linux/issues/3671
How is that related to the article aside from "both involve bash"?
Re: How to write idempotent Bash scripts
#43Great article, but a common scenario is missing: I want to run a long running script, but not if another copy of it is already running.
https://linux.die.net/man/1/flock would probably help
Re: How to write idempotent Bash scripts
#44Earlier quoted context omitted.
Chef isn't magic, it just uses a lot of cookbooks which are just running things to check what the state is. Usually those cookbooks will want to have things in a particular way, which might not be the way you want. Chef/Ansible/Puppet all have the problem of having many layers of overhead for the same thing. This makes them slow, hard to debug, hard to read and even write (in my opinion). Sure, bash is hell, but the…
Writing a shell script in rust would be obnoxiously difficult for no real benefit.
Yeah, and tools like Chef are 99% blocked on disk operations anyways, so I have no idea why GP thinks Rust would help
Re: How to write idempotent Bash scripts
#45Great article. Very helpful.
Re: How to write idempotent Bash scripts
#46It’s good to know which command line tools are idempotent, but I think once you start running automated scripts repeatedly you’re better off with something like Chef where you declaratively define the end state and let chef figure out how to converge upon that state. Taking the “create a directory” example, you would say “I want to end up in a state where this directory is created and has these permissions” and chef…
Chef isn't magic, it just uses a lot of cookbooks which are just running things to check what the state is. Usually those cookbooks will want to have things in a particular way, which might not be the way you want. Chef/Ansible/Puppet all have the problem of having many layers of overhead for the same thing. This makes them slow, hard to debug, hard to read and even write (in my opinion). Sure, bash is hell, but the…
Rust doesn't get you anything resembling what you want here.
Chef/Ansible/Salt could be described as rudimentary type systems for operating system states, that enable you to convert between different types/states.
It's conceivable you could build such a system in Rust, but Rust itself has no primitives that would make it easy. It certainly isn't a good starting point when you want to set up a database server.
Re: How to write idempotent Bash scripts
#47This is generally terrible advice. A better option is to 'set -e' and ensure that the bash script exits when there's a failure. Bash scripts can't be idempotent because they operate in an external environment that can't ever be. The better option is to just be extra safe. See: > http://redsymbol.net/articles/unofficial-bash-strict-mode/
That's orthogonal to the article - in fact, the article hints that you'd take this advice when your bash script is failing midway, such as it might with `set -e`. The article is about writing bash scripts that don't care if you've ran them once, or 10 times.
Precisely.
Re: How to write idempotent Bash scripts
#48Earlier quoted context omitted.
Writing a shell script in rust would be obnoxiously difficult for no real benefit.
Yeah, and tools like Chef are 99% blocked on disk operations anyways, so I have no idea why GP thinks Rust would help
Rust would help due to the type system and lack of library dependencies (it does static linking AFAIK), I suppose.
Re: How to write idempotent Bash scripts
#49A good way to remember idempotent is to think of the 'delete' or 'mark as read' function for gmail.
You can delete an email or mark it as read even if it's been done before (on another open screen as an example).
Re: How to write idempotent Bash scripts
#50It’s good to know which command line tools are idempotent, but I think once you start running automated scripts repeatedly you’re better off with something like Chef where you declaratively define the end state and let chef figure out how to converge upon that state. Taking the “create a directory” example, you would say “I want to end up in a state where this directory is created and has these permissions” and chef…
Chef isn't magic, it just uses a lot of cookbooks which are just running things to check what the state is. Usually those cookbooks will want to have things in a particular way, which might not be the way you want. Chef/Ansible/Puppet all have the problem of having many layers of overhead for the same thing. This makes them slow, hard to debug, hard to read and even write (in my opinion). Sure, bash is hell, but the…
Yeah, I find out-of-the-box cookbooks about as helpful as out-of-the-box docker images: useful as a starting place but to leverage chef you really need to write your own. Chef resources/providers are nicely composable and the primitives are pretty easy to rewrite if you want, my main point is that chef gets you to think in terms of convergence instead of brittle imperative changes.
You could rewrite your bash scripts in rust but while it would probably be safer/more correct than bash, the whole paradigm of writing imperative scripts to manipulate existing machines is flawed, whichever language you write it in. And executor performance is almost never the bottleneck in my experience.