How to write idempotent Bash scripts • Hacker News Board

Re: How to write idempotent Bash scripts

#191

Post by dlgtho » Wed, Jul 10, 2019, 1:19 PM UTC

zamadatix wrote: post #24

"The -f flag removes the target destination before creating the symbolic link, hence it’ll always succeed." Removing something and adding it back seems to be, by definition, not idempotent. What if something tries to access that file in between? What about the filesystem timestamps (same issues as with the "touch" claim)?

True. In such case you need to create a different symlink within the same filesystem and then use "mv" to atomically overwrite the current symlink.

ln -sfn target new && mv -Tf new old

Re: How to write idempotent Bash scripts

#192

Post by michaelmcmillan » Wed, Jul 10, 2019, 3:56 PM UTC

dymk wrote: post #117

pmarreck wrote: post #70

Earlier quoted context omitted.

It's because rm -rf "$STEAMROOT/"* will evaluate as rm -rf "/"* if $STEAMROOT for some reason is blank or unset, which is what happened here. (As someone in that discussion mentioned, a little more Bash knowledge might have suggested using rm -rf "${STEAMROOT:?}/"* instead, to force it to (at least) error if it is empty or unset.)

So literally the only thing in common with the article is that they involve bash

I can't help you if you don't see the connection.

Re: How to write idempotent Bash scripts

#193

Post by slrz » Thu, Jul 11, 2019, 5:14 PM UTC

arendtio wrote: post #179

slrz wrote: post #149

Earlier quoted context omitted.

You can invoke mktemp(1) in ways that work across most contemporary systems. At least if you don't care that the resulting names are going to look a bit weird (have lots of XXXXXX in them due to name prefix vs. template differences among implementations).

Actually, I am not sure if that is true. I have no example at hand (I could look it up, if you are interested), but a few months ago I was trying hard to create a folder with mktemp using the same command for Linux, MacOS and Android, and as far as I remember, that wasn't as easy as it is supposed to be.

Shouldn't mktemp -d -t name.XXXXXX do it? Possibly in combination with setting TMPDIR if you want to create the directory somewhere else than within the default temp directory.

Works with GNU mktemp, the old BSD variant on macOS and also Busybox. Current Android uses Toybox, I think? Its mktemp implementation looks like it takes the same flags as Busybox mktemp.

https://landley.net/toybox/help.html#mktemp

Re: How to write idempotent Bash scripts

#194

Post by ajxs » Mon, Jul 15, 2019, 4:39 AM UTC

pas wrote: post #190

ajxs wrote: post #160

Earlier quoted context omitted.

Many of the comments in this thread seem to be sucked into a false dichotomy. There are better options than _either_ Rust or Bash for writing complicated shell scripts. Perl/Python are more suited to the task than either. The challenge of deciding what degree of complexity warrants the investment of more complicated language is a design decision left to the developer to figure out. Any thought given to designing more…

My experience is that going from bash to something without a compiler has an overhead that rarely pays off. mypy or TS can probably do the job well, but at that point Rust gives you a better deployment story.

Don't get me wrong, I'm absolutely an advocate for statically typed languages. Most of the work I'm doing with bash scripts involves command invocation, piping, and filesystem I/O. Which are things that I've never seen compiled, statically typed languages do particularly better than Perl/Python. I've never done this kind of programming in Rust, but my experience with Rust suggests to me that it's not any easier or more robust to do this work in Rust than it is in Java or C++. Mind you, you might have a completely different experience depending on the task at hand.

Re: How to write idempotent Bash scripts

#195

Post by arendtio » Tue, Jul 16, 2019, 5:27 PM UTC

slrz wrote: post #193

arendtio wrote: post #179

Earlier quoted context omitted.

Actually, I am not sure if that is true. I have no example at hand (I could look it up, if you are interested), but a few months ago I was trying hard to create a folder with mktemp using the same command for Linux, MacOS and Android, and as far as I remember, that wasn't as easy as it is supposed to be.

Shouldn't mktemp -d -t name.XXXXXX do it? Possibly in combination with setting TMPDIR if you want to create the directory somewhere else than within the default temp directory. Works with GNU mktemp, the old BSD variant on macOS and also Busybox. Current Android uses Toybox, I think? Its mktemp implementation looks like it takes the same flags as Busybox mktemp. https://landley.net/toybox/help.html#mktemp

Well, at least on my Android 7.1.2 (toybox --version 0.7.1-3125af0e06f4-android) mktemp doesn't recognize the -t option (https://github.com/landley/toybox/blob/0.7.1/toys/lsb/mktemp...). So it seems to be fixed in newer versions, but I am sure there will be some installations with the old versions around for a while.

Instead, I could use --tmpdir, but somehow that one seems to be buggy:

  $ mktemp -d --tmpdir name.XXXXXX                                                                                                                                                                                                                                   
  mktemp: Failed to create directory name.XXXXXX/name.XXXXXX/tmp.sL2WbI: No such file or directory

The macOS version, on the other hand, does work with those options, but it creates a file like name.XXXXXX.veCNnwkX (instead of name.veCNnwkX) so not a deal-breaker, but it is certainly not what you would have expected. And using --tmpdir with macOS doesn't work either.

So yes, in theory, it shouldn't be too hard but sadly, the reality is often buggy and outdated :-/