Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Writing Safe Shell Scripts (2019)

sipb.mit.edu

Post reply on HN
Print view View original

71–80 of 166 posts

Re: Writing Safe Shell Scripts (2019)

#71

Post by chrisfinazzo »

Pxtl wrote: post #58
heavyset_go wrote: post #30

Earlier quoted context omitted.

I've yet to find a programming language that makes I/O redirection, piping, and process substitution[1] as easy as Bash does. Process substitution is where the shell really shines, in my opinion. Bash, and Bash-like shells, are literally everywhere. I have to be wary about what Python 3 features I use, and if there will even be a Python interpreter available. My OpenWRT router has a Bash shell, but I don't care to in…

PowerShell, mostly because PowerShell was designed as a mashup of Bash and C#.... And it's kind of a trainwreck in a lot of ways. It really feels like piping and easy process invocation and compile-time directory awareness wouldn't be massively onerous to add to an existing full-featured programming language so you wouldn't have to sacrifice a good type system and powerful syntax when you want to do scripty things.

As I understand it, PowerShell came about because porting the standard Linux/Unix utilities to Windows didn't work out.

PowerShell meets both requirements as it's tuned for the Windows environment, but can still play nicely with the rest of the world.

Of course, now that WSL is a thing, it's a real question why someone would want to use PS unless they had no other options.

https://www.heavybit.com/library/podcasts/to-be-continuous/e...

Re: Writing Safe Shell Scripts (2019)

#72

Post by mfontani »

kohtatsu wrote: post #32
kohtatsu wrote: post #29

Earlier quoted context omitted.

Cool repo, but the example is lame because a shell, or a git alias would do just fine. A shell script that consists of a one-line exec isn't what people concerned about. FWIW I'm a 99.9% POSIX masochist (probably closer to 4N)

Looks like a lot of your scripts couldbe aliases, I store mine in ~/.aliases, and slightly more complicated things (e.g. take arguments) in ~/.functions, and source both those files from bashrc. I have maybe 3 standalone shell scripts in my PATH, despite writing thousands of lines of shell.

I've moved from aliases to scripts as I can't seem to have found the (if any exists...) way to ensure aliases can be used from within vim's ":!..." or ":'!...", and as I often find myself wanting to either execute something while I'm amending code, or wanting to filter text with a purpose-built command, I end up writing (sometimes very short!) shell scripts.

Re: Writing Safe Shell Scripts (2019)

#74

Post by acqq »

jonahx wrote: post #59
chubot wrote: post #52

Earlier quoted context omitted.

FWIW I think ShellCheck is great and the state of the art, but Oil is partly (negatively) inspired by ShellCheck :) Somebody integrated ShellCheck into Google's code review system about four years ago, right before I left. So the result was that every code review I sent with a shell script was filled with red squigglies -- "add double quotes here". Most code reviewers don't really know shell, but if they see red squi…

> The double quotes are of course technically correct, but when you're writing a shell script in a subdir 8 levels deep in the repo that only operates on 3 files in that subdir, it's overkill and makes everything ugly. Once it becomes habit the cost is zero, though. And the alternative you're implicitly suggesting is: "Follow shellcheck except you are allowed to make an expert judgement and ignore it when you know it…

Exactly. If one "expert" is developing only for himself he can allow himself not to bother writing a script "clearly" (i.e. failing the programs that perform source checks).

But as soon as the script is something that should be potentially sooner or later maintained by anybody else than the original author (and in any successful project that's practically sure) the only right policy is to produce the source code that's obvious and passes the source checks.

An "expert" would have no problem to in no time adjust his three-line code to fulfill all the expectations of ShellCheck.

If it takes more than that, that only proves that the checks are actually beneficial and even more important than the "expert" tells to himself: it proves that writing that specific code "cleanly" is actually hard for everybody.

Re: Writing Safe Shell Scripts (2019)

#75

Post by arendtio »

Some time ago I started to doubt if using 'set -e' is a good idea.

I mean, if it would work as you expect it to, it certainly is a good idea to exit a script as soon as something fails. But sadly not all implementations behave similarly [1] and if you call a function from within a condition, 'set -e' gets deactivated/doesn't work.

For illustration, take a look at the following example:

  #!/bin/bash
  
  foo() {
    set -e
    false
    echo "Sucks"
  }
  
  printf 'Normal: %s\n' "$(foo)"
  printf 'Condition: %s\n'  "$(foo || true)"
Output:

  Normal: 
  Condition: Sucks
Probably not what you would have expected. Not using 'set -e' is no good solution either, so, for the time being, I still use it, but I am still looking for a better solution.

[1] https://www.in-ulm.de/~mascheck/various/set-e/

Re: Writing Safe Shell Scripts (2019)

#76

Post by j1elo »

lalaland1125 wrote: post #15

I really wonder whether there is really any point in writing shell scripts anymore. Practically every Unix/Linux box in existence has at least some version of Python 2 that can be used as a complete and total replacement. I can't think of a single situation where I would need a shell script and a Python script wouldn't be much cleaner, simpler, and more maintainable.

> Practically every Unix/Linux box in existence has at least some version of Python

If we understand "box" as "environment", in general, and not literally a physical or virtual machine...

Not really. E.g. see: every clean Ubuntu Docker container. They don't have Python, nor should have to include it by default.

And if you are going to do something like this:

  apt-get update && apt-get install -y stuff
  wget https://file.tgz
  tar xf file.tgz
  cd file
  make
  make install
  value="$(cat result | grep something)"
  mvn clean install -Dapp.value="$value"
Yeah, this would need proper securing and Shellcheck, but still I'd rather have it written like this, a concise and to the point script, instead of having to install a whole secondary language interpreter and writing a script among 3 to 10 times longer for the same result.

I recently did the opposite of what everybody seems to be doing: moved an 800-line Python script, which you had to read carefully to understand what it was doing, to a 150-line Bash script that you only have to look at, because the important things to know is to see what other, external programs are being called. Just the apt-get stuff in Python was a mess with the poorly documented Python API for Apt, and some 50 lines to just do the same of an apt-get install... effectively wasted mental cycles.

(EDIT: formatting)

Re: Writing Safe Shell Scripts (2019)

#78

Post by [deleted] »

arendtio wrote: post #75

Some time ago I started to doubt if using 'set -e' is a good idea. I mean, if it would work as you expect it to, it certainly is a good idea to exit a script as soon as something fails. But sadly not all implementations behave similarly [1] and if you call a function from within a condition, 'set -e' gets deactivated/doesn't work. For illustration, take a look at the following example: #!/bin/bash foo() { set -e fals…

[deleted]

Re: Writing Safe Shell Scripts (2019)

#79

Post by chubot »

jonahx wrote: post #59
chubot wrote: post #52

Earlier quoted context omitted.

FWIW I think ShellCheck is great and the state of the art, but Oil is partly (negatively) inspired by ShellCheck :) Somebody integrated ShellCheck into Google's code review system about four years ago, right before I left. So the result was that every code review I sent with a shell script was filled with red squigglies -- "add double quotes here". Most code reviewers don't really know shell, but if they see red squi…

> The double quotes are of course technically correct, but when you're writing a shell script in a subdir 8 levels deep in the repo that only operates on 3 files in that subdir, it's overkill and makes everything ugly. Once it becomes habit the cost is zero, though. And the alternative you're implicitly suggesting is: "Follow shellcheck except you are allowed to make an expert judgement and ignore it when you know it…

Yeah I don't disagree with that. If you're working in a big group with some shell scripts, and want to get stuff done, use ShellCheck.

I just find it annoying, hence the new shell :)

There's a lot more to Oil, but that's definitely one of the surface annoyances I want to fix. Although zsh actually does fix that, and for some reason I've almost never seen a zsh script.

Post reply on HN