Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Safe ways to do things in bash

github.com

Post reply on HN
Print view View original

31–40 of 255 posts

Re: Safe ways to do things in bash

#31

Post by _kst_ »

Twisol wrote: post #17
jwilk wrote: post #5

> POSIX mandates /bin/sh Nope. On the contrary, it says: Applications should note that the standard PATH to the shell cannot be assumed to be either /bin/sh or /usr/bin/sh Source: http://pubs.opengroup.org/onlinepubs/009695399/utilities/sh.... A pedantically-compliant shell script should not have shebang at all.

What's wrong with `#!/usr/bin/env sh`?

I've worked on systems where env was installed as /bin/env and not as /usr/bin/env . (I think it was SunOS 4.)

For that matter, under Termux on Android it's /data/data/com.termux/files/usr/bin/env (but termux has a hack to make normal shebangs work).

Re: Safe ways to do things in bash

#32

Post by ta4354546444 »

Bash strikes me as a bit of a mess, as in people threw the kitchen sink into it for 'portability'.

Things like being able to open a socket e.g. using the /dev/tcp// stuff give me the willies a bit.

I lean towards Ruby if it's going to be anything longer than a few lines, or requires anything but the simplest of logic/commands. Otherwise I was always told /bin/sh is likely to be the most portable, so tend to use that in the absence of any other good reason.

Re: Safe ways to do things in bash

#33

Post by jwilk »

_kst_ wrote: post #31
Twisol wrote: post #17

Earlier quoted context omitted.

What's wrong with `#!/usr/bin/env sh`?

I've worked on systems where env was installed as /bin/env and not as /usr/bin/env . (I think it was SunOS 4.) For that matter, under Termux on Android it's /data/data/com.termux/files/usr/bin/env (but termux has a hack to make normal shebangs work).

https://www.in-ulm.de/~mascheck/various/shebang/#env mentions other OSes that have only /bin/env, although admittedly they are all quite old.

Re: Safe ways to do things in bash

#35

Post by jackewiehose »

jwilk wrote: post #5

> POSIX mandates /bin/sh Nope. On the contrary, it says: Applications should note that the standard PATH to the shell cannot be assumed to be either /bin/sh or /usr/bin/sh Source: http://pubs.opengroup.org/onlinepubs/009695399/utilities/sh.... A pedantically-compliant shell script should not have shebang at all.

> it says: Applications should note that the standard PATH to the shell cannot be assumed to be either /bin/sh or /usr/bin/sh

It also recommends a script: "Installation time script to install correct POSIX shell pathname".

But I wonder how they execute this script. So this is all crap. They should remove that and instead put in something like "you have to test your script on the platform where you deploy.".

The article is missing the one big problem I encountered at most: bin/sh instead of bin/bash or the wrong version of sh|bash.

Re: Safe ways to do things in bash

#36

Post by thisacctforreal »

nisa wrote: post #12
mixedCase wrote: post #10

Earlier quoted context omitted.

Sometimes you just want some 30-50 lines of piping a few commands and a couple of conditionals. Python (the language with the most community traction to replace bash for scripts) is a royal pain to use for this without libraries that are present in no default system, and even with those it often ends up being more verbose than it should.

> Sometimes you just want some 30-50 lines of piping a few commands and a couple of conditionals. maybe I was not clear - nothing against shell-scripting but doing some weird dancing like in the article is imho useless, also depending on bash is a stupid idea imho. posix sh + shellcheck is all you need. if you can't solve your problem in posix sh rethink your code / approach and simplify until it will work.

I agree.

I've used this page successfully as reference for portable syntax:

http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3...

Some sections on features I use heavily:

- Parameter Expansion (specifically :-, %, %%, #, ##)

- Special Parameters (specifically "$@", $#, $?)

- set --, this lets you set the $1, $2, etc. variables. I use this with "$@" for arrays, primarily for building command strings.

Here's an small shell script demonstrating some of them:

    #!/bin/sh
    
    # err function
    err() { echo "$1" >&2; exit 1; }
    
    # init variables
    unset src
    unset dst
    dry_run=false
    
    # get arguments
    while [ $# -gt 0 ]; do
        case "$1" in
            -d|--dry-run) dry_run=true ;;
            --) shift; break ;;
            -*) err "unknown option: $1" ;;
            *)
                if [ -z "$src" ]; then src="$1"
                elif [ -z "$dst" ]; then dst="$1"
                else err "unexpected argument: $1"
                fi
            ;;
        esac
        shift
    done
    
    # sanity checks
    ## TODO: print usage
    if [ -z "$src" ]; then err "source not specified"; fi
    if [ ! -d "$src" ]; then err "source does not exist"; fi
    if [ ! -r "$src" ]; then err "cannot read source directory"; fi
    
    if [ -z "$dst" ]; then err "destination not specified"; fi
    
    if ! rsync --version >/dev/null 2>&1; then
        err "missing rsync(1)"
    fi
    
    # build rsync command
    set -- rsync -aq "$src" "$dst"
    
    # log command
    echo "copying $src to $dst"
    echo "    $@"
    if ! $dry_run; then
        if "$@"; then
            echo "success"
        else
            # rsync will have printed an error message
            err "rsync exited with error code $?"
        fi
    else
        echo "dry run; not executing"
    fi
Also be sure to read the man page for test (the [ command).

Re: Safe ways to do things in bash

#38

Post by dozzie »

mehrdadn wrote: post #19

A little heads-up: "$var" does what you think, but "$(cmd)" likely does not do what you think: - The former just gives you a string whose contents are identical to that of var. - The latter would do similar for the output of cmd, except that it strips away the trailing newline . This is often not an issue, but can be crucially important in some cases, and can catch you off-guard. The point I'm making here is that it'…

> The point I'm making here is that it's actually quite difficult to get a string that literally has the contents you want.

It's not difficult, it's just tedious.

  foo=$(whatever).
  foo=${foo%.}

Re: Safe ways to do things in bash

#39

Post by ljm »

> Gotcha: Errexit is ignored depending on caller context

It proves the point that it's a gotcha but those examples seemed sensible to me. As far as I understand it, `set -e` doesn't turn every unchecked, non-zero exit code into an exception because there's no way of knowing whether the function or sub shell you're invoking was written by you or pulled in elsewhere, and as a result you don't know if a non-zero exit code is a legitimate, show-stopping failure.

Those functions and subshells might as well be mini inline executables and in that context it makes sense to only check the final output. If that's horribly wrong and confusing, maybe there should be a move to make `set -e` the default so all error handling is explicit, but you've got other languages for that that don't involve throwing `|| true` onto the end of every unimportant command you run.

I also realise that this doesn't make a case for Bash being intuitive. Precisely the opposite. But I suppose you have that with a shell where it's more important to be adaptive to the person behind the keyboard at the expense of the purity of the implementation. Especially considering the history of it all.

Re: Safe ways to do things in bash

#40

Post by mehrdadn »

dozzie wrote: post #38
mehrdadn wrote: post #19

A little heads-up: "$var" does what you think, but "$(cmd)" likely does not do what you think: - The former just gives you a string whose contents are identical to that of var. - The latter would do similar for the output of cmd, except that it strips away the trailing newline . This is often not an issue, but can be crucially important in some cases, and can catch you off-guard. The point I'm making here is that it'…

> The point I'm making here is that it's actually quite difficult to get a string that literally has the contents you want. It's not difficult, it's just tedious. foo=$(whatever). foo=${foo%.}

That's not general POSIX, right? I seem to recall it's Bash-specific? (P.S. I think you forgot quotes?)

The other problem (which I guess I accidentally brushed under the rug when I singled out "variables") is that having to do this actually means you need to put it in a variable. If you're nesting subshells, this gets pretty darn tedious, easy to forget about, and difficult to read pretty quickly... it seems you wouldn't consider that "difficulty" and think of it as "just tediousness", but I think if something is too easy to do incorrectly and too tedious to get right, that's also a kind of added difficulty.

Post reply on HN