Live data from Hacker News

Shell script best practices, from a decade of scripting things

sharats.me

221–230 of 500 posts

Re: Shell script best practices, from a decade of scripting things

#221

This is not a best practices guide, please look forward to: https://mywiki.wooledge.org/BashGuide For example, using cd "$(dirname "$0")" to get the scripts location is not reliable, you could use a more sophisticated option such as: $(dirname $BASH_SOURCE)

Thank you to share. Can you provide an example where "$0" != "$BASH_SOURCE"?

Also, did you mean to write...?

    $(dirname "$BASH_SOURCE")"

Re: Shell script best practices, from a decade of scripting things

#223
post #207

I think BASH scripting is the opposite of riding a bike - you end up re-learning it almost every time you need to do it

What really put a stick in my spokes early on was not realising how whitespace acts differently to what I was used to. syntax error near unexpected token ? I was missing a space inside a [[ ]] - I started paying more close attention, this isn't javascript.

Use shellcheck as a linter for your scripts as that'll catch stuff like that.

Re: Shell script best practices, from a decade of scripting things

#224

I agree with basically all of this. A few more: The order of commandline args shouldn't matter. Env vars are better at passing key/value inputs than commandline arguments are. Process-substitution can often be used to avoid intermediate files, e.g. `diff some-file temp; diff some-file temp` If you're making intermediate files, make a temp dir and `cd` into that - Delete temp dirs using an exit trap (more reliable tha…

> The order of commandline args shouldn't matter.

Ugh I've seen so many (bash and non bash) cmdline tools that made it utterly annoying like that

The special place in hell goes to people who force users to write

    cmd help subcmd
instead of

    cmd subcmd --help
or ones that do not allow doing say

    cmd subcmd --verbose
because "verbose is global and doesn't belong to subcmd`

or ones where you need to write

    cmd --option1 subcmd --option2 subsubcomd --option3
and need to jump all over the cmdline if you want to add some option after previous invocation

and if you go "well but the option for command and subcommand might have same name" DONT NAME THEM THE SAME, that's just confusing people and search results.

Re: Shell script best practices, from a decade of scripting things

#225

Hands down, shell scripting is one of my all time favorite languages. It gets tons of hate, e.g. "If you have to write more than 10 lines, then use a real language," but I feel like those assertions are more socially-founded opinions than technically-backed arguments. My basic thesis is that Shell as a programming language---with it's dynamic scope, focus on line-oriented text, and pipelines---is simply a different p…

>However, if you instead organize all your data in a format that's sympathetic to line-oriented processing on stdin-stdout, then shell will work with you instead of against. Not even that is necessary. Just use structured data formats like json. If you are consuming some API that is not json but still structured, use `rq` to convert it to json. Then use `jq` to slice and dice through the data. dmenu + fzf + jq + curl…

new to 'rq', it's not in active development, any other alternatives? it seems doing a lot other than convert structured data to json.

Re: Shell script best practices, from a decade of scripting things

#227

Hands down, shell scripting is one of my all time favorite languages. It gets tons of hate, e.g. "If you have to write more than 10 lines, then use a real language," but I feel like those assertions are more socially-founded opinions than technically-backed arguments. My basic thesis is that Shell as a programming language---with it's dynamic scope, focus on line-oriented text, and pipelines---is simply a different p…

Shell and SQL make you 10x productive over any alternative. Nothing even comes close. I've seen people scrambling for 1 hours to write some data munging, then spend another 1 hour to run it through a thread pool to utilize those cores , while somebody comfortable is shell writes a parallelized one liner, rips through GBs of data, and delivers the answer in 15 minutes. What Python is to Java, Shell is to Python. It sp…

> getting the shell quoting hell right

Shameless plug coming, it this has been a pain point for me too. I found the issue with quotes (in most languages, but particularly in Bash et al) is that the same character is used to close the quote as is used to open it.m. So in my own shell I added support to use parentheses as quotes in addition to the single and double quotation ASCII symbols. This then allows you to nest quotation marks.

https://murex.rocks/docs/parser/brace-quote.html

You also don’t need to worry about quoting variables as variables are expanded to an argv[] item rather than expanded out to a command line and then any spaces converted into new argv[]s (or in layman’s terms, variables behave like you’d expect variables to behave).

https://github.com/lmorg/murex

Re: Shell script best practices, from a decade of scripting things

#228
post #128

More opinions 1. Bash shouldn't be used, not because of portability, but because its features aren't worth their weight and can be picked up by another command, I recommend (dash) any POSIX complaint shell (bash --posix included) so you aren't tempted to use features of bash and zsh that are pointless, tricky or are there for interactivity. Current POSIX does quite well for what you would use shell for. 2. Never use…

> 1. Bash shouldn't be used, not because of portability, but because its features aren't worth their weight and can be picked up by another command,

I agree but

> I recommend (dash) any POSIX complaint shell (bash --posix included) so you aren't tempted to use features of bash and zsh that are pointless, tricky or are there for interactivity. Current POSIX does quite well for what you would use shell for.

That's just terrible recommendation. It's saying "well, bash is terrible, use a terrible one that also have less functions"

Re: Shell script best practices, from a decade of scripting things

#229
I'm surprised that no one mentioned a pair of tiny functions to log each command before it is run. Nicer versions also print a timestamp. Of course, this setup assumes: set -e

    echo_command()
    {
        echo
        echo '$' "$@"
    }

    echo_and_run_command()
    {
        echo_cmd "$@"
        "$@"
    }
Then something like:

    main()
    {
        # For simple commands that do not use |  etc.
        echo_and_run_command cp --verbose ...

        # More complex commands
        echo_command grep ... '|' find ...
        grep ... | find ...
    }

    main "$@"

Re: Shell script best practices, from a decade of scripting things

#230

Hands down, shell scripting is one of my all time favorite languages. It gets tons of hate, e.g. "If you have to write more than 10 lines, then use a real language," but I feel like those assertions are more socially-founded opinions than technically-backed arguments. My basic thesis is that Shell as a programming language---with it's dynamic scope, focus on line-oriented text, and pipelines---is simply a different p…

Eh, as soon as you have to deal with arrays and hash tables/dicts or something like JSON, bash becomes very painful and hard to read.
Post reply on HN