Live data from Hacker News

Writing Safe Shell Scripts (2019)

sipb.mit.edu

41–50 of 166 posts

Re: Writing Safe Shell Scripts (2019)

#41

I really wonder whether there is really any point in writing shell scripts anymore. Practically every Unix/Linux box in existence has at least some version of Python 2 that can be used as a complete and total replacement. I can't think of a single situation where I would need a shell script and a Python script wouldn't be much cleaner, simpler, and more maintainable.

If it's a small thing (100 lines or less) I use shell. Anything bigger goes to Perl.

Re: Writing Safe Shell Scripts (2019)

#42

I really wonder whether there is really any point in writing shell scripts anymore. Practically every Unix/Linux box in existence has at least some version of Python 2 that can be used as a complete and total replacement. I can't think of a single situation where I would need a shell script and a Python script wouldn't be much cleaner, simpler, and more maintainable.

You don't have to pick between the two. Shell scripts are great and give you the option of using battle tested commands that have stood the test of time.

Re: Writing Safe Shell Scripts (2019)

#43
Instead of

    set -euf -o pipefail
please consider

    set -o errexit
    set -o nounset
    set -o noglob
    set -o pipefail
which is easier to lookup/search for if the reader is less familiar with shell scripting, and makes for cleaner diffs when a flag is removed/added.

Caveat: some of the longer forms might be Bashisms (e.g. not present in ksh, dash, etc.)

Re: Writing Safe Shell Scripts (2019)

#44
post #14

Earlier quoted context omitted.

With the "--" and assuming GNU rm, rm -f -- /path/foo* is safe. If you use a glob pattern that does not match itself, then you will want nullglob: rm foo.[cC] will remove a file named literally foo.[Cc] if neither foo.c nor foo.C exist, despite the fact that foo.[Cc] does not match the glob. nullglob is probably a good default for bashscripts, but I don't think it exists in posix.

Wouldn't the glob still have issues with files that contain spaces? My defensiveness generally takes a sharp upturn when I write scripts that will be run as root or ones that delete data, but I'll admit not trusting mktemp's exit code is probably taking it a bit too far.

globs have no issues with spaces.

Re: Writing Safe Shell Scripts (2019)

#45
post #22

I really wonder whether there is really any point in writing shell scripts anymore. Practically every Unix/Linux box in existence has at least some version of Python 2 that can be used as a complete and total replacement. I can't think of a single situation where I would need a shell script and a Python script wouldn't be much cleaner, simpler, and more maintainable.

Shell scripts are readable by just about anyone, they're available on every UNIX system, not just the Red Hat/Debian-derivatives of the last twenty years, they're fast as long as you're not doing stupid things, they're easily maintainable, they don't handle dependencies terribly (unlike Python), and so forth. There's a reason AT&T used to run ads that showed their secretaries, managers, and so on using and writing sh…

Is this because both languages are from different eras? When did you last see a consumer ad about scripting?

Re: Writing Safe Shell Scripts (2019)

#46

Earlier quoted context omitted.

Cool repo, but the example is lame because a shell, or a git alias would do just fine. A shell script that consists of a one-line exec isn't what people concerned about. FWIW I'm a 99.9% POSIX masochist (probably closer to 4N)

Looks like a lot of your scripts couldbe aliases, I store mine in ~/.aliases, and slightly more complicated things (e.g. take arguments) in ~/.functions, and source both those files from bashrc. I have maybe 3 standalone shell scripts in my PATH, despite writing thousands of lines of shell.

I had a mess of shell functions. It is nice that they run more quickly, but I don't have any shell commands that need to run that often.

What moved me to putting them in a bin directory was that it's just easier to edit a standalone script.

Re: Writing Safe Shell Scripts (2019)

#47
post #43

Instead of set -euf -o pipefail please consider set -o errexit set -o nounset set -o noglob set -o pipefail which is easier to lookup/search for if the reader is less familiar with shell scripting, and makes for cleaner diffs when a flag is removed/added. Caveat: some of the longer forms might be Bashisms (e.g. not present in ksh, dash, etc.)

"which is easier to lookup/search for"

From my experiment:

Googling "set euf" (without quotes) -- 838,000 results, first 6 results are all helpful and explain it in detail.

Googling "set o nounset" (without quotes) -- 38,700 results, entire first page seems helpful.

I dunno, looks like they're identically easy to learn the meaning of.

Re: Writing Safe Shell Scripts (2019)

#48
post #8
post #5

Can mktemp fail? I recently wrote a script that ensures that the directory created by mktemp actually exists and starts with /tmp so that when the script wipes out its temporary data at the end, it will not ever run something like "rm -rf /". Using fully-qualified paths for everything is also (not) fun. In another script I was leery about running rm -f -- /path/foo* so I instead used find /path -mindepth 1 -maxdepth…

> Can mktemp fail? Yes it can. You should use its exit status (or set -e). See http://man.openbsd.org/mktemp Note that there is a nasty trap if you're using local to declare and set a variable: a () { local M=`ls /asdf` && echo y; } b () { N=`ls /asdf` && echo y; } a ; b ; One will print y, the other will not.

Shellcheck helpfully warns about declaring and assigning in the same statement exactly because it masks exit status.

Re: Writing Safe Shell Scripts (2019)

#49
post #6

Earlier quoted context omitted.

Shellcheck is amazingly impressive at catching issues with shell scripts. It makes it very hard to write a shell script that does the wrong thing. Also, look at oilshell[1]; it is bash compatible out-of-the-box, but has several options to make it incompatible, but safer (e.g. no field splitting of parameter expansion by default, making quotes much less needed). 1: https://www.oilshell.org/

I really want someone to integrate fish and oil, and call the project "fish-oil" or something along those lines.

(Oil author here) Yes I think that's a great idea, and I hope to release a "liboil" so that people can do that. Let a thousand flowers bloom, etc.

The most likely thing is that Oil is going to be a much better language this year (more expressive, with safety features, and it should be fast), but the interactive UI will still be more bash-like than fish-like:

https://www.oilshell.org/blog/2020/01/making-plans.html

So I encourage anyone who's interested in that to get involved now as the interactive shell is also a years-long effort :) There are many links on the home page about how to get involved, and I wrote many blog posts so people can understand how it works.

----

Here are a couple posts on why Oil is a good foundation for an interactive shell:

https://www.oilshell.org/blog/2020/01/history-and-completion...

https://www.oilshell.org/blog/2020/01/alias-and-prompt.html

Post reply on HN