Writing Safe Shell Scripts (2019) • Hacker News Board

Re: Writing Safe Shell Scripts (2019)

#11

Post by clarry » Sat, Feb 01, 2020, 7:49 PM UTC

emj wrote: post #10

TheDong wrote: post #9

Earlier quoted context omitted.

Yes, mktemp can fail. For example, "TMPDIR=/dev/null mktemp -d" will reliably fail. You shouldn't be validating it starts with "/tmp" though, because on quite a few systems, people set TMPDIR=/var/tmp. You absolutely should check if mktemp exited nonzero, but if it exited 0, the directory should be safe to use. If you want to be really paranoid, you can check if the output is an empty string too after checking the ex…

Please tell me it's enough to check the exit code! That's all I ever do and I have hundreds of mktemp scripts out there. In the end I think shelling is mostly about controlling your inputs.

Not just inputs, unfortunately. TOCTOUs (and races in general), for instance, are very common problem in shell scripts.

Re: Writing Safe Shell Scripts (2019)

#12

Post by lalaland1125 » Sat, Feb 01, 2020, 7:53 PM UTC

tyingq wrote: post #7

Checking $PATH or explicitly calling outside resources is probably also advisible. In general, this article seems very light. No mention of LD_LIBRARY_PATH, for example. I imagine there's probably a better guide to writing secure scripts somewhere else.

These sorts of security measures are only really necessary when you have a setuid or setgid shell script. (However, a much better suggestion would be to simply not write setuid or setgid shell scripts because they are a security nightmare.)

Re: Writing Safe Shell Scripts (2019)

#13

Post by vbernat » Sat, Feb 01, 2020, 7:54 PM UTC

Since it's bash-specific, I think people should use zsh instead. No need to quote variables as by default, splitting is not done. And you get access to arrays and associative arrays.

Re: Writing Safe Shell Scripts (2019)

#14

Post by aidenn0 » Sat, Feb 01, 2020, 7:55 PM UTC

chousuke wrote: post #5

Can mktemp fail? I recently wrote a script that ensures that the directory created by mktemp actually exists and starts with /tmp so that when the script wipes out its temporary data at the end, it will not ever run something like "rm -rf /". Using fully-qualified paths for everything is also (not) fun. In another script I was leery about running rm -f -- /path/foo* so I instead used find /path -mindepth 1 -maxdepth…

With the "--" and assuming GNU rm, rm -f -- /path/foo* is safe. If you use a glob pattern that does not match itself, then you will want nullglob:

  rm foo.[cC]

will remove a file named literally foo.[Cc] if neither foo.c nor foo.C exist, despite the fact that foo.[Cc] does not match the glob. nullglob is probably a good default for bashscripts, but I don't think it exists in posix.

Re: Writing Safe Shell Scripts (2019)

#15

Post by lalaland1125 » Sat, Feb 01, 2020, 7:55 PM UTC

I really wonder whether there is really any point in writing shell scripts anymore. Practically every Unix/Linux box in existence has at least some version of Python 2 that can be used as a complete and total replacement. I can't think of a single situation where I would need a shell script and a Python script wouldn't be much cleaner, simpler, and more maintainable.

Re: Writing Safe Shell Scripts (2019)

#16

Post by savolai » Sat, Feb 01, 2020, 7:57 PM UTC

smartmic wrote: post #3

Shell scripts have their well-deserved place in Unix systems. A general recommendation to use Python or other high-level scripting languages without a discussion about the reasons why and when to use shell may lead to wrong conclusions.

I'd love to read a good blog post on when to use shell scripts vs Python or other programming languages, which seem far more accessible to me.

Re: Writing Safe Shell Scripts (2019)

#17

Post by arno_v » Sat, Feb 01, 2020, 8:03 PM UTC

lalaland1125 wrote: post #15

I really wonder whether there is really any point in writing shell scripts anymore. Practically every Unix/Linux box in existence has at least some version of Python 2 that can be used as a complete and total replacement. I can't think of a single situation where I would need a shell script and a Python script wouldn't be much cleaner, simpler, and more maintainable.

I still think for very small stuff it's easier to just write a shell script. For instance a docker entry point, where you set a few env vars, download some required files and start your main application.

Re: Writing Safe Shell Scripts (2019)

#18

Post by alexis_fr » Sat, Feb 01, 2020, 8:07 PM UTC

At the beginning of each file:

    #!/bin/bash
    set -euf -o pipefail
    cd $(dirname $0)

Then add "" everywhere ;) And they say, write Python instead, except I’m dubious because python programs can have a lot of dependencies which can be tricky to install.

Re: Writing Safe Shell Scripts (2019)

#19

Post by sillysaurusx » Sat, Feb 01, 2020, 8:08 PM UTC

lalaland1125 wrote: post #15

I really wonder whether there is really any point in writing shell scripts anymore. Practically every Unix/Linux box in existence has at least some version of Python 2 that can be used as a complete and total replacement. I can't think of a single situation where I would need a shell script and a Python script wouldn't be much cleaner, simpler, and more maintainable.

Ok, go ahead and port all 200 of these scripts to Python please: https://github.com/shawwn/scrap

PR's welcome. ;)

Python makes sense for scripts that need good argument parsing, or complicated intermediate input processing. But it's pretty annoying to get a shell pipeline working in Python. `foo | bar | baz` is about 15 characters in bash.

Here's an example. I use `llbranch` all the time: https://github.com/shawwn/scrap/blob/master/llbranch

It lists all branches in a project. It's nicely colorized. What would the equivalent Python be? Who knows, but it'd be much longer.

Re: Writing Safe Shell Scripts (2019)

#20

Post by TheDong » Sat, Feb 01, 2020, 8:08 PM UTC

emj wrote: post #10

TheDong wrote: post #9

Earlier quoted context omitted.

Yes, mktemp can fail. For example, "TMPDIR=/dev/null mktemp -d" will reliably fail. You shouldn't be validating it starts with "/tmp" though, because on quite a few systems, people set TMPDIR=/var/tmp. You absolutely should check if mktemp exited nonzero, but if it exited 0, the directory should be safe to use. If you want to be really paranoid, you can check if the output is an empty string too after checking the ex…

Please tell me it's enough to check the exit code! That's all I ever do and I have hundreds of mktemp scripts out there. In the end I think shelling is mostly about controlling your inputs.

Yes, it's enough to check the exit code. The parent poster's being overly paranoid.

The only thing that checking the exit code won't catch is bugs in mktemp/bash, or bad memory / solar flare bitflips.