How to do things safely in Bash (2018) • Hacker News Board

Re: How to do things safely in Bash (2018)

#11

Post by woudsma » Sat, Apr 03, 2021, 6:44 PM UTC

alblue wrote: post #5

If you’re interested in writing safe shell scripts then check out shellcheck: https://github.com/koalaman/shellcheck If you’re interested I’ve written a git hook for it that runs a check when you git commit: https://github.com/alblue/scripts/blob/main/shellcheck-pre-c... You should also check out her Google shell script style guide: https://google.github.io/styleguide/shellguide.html

Shellcheck and the VS Code Shellcheck extension[1] are really good. I never write a bash script without it.

Built-in autofixes and direct links to extensive documentation on each rule, with easy to grasp examples. It's just awesome.

[1]: https://marketplace.visualstudio.com/items?itemName=timonwon...

Re: How to do things safely in Bash (2018)

#12

Post by hutrdvnj » Sat, Apr 03, 2021, 6:44 PM UTC

I think the best way to write things safely is to not write them in Bash in the first place. Better pick a strongly typed language of your choice.

Re: How to do things safely in Bash (2018)

#13

Post by neolog » Sat, Apr 03, 2021, 7:01 PM UTC

yeowMeng wrote: post #7

pwdisswordfish8 wrote: post #2

I've recently taken to not bothering with shell scripts at all. I just use Python instead. The type system, while it's by no means state of the art, is still miles ahead of the stringly-typed bash. The libraries are excellent; much can be done with the standard library alone. The scripts are much faster (even though the language isn't particularly tuned for performance), since I don't need to spawn a subprocess for e…

> The [python] scripts are much faster... Pipelines are performant; many algorithms can be expressed as pipelines. Bash has many built-in string manipulation mechanisms which do not require a sub shell. Why do people who subscribe to the “avoid shell scripts at all costs” ideology feel so passionately about prescribing to others? I like python, but if I am in a rush it’s shell.

> Why do people who subscribe to the “avoid shell scripts at all costs” ideology feel so passionately about prescribing to others?

Are you asking why people make recommendations in general?

Re: How to do things safely in Bash (2018)

#14

Post by stormbrew » Sat, Apr 03, 2021, 7:05 PM UTC

pwdisswordfish8 wrote: post #2

I've recently taken to not bothering with shell scripts at all. I just use Python instead. The type system, while it's by no means state of the art, is still miles ahead of the stringly-typed bash. The libraries are excellent; much can be done with the standard library alone. The scripts are much faster (even though the language isn't particularly tuned for performance), since I don't need to spawn a subprocess for e…

This, and I mean very specifically this with python always as the go-to replacement, comes up every single time anyone even utters the words "shell" or "bash" and every time all I can think is that the people saying it aren't writing the same kinds of shell scripts I ever see, use, or wind up writing.

And every time I've ever seen something that's allegedly "a shell script replaced with a python script" it's way more complicated than an equivalent shell script would have been.

I just wonder, really, what "a shell script" even is to you if not something where you need to "shell out to external processes"?

Re: How to do things safely in Bash (2018)

#15

Post by arielb1 » Sat, Apr 03, 2021, 7:11 PM UTC

When calling shell from bash (to use pipes etc.), I found it useful to pass variables via the environment, like this:

    def safe_call(command, **keywds):
        return subprocess.check_call(f'set -euo pipefail; {command}', shell=True, env=keywds)

    safe_call('command1 -- "$bar" | command2 --baz="$baz" | command3 > "$output_file"', bar=bar, baz=baz, output_file=output_file)

Re: How to do things safely in Bash (2018)

#16

Post by macintux » Sat, Apr 03, 2021, 7:16 PM UTC

hutrdvnj wrote: post #12

I think the best way to write things safely is to not write them in Bash in the first place. Better pick a strongly typed language of your choice.

How many strongly typed languages are:

+ Installed everywhere

+ Even within 20% as concise as bash

Re: How to do things safely in Bash (2018)

#17

Post by montroser » Sat, Apr 03, 2021, 7:18 PM UTC

This all has its place, and I learned a good few things from the article. But also, in many situations when you may not need to deal with completely arbitrary input, it can be nice to shed all of the "correct" syntax and just optimize for readability.

Re: How to do things safely in Bash (2018)

#18

Post by blabitty » Sat, Apr 03, 2021, 7:23 PM UTC

stormbrew wrote: post #14

pwdisswordfish8 wrote: post #2

I've recently taken to not bothering with shell scripts at all. I just use Python instead. The type system, while it's by no means state of the art, is still miles ahead of the stringly-typed bash. The libraries are excellent; much can be done with the standard library alone. The scripts are much faster (even though the language isn't particularly tuned for performance), since I don't need to spawn a subprocess for e…

This, and I mean very specifically this with python always as the go-to replacement, comes up every single time anyone even utters the words "shell" or "bash" and every time all I can think is that the people saying it aren't writing the same kinds of shell scripts I ever see, use, or wind up writing. And every time I've ever seen something that's allegedly "a shell script replaced with a python script" it's way more…

I wonder too. Things like file and directory manipulations are way more complicated in native Python and I find the exceptions harder to troubleshoot than native commands. I'm not even going to get into pythons version and dependency hell, suffice it to say I have more confidence in a standard version of bash and standard utilities generally being present on a modern Linux system. What I've seen from pythonistas as reasoning for this replacememt love is usually one or a combination of a few things - a claimed write once run anywhere ability that I think is generally fantasy, a belief that Python is a "real" language and shell is not which usually just means "I don't like shell or think it is unfashionable", or a belief that Python code can be made more reusable. The person in question almost always has a background of feature developer that is now doing devops type tasks.

All that said I like python and there are tasks I absolutely prefer it for, usually anything to do with scraping some web API that returns a complex piece of JSON or xml.

Re: How to do things safely in Bash (2018)

#19

Post by maxrev17 » Sat, Apr 03, 2021, 7:31 PM UTC

See the same debate ensue each time with bash - use python, they shout. Perhaps the overriding factor here is familiarity? Past looping and some basic flow control I'm guessing most people run out of being comfortable in bash and can get things done quicker in their fave Lang?

Re: How to do things safely in Bash (2018)

#20

Post by geocrasher » Sat, Apr 03, 2021, 7:32 PM UTC

I've been shell scripting intermittently for a long time and have written some bash scripts that have seen a lot of use. This document really showed me how bad some of my programming is. But then again, the first thing I tell people when demonstrating one of these programs: "I am not a programmer. I am a bash scripter. And, a bad one." The stuff still works. But I would feel a lot better if I were doing things the right way. I'll be reviewing this next time I write a bigger s/program/script/