Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Shell Style Guide

google.github.io

Post reply on HN
Print view View original

121–130 of 336 posts

Re: Shell Style Guide

#121

Post by davidcuddeback »

nvarsj wrote: post #105
_arvin wrote: post #69

I'm really surprised they went with: #!/bin/bash as opposed to: #!/usr/bin/env bash the latter feels more flexible and dependable for a script to be passed around.

I really don't like the `/usr/bin/env bash` approach (it came from the ruby world I think?). It's less portable than /bin/bash in my experience - I've been in environments, usually older ones, where it doesn't work at all. But /bin/bash works everywhere. You also can't pass command line arguments. And it's slightly more complex than just invoking /bin/bash.

> It's less portable than /bin/bash in my experience

/bin/bash breaks on FreeBSD, which installs Bash at /usr/local/bin/bash. /usr/bin/env bash works though.

> You also can't pass command line arguments.

Are you sure about that? Given this script:

    #!/usr/bin/env bash
    echo "args: $@"
it outputs:

    $ ./foo 1 2 3
    args: 1 2 3
Is that what you meant?

Re: Shell Style Guide

#122

Post by jerf »

bm1362 wrote: post #82
jerf wrote: post #75

Earlier quoted context omitted.

It's a 10-20 line Python script if you write it without any support code. If you're writing any amount of Python code that does shell things, you should either write or get an existing library. Also, it really isn't 10-20 lines to spit out the contents of some files in Python: >>> filelist = ["tmp.pl", "tmp.go", "Tmp.hs"] >>> for f in filelist: ... print(open(f).read()) It's trivial to code golf that down to 1 line,…

for i in ‘ls /tmp/foo’; do echo $i | grep bar | cut -d’-‘ -f3,4; done Is just so much easier to remember than Python’s OS library, right? I can intuitively string that together but can’t write python without looking up the docs and reading a paragraph about idiosyncrasies.

I said Python wasn't going to be more concise. I said it has a variety of other useful properties. For instance, at least as I write this, you have "cut -d’-‘"; I assume you meant apostrophes (something getting too polite in a c&p, I assume) but the apostrophes are unnecessary, but you're so used to the compromises in shell I talked about you probably put those in there automatically. I'm not criticizing that; it's a reasonable accommodation to shell's quirks. But it is a quirk.

Your code also breaks if the filenames have spaces:

    $ ls -1                                                                      
    bar-def-gh i-jkl-mno-p                                                                         
    bar-def-ghi-jkl-mno-p                                                                          
    $ for i in `ls`; do echo $i | grep bar | cut -d- -f3,4; done
    gh
    ghi-jkl
The natural Python code you'd write to replace it will do what you expect and print the file with a space in it in the way you'd expect. You can fix this in bash via "for i in [asterisk]" (or, in this case, "for i in [asterisk]bar[asterisk]"), but, well, remember when I said bash requires you to understand the half-a-dozen ways strings and filenames interact...? In Python, there's just the one way that strings work.

(HN is forcing me to say [asterisk]. AFAIK there's still no escaping for them.)

You lucked out a bit here too; I don't think I can get echo to do anything very interesting based on file names. Had you interpolated that somewhere more interesting I'd give you some security issues too, if anybody not trusted can create files, which, again, the naive Python code would be much more robust to, because in Python, you have to explicitly ask to bring a string up to an execution context or a function parameter context, unlike shell's complicated rules for interleaving them. You pay a steep price for the convenience.

Of course, this is a matter of scale. I use that sort of pipeline interactively every day. I just start shuddering when it goes in a file for a script, and shudder more if it goes into source control.

Re: Shell Style Guide

#123

Post by taesis »

pas wrote: post #81
bitlax wrote: post #49

Earlier quoted context omitted.

This document agrees with you. - If you find you need to use arrays for anything more than assignment of ${PIPESTATUS}, you should use Python. - If you are writing a script that is more than 100 lines long, you should probably be writing it in Python instead. Bear in mind that scripts grow. Rewrite your script in another language early to avoid a time-consuming rewrite at a later date.

Exchanging Bash for Python is a folly. Python seems to handle complexity better, but only on the surface. (Basically Python has nice data structures, great string formatting, and .. that's it.) Setting up the Python environment requires something, because Python comes in many flavors (2 vs 3, system installed, user installed, /usr/local) and there are a bunch of factors (pip, pyenv, pipenv, PYTHONPATH, current direct…

Re. a recommended alternative to quick and dirty bash scripts, do you think golang would do? I've barely dabbled with the language, but its default packaging (a single, static binary) and build system (`go build x`) seem well suited to rapid setup, deployment and testing, which is presumably what you'd want in this scenario.

Re: Shell Style Guide

#124

Post by peterwwillis »

Sir_Cmpwn wrote: post #3

>Bash is the only shell scripting language permitted for executables. Whelp, wrong right off the bat. I'm gonna get down my my knees here and beg everyone reading: use POSIX shell. Do not write scripts with bash. Do not write scripts with zsh. Do not write scripts with fish. Use POSIX shell. sh has a really bad interactive mode (so does bash), so I'm not gonna give anyone a hard time for using another shell as their…

Only if you need portability. If you don't need portability, write it in whatever you want. I prefer bash because it's everywhere. Zsh is too esoteric for me, but I wouldn't judge anyone for using it.

Re: Shell Style Guide

#125

Post by pas »

simias wrote: post #87
pas wrote: post #81

Earlier quoted context omitted.

Exchanging Bash for Python is a folly. Python seems to handle complexity better, but only on the surface. (Basically Python has nice data structures, great string formatting, and .. that's it.) Setting up the Python environment requires something, because Python comes in many flavors (2 vs 3, system installed, user installed, /usr/local) and there are a bunch of factors (pip, pyenv, pipenv, PYTHONPATH, current direct…

It's really not about data structure or anything like that. The big problem with any large shell script is that it's utterly difficult to do proper error handling. Usually the best you can do is check return values to detect an error and 'exit' immediately, hoping that your "trap" works correctly to clean up behind you. >Python lacks efficient, quick and dirty process control. Yeah, quick and dirty, that's kind of th…

> trap

I don't have more faith in Python's except than in Bash's trap.

If you use 3rd party code or a subprocess, then both are pretty weak compared to something kernel enforced.

> makes it massively easier to handle errors and give decent diagnostics to the user.

I don't really find that. You have to fight for input/output for subprocesses in Python.

Sure, logging is easier in Python/perl/PHP/whatever, than in Bash, but error handling is about the same. You have to do everything manually, and there's no compiler, nor type (or other magical) system, that'd warn you if you have missed something.

Re: Shell Style Guide

#126

Post by Sir_Cmpwn »

pas wrote: post #103
Sir_Cmpwn wrote: post #46

Earlier quoted context omitted.

>Where is the reward when bash is present (almost) everywhere? It's really not. It's only present on some desktop Linux distributions. It's not present on almost any other OS or anywhere in the embedded space. POSIX shell, on the other hand, is everywhere.

Why wouldn't you install Bash then, if you want to run shell scripts on those platforms? I mean, what are the use cases? Where universality is important (so let's say when someone embeds scripts in Makefiles), I understand the need for POSIX, but other that I can't even come up with a good use case for having sh as default. (Maybe to save space, you ship the embedded device or Docker container with sh, sure, but then…

>but then what kind of script you'd want to run there? Okay, maybe all of them

I think you answered your own question here.

I can elaborate, though. bash isn't portable, it uses Glibc-isms. It has to be explicitly ported to new platforms. I also pointed out elsewhere that the value-add of bash is dubious at best - if you need to use bashisms you're better off not writing a shell script at all. So really, the question is: why do you need bash? You have to justify using non-standard tools, not the other way around.

Re: Shell Style Guide

#127

Post by gre »

> If performance matters, use something other than shell.

I wrote a script that wrapped compiler calls to make dummy output files to debug a huge compilation. Since startup time was the most important metric, bash actually had by far the best performance.

Re: Shell Style Guide

#128

Post by joshuamorton »

Sir_Cmpwn wrote: post #111
joshuamorton wrote: post #104

Earlier quoted context omitted.

The second thing is not more portable than the first (unfortunately).

https://www.python.org/dev/peps/pep-0394/

That, unfortunately, only works if you don't need to maintain backwards compatibility with older systems. `python` will exist and point to python2 on any system. `python2` may not if it predates or ignores the PEP. `python3` didn't even come installed by default on a lot of nixes until recently, making it the worst of option of the three.

Re: Shell Style Guide

#129

Post by billhathaway »

_arvin wrote: post #69

I'm really surprised they went with: #!/bin/bash as opposed to: #!/usr/bin/env bash the latter feels more flexible and dependable for a script to be passed around.

Google's linux systems are standardized, there isn't a concern about running a script on 10 different OS variants

Re: Shell Style Guide

#130

Post by falsedan »

moviuro wrote: post #55
thankthunk wrote: post #42

Earlier quoted context omitted.

> Do not write scripts with bash. It's a bit too late for that. Bash is pretty much the de facto default shell for linux. Telling people not to use bash scripts is like telling web devs not to use javascript or OS devs not to use C. It may change in the future, but bash is so entrenched, it's going to take an extraordinary use case for people to move from bash.

bash is completely out of date on macOS (GPL2 vs. GPL3), and not shipped on any *BSD AFAICT. Some Linux distro don't ship it either (void, alpine IIRC). And, most of the time, bash is used because people don't know any better, see e.g. https://github.com/OpenRA/OpenRA/pull/8405/files

> bash is completely out of date on macOS

I love finding and removing all the GNU/Linux-ism in my bash code when I move it from a dev host to my personal laptop. macOS coreutils don't have GNU longopts, surprise!

Post reply on HN