Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Shell Style Guide

google.github.io

Post reply on HN
Print view View original

91–100 of 336 posts

Re: Shell Style Guide

#91

Post by NiveaGeForce »

chrisfinazzo wrote: post #58
NiveaGeForce wrote: post #37

Earlier quoted context omitted.

And on Windows there is something better than Bash, called PowerShell.

Not to put too fine a point on it, but that's nonsense. Historically, PowerShell was supposed to be a replacement for cmd.exe and roughly equivalent to bash, yet suitable for the Windows environment. They tried early on to port some of the UNIX & Linux tools to Windows and it didn't fly, so this is what we're left with. Although parts of Powershell are open source now, I suspect the mere existence of WSL means that M…

> Not to put too fine a point on it, but that's nonsense.

In PowerShell you pass structured data around instead of globs of text, which is much more maintainable. Using Bash on Windows is like going backwards, and it's sad that most developers don't even realize it.

> Historically, PowerShell was supposed to be a replacement for cmd.exe and roughly equivalent to bash, yet suitable for the Windows environment. They tried early on to port some of the UNIX & Linux tools to Windows and it didn't fly, so this is what we're left with.

Nonsense, Powershell is a fundamental improvement. It's not a "this is what's we're left with" situation.

> Although parts of Powershell are open source now, I suspect the mere existence of WSL means that Microsoft understands it has more to gain from compatibility with the rest of the world (Bash is nigh-everywhere) than imposing their particular philosophy on people.

Yeah, let's just drag everything down to the lowest common denominator.

> (See also, Apple's decision to adopt UNIX underpinnings)

Which is sad, since it could have been so much more. The Unix way of doing things, isn't the be all and all.

https://www.youtube.com/watch?v=rmsIZUuBoQs

https://news.ycombinator.com/item?id=16813796

Re: Shell Style Guide

#92

Post by laurent123456 »

pas wrote: post #81
bitlax wrote: post #49

Earlier quoted context omitted.

This document agrees with you. - If you find you need to use arrays for anything more than assignment of ${PIPESTATUS}, you should use Python. - If you are writing a script that is more than 100 lines long, you should probably be writing it in Python instead. Bear in mind that scripts grow. Rewrite your script in another language early to avoid a time-consuming rewrite at a later date.

Exchanging Bash for Python is a folly. Python seems to handle complexity better, but only on the surface. (Basically Python has nice data structures, great string formatting, and .. that's it.) Setting up the Python environment requires something, because Python comes in many flavors (2 vs 3, system installed, user installed, /usr/local) and there are a bunch of factors (pip, pyenv, pipenv, PYTHONPATH, current direct…

I had some success with Node.js (though it was in a Node-based project to begin with). I haven't tried piping commands, but to run a bunch of commands and then have all the logic in JavaScript it's quite convenient.

Here's an example that builds a React Native app, while updating version numbers and Git tags:

https://github.com/laurent22/joplin/blob/master/Tools/releas...

Re: Shell Style Guide

#93

Post by susam »

Sir_Cmpwn wrote: post #46
reacweb wrote: post #43

Earlier quoted context omitted.

I have the opposite opinion. I have seen developers proudly putting /bin/sh instead of /bin/bash, without any other shell to test. When the script was run on ubuntu instead of redhat, this called dash and we discovered that the script was finally not compliant on subtle details. Writing POSIX compliant scripts is harder. Where is the reward when bash is present (almost) everywhere?

>Where is the reward when bash is present (almost) everywhere? It's really not. It's only present on some desktop Linux distributions. It's not present on almost any other OS or anywhere in the embedded space. POSIX shell, on the other hand, is everywhere.

> POSIX shell, on the other hand, is everywhere.

Indeed! The significance of this, I believe, is underappreciated.

During 2008-2015, when I used to work for a network security company, we had a file transfer script[1] for Unix and Linux systems. It was written entirely as a shell script. It began as a simple wrapper around the scp and sftp commands for which a shell script was appropriate. The fact that every Unix and Linux system comes with a shell meant that we could ship a single file to all target Unix and Linux systems without requiring the administrator to install any new packages. The alternative was writing the file transfer functionality in C or C++ which would have required us to build and test the solution for every possible Unix system that our customers used or writing it in Java which would have required us to require the customer to install JVM on their systems. Many customer admins were averse to installing any additional packages on their systems. Given these constraints, a shell script was a good solution.

Unfortunately, the initial shell script relied on several bashisms.[2] Some customers began complaining that their Solaris or AIX system did not have bash, and they needed the script to work with Bourne shell or Korn shell. The customers refused to install bash. So I decided to clean up the script, remove all bashisms from it, and ensure that only POSIX specified features were used, only POSIX specified commands were invoked with POSIX specified options only (with the exception of the scp and sftp commands of course because that was the primary reason why the customers wanted the script).

I am not fond of bashisms. If I need bashims like arrays or regex substitutions in parameter expansions, it's a sign that the script is becoming complex and I switch to a modern programming language. These days, when I write a shell script[3], I restrict myself to POSIX.2001 (2004 edition)[4], write unit tests to exercise every line of code, run the unit tests with bash, ksh, zsh, dash, posh, and yash on Debian, run the tests with bash, ksh, and zsh on macOS, and use Kcov[5] to measure test coverage.

[1]: https://community.rsa.com/docs/DOC-53124

[2]: https://mywiki.wooledge.org/Bashism

[3]: https://github.com/susam/vimer

[4]: http://pubs.opengroup.org/onlinepubs/009695399/

[5]: https://github.com/SimonKagstrom/kcov

Re: Shell Style Guide

#94

Post by chrisfinazzo »

pas wrote: post #81
bitlax wrote: post #49

Earlier quoted context omitted.

This document agrees with you. - If you find you need to use arrays for anything more than assignment of ${PIPESTATUS}, you should use Python. - If you are writing a script that is more than 100 lines long, you should probably be writing it in Python instead. Bear in mind that scripts grow. Rewrite your script in another language early to avoid a time-consuming rewrite at a later date.

Exchanging Bash for Python is a folly. Python seems to handle complexity better, but only on the surface. (Basically Python has nice data structures, great string formatting, and .. that's it.) Setting up the Python environment requires something, because Python comes in many flavors (2 vs 3, system installed, user installed, /usr/local) and there are a bunch of factors (pip, pyenv, pipenv, PYTHONPATH, current direct…

I would be remiss if I didn't mention shellcheck, which will also warn you against doing stupid things.

Very useful when debugging.

https://github.com/koalaman/shellcheck

Re: Shell Style Guide

#95

Post by [deleted] »

bm1362 wrote: post #82
jerf wrote: post #75

Earlier quoted context omitted.

It's a 10-20 line Python script if you write it without any support code. If you're writing any amount of Python code that does shell things, you should either write or get an existing library. Also, it really isn't 10-20 lines to spit out the contents of some files in Python: >>> filelist = ["tmp.pl", "tmp.go", "Tmp.hs"] >>> for f in filelist: ... print(open(f).read()) It's trivial to code golf that down to 1 line,…

for i in ‘ls /tmp/foo’; do echo $i | grep bar | cut -d’-‘ -f3,4; done Is just so much easier to remember than Python’s OS library, right? I can intuitively string that together but can’t write python without looking up the docs and reading a paragraph about idiosyncrasies.

[deleted]

Re: Shell Style Guide

#96

Post by moviuro »

_arvin wrote: post #69

I'm really surprised they went with: #!/bin/bash as opposed to: #!/usr/bin/env bash the latter feels more flexible and dependable for a script to be passed around.

It is, but don't forget it's a guide for Google engineers on the Google machines.

People who share code for the world to use should use:

- `#!/usr/bin/env bash`

- or `#!/bin/sh` for POSIX shell scripting

Re: Shell Style Guide

#97

Post by deeg »

This quote on whether to always use braces with variable names (e.g. "${var}" or "$var"):

> These are meant to be guidelines, as the topic seems too controversial for a mandatory regulation.

It's nice to see that even Google has bike-shedding arguments.

Re: Shell Style Guide

#98

Post by plorkyeran »

bm1362 wrote: post #82
jerf wrote: post #75

Earlier quoted context omitted.

It's a 10-20 line Python script if you write it without any support code. If you're writing any amount of Python code that does shell things, you should either write or get an existing library. Also, it really isn't 10-20 lines to spit out the contents of some files in Python: >>> filelist = ["tmp.pl", "tmp.go", "Tmp.hs"] >>> for f in filelist: ... print(open(f).read()) It's trivial to code golf that down to 1 line,…

for i in ‘ls /tmp/foo’; do echo $i | grep bar | cut -d’-‘ -f3,4; done Is just so much easier to remember than Python’s OS library, right? I can intuitively string that together but can’t write python without looking up the docs and reading a paragraph about idiosyncrasies.

This is going to come down to what you're most familiar with. I personally can never remember what the arguments to `cut` are due to rarely using it, while I'd be able to write the equivalent python from memory.

Re: Shell Style Guide

#99

Post by joshuamorton »

bm1362 wrote: post #82
jerf wrote: post #75

Earlier quoted context omitted.

It's a 10-20 line Python script if you write it without any support code. If you're writing any amount of Python code that does shell things, you should either write or get an existing library. Also, it really isn't 10-20 lines to spit out the contents of some files in Python: >>> filelist = ["tmp.pl", "tmp.go", "Tmp.hs"] >>> for f in filelist: ... print(open(f).read()) It's trivial to code golf that down to 1 line,…

for i in ‘ls /tmp/foo’; do echo $i | grep bar | cut -d’-‘ -f3,4; done Is just so much easier to remember than Python’s OS library, right? I can intuitively string that together but can’t write python without looking up the docs and reading a paragraph about idiosyncrasies.

Os.listdir isn't much harder to remember, and of course os.walk is actually probably easier for recursive filesystem actions.

Re: Shell Style Guide

#100

Post by coldtea »

pas wrote: post #81
bitlax wrote: post #49

Earlier quoted context omitted.

This document agrees with you. - If you find you need to use arrays for anything more than assignment of ${PIPESTATUS}, you should use Python. - If you are writing a script that is more than 100 lines long, you should probably be writing it in Python instead. Bear in mind that scripts grow. Rewrite your script in another language early to avoid a time-consuming rewrite at a later date.

Exchanging Bash for Python is a folly. Python seems to handle complexity better, but only on the surface. (Basically Python has nice data structures, great string formatting, and .. that's it.) Setting up the Python environment requires something, because Python comes in many flavors (2 vs 3, system installed, user installed, /usr/local) and there are a bunch of factors (pip, pyenv, pipenv, PYTHONPATH, current direct…

>Python seems to handle complexity better, but only on the surface. (Basically Python has nice data structures, great string formatting, and .. that's it.)

It's a complete scripting language, with tons of features, from a comprehensive standard library with something for most needs, to a packaging system, isolated environments, async io, and more. And lots of expressivity in structuring your program (including optional types).

So, not sure what the above "and that's it" means. That it's not Haskell? Yes, it's not.

>Setting up the Python environment requires something, because Python comes in many flavors (2 vs 3, system installed, user installed, /usr/local) and there are a bunch of factors (pip, pyenv, pipenv, PYTHONPATH, current directory with modules, requirement for python-dev headers, and GCC to compile native parts of packages, and then of course maybe extra shared libraries too) that can affect the "script".

If you're doing dev-ops with such scripts, you already need to handle the environment in general.

Post reply on HN