Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Shell script best practices, from a decade of scripting things

sharats.me

Post reply on HN
Print view View original

291–300 of 500 posts

Re: Shell script best practices, from a decade of scripting things

#291

Post by Cpoll »

throwawaaarrgh wrote: post #203

"Use set -o errexit" Only if it doesn't matter that the script fails non-gracefully. Some scripts are better to either have explicit error handling code, or simply never fail. In particular, scripts you source into your shell should not use set options to change the shell's default behavior. "Prefer to use set -o nounset." ALWAYS use this option. You can test for a variable that might not be set with "${FOO:-}". Ther…

> YAGNI

For most people, YAGNI means using convenient Bash-isms, because their scripts won't ever be run on environments that don't have Bash.

Edit: Admittedly, someone in this thread pointed out the flaw in my argument, there are plenty of cases where you can't assume you have Bash. I still hold that proofing something for all possible environments is itself a YAGNI.

Re: Shell script best practices, from a decade of scripting things

#292

Post by Cpoll »

kdmccormick wrote: post #286
Cerium wrote: post #248

Earlier quoted context omitted.

"Use [[ ]] for conditions" Oh how I hate the double square bracket. It is the source of many head scratching bugs and time wasted. "The script works in my machine!" It doesn't work in production where we only have sh. It won't exit due to an error, the if statement will gobble the error. You only find the bug after enough bug reports hit that particular condition. After a couple shots to the foot I avoid double squar…

If I may ask, why do you only have sh in production?

A common containerization philosophy is to use a bare-minimum base image and add only what you need. Something like an Alpine container doesn't come with Bash.

Re: Shell script best practices, from a decade of scripting things

#293

Post by monknomo »

Beltalowda wrote: post #51
benreesman wrote: post #32

Earlier quoted context omitted.

A little personal color: I’m kind of a terminal tweak-fanatic but I’ve stuck with bash. Ten years or so ago the cool kids were using zsh: which is in general a pretty reasonable move, it’s got way more command-line amenities than bash (at least built in). Today fish is the fucking business: fish is so much more fun as a CLI freak. But I guess I’ve got enough PTSD around when k8s or it’s proprietary equivalents get st…

I personally really dislike fish as an interactive shell as it's just so busy. Things keep popping up, everything is in so many different colours, etc. It's great if you like that sort of stuff, but I really appreciate a "quiet" environment. This is also why I use Vim: all the IDEs I tried are just so "busy". I was only talking about scripting; I know fish scripting is different, but I have no idea if it's any good.…

if you want `fish_config` opens up an easy editor for changing all the colors to whatever you find quiet and soothing.

You have a level of control over things popping up too

Re: Shell script best practices, from a decade of scripting things

#294

Post by 0xbadcafebee »

kdmccormick wrote: post #286
Cerium wrote: post #248

Earlier quoted context omitted.

"Use [[ ]] for conditions" Oh how I hate the double square bracket. It is the source of many head scratching bugs and time wasted. "The script works in my machine!" It doesn't work in production where we only have sh. It won't exit due to an error, the if statement will gobble the error. You only find the bug after enough bug reports hit that particular condition. After a couple shots to the foot I avoid double squar…

If I may ask, why do you only have sh in production?

Alpine docker container only comes with ash shell by default. If you don't use any Bash-isms, you can just write a POSIX shell script. Otherwise if you have many containers from many different sources, you might have to bake all new containers just to add Bash to them.

Re: Shell script best practices, from a decade of scripting things

#295

Post by Cerium »

kdmccormick wrote: post #286
Cerium wrote: post #248

Earlier quoted context omitted.

"Use [[ ]] for conditions" Oh how I hate the double square bracket. It is the source of many head scratching bugs and time wasted. "The script works in my machine!" It doesn't work in production where we only have sh. It won't exit due to an error, the if statement will gobble the error. You only find the bug after enough bug reports hit that particular condition. After a couple shots to the foot I avoid double squar…

If I may ask, why do you only have sh in production?

In this case production is a fleet of embedded devices.

Re: Shell script best practices, from a decade of scripting things

#296

Post by HyperSane »

GuB-42 wrote: post #151
HyperSane wrote: post #113

Earlier quoted context omitted.

sh and bash feel pretty primitive after learning PowerShell.

I tried PowerShell, hated it. The idea of manipulating objects instead of text streams is interesting, and avoids most of the footguns sh/bash have, but you also lose in flexibility. One reason is that there are thousands of command line tools in the UNIX ecosystem that process text streams and are designed to work with shells like bash. You have much less options when you are processing PowerShell objects. Note: I t…

"You have much less options when you are processing PowerShell objects."

There is simply no need for the kind of extensive text processing common in Linux because every command returns an object whose fields can be directly referenced. Combined with the ConvertTo-Json command this is incredibly powerful. Honestly it seems like you are attempting to do things in PowerShell the bash way instead of the PowerShell way.

" I think the first thing I tried to do in PowerShell was a script that scanned a directory recursively for files containing a CRC in their name, and then check it"

I wrote a PowerShell script that recursively scanned every filed and folder on our fileshares and wrote the permissions to a file for later indexing.

Re: Shell script best practices, from a decade of scripting things

#297

Post by floitsch »

xelxebar wrote: post #182

Hands down, shell scripting is one of my all time favorite languages. It gets tons of hate, e.g. "If you have to write more than 10 lines, then use a real language," but I feel like those assertions are more socially-founded opinions than technically-backed arguments. My basic thesis is that Shell as a programming language---with it's dynamic scope, focus on line-oriented text, and pipelines---is simply a different p…

Shell scripting also inspired some choices (especially syntax) of the Toit language (toitlang.org).

Clearly, it's for a different purpose, and there are some things that wouldn't work in a general-purpose language that isn't as focused on line-based string processing, but we are really happy with the things we took from bash.

Re: Shell script best practices, from a decade of scripting things

#298

Post by chubot »

blueflow wrote: post #285
coliveira wrote: post #187

Earlier quoted context omitted.

This battle was lost a long time ago. Bash is the standard on most UNIX systems. If you change this reality, one might even start to try to think about writing in fish or some other new shell. But I will not even consider another shell for scripts that need to be run by other people.

POSIX shell is the standard, not bash.

That ship has sailed, because busybox ash and dash continually implement some of bash features and semantics, which come from ksh.

And OSH implements almost all of bash

That is, The posix shell spec is missing a lot of reality. It’s not very actively maintained, unfortunately

The canonical example is not having local vars, which basically every shell supports

Re: Shell script best practices, from a decade of scripting things

#299

Post by HyperSane »

heresie-dabord wrote: post #189
HyperSane wrote: post #113

Earlier quoted context omitted.

sh and bash feel pretty primitive after learning PowerShell.

"Primitive" seems like the wrong word to apply. A modern Python programmer (a common example) might think that anything not OOP is archaic. But OOP is just one design pattern. And... it can easily lead to over-engineering and delight in the pattern per se. Bash, awk, grep et alii don't do OOP. But they are close to the data and are powerful. Compaints about the notation of these tools (e.g. "line noise") are becoming…

""Primitive" seems like the wrong word to apply"

Painful, ugly, unpleasant?

Re: Shell script best practices, from a decade of scripting things

#300

Post by asicsp »

lbhdc wrote: post #287
usrbinbash wrote: post #264

Earlier quoted context omitted.

> What is the Shell paradigm? I would argue that it's line-oriented pipelines. Which python can do realitively well, by using the `subprocess` module. Here is an example including a https://porkmail.org/era/unix/award (useless use of cat) finding all title lines in README.md and uppercasing them with `tr` import subprocess as sp cat = sp.Popen( ["cat", "README.md"], stdout=sp.PIPE, ) grep = sp.Popen( ["grep", "#"], s…

> But on the other side of that coin its alot easier in python to do a complex regular expression I am not sure I would agree. Sed fills this role quite nicely. cat README.md | grep # | tr '[:lower:] [:upper:]' | sed 's/something/something_else/'

grep+tr can be done within sed too (or go with perl for more features and easier portability)
Post reply on HN