Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Shell script best practices, from a decade of scripting things

sharats.me

Post reply on HN
Print view View original

181–190 of 500 posts

Re: Shell script best practices, from a decade of scripting things

#181

Post by kwertzzz »

jraph wrote: post #126
ndsipa_pomu wrote: post #116

Earlier quoted context omitted.

I see your point, but you can just as easily run "sh script" although that does imply that you already know that it's a shell script (obviously you wouldn't just run something from the internet without checking it first).

> obviously you wouldn't just run something from the internet without checking it first This died a long time ago with the pervasive use of NPM and PIP and the likes. Most developers probably run a lot of random unchecked shit all the time with local user privileges today without a blink. Somehow people are ready for all this, but are still afraid to run a random shell script from the internet. I guess this fear is o…

> This died a long time ago with the pervasive use of NPM and PIP and the likes.

When a malicious package is found on NPM or PIP, it will get removed. However, it is quite unlikely that a website will be taken town for a malicious script (or only after a long time).

I really doubt that most readers of HN would run a random script unless it comes from a source they trust (trusted enough to least to remove a malicious script in a timely fashion).

Re: Shell script best practices, from a decade of scripting things

#182

Post by xelxebar »

Hands down, shell scripting is one of my all time favorite languages. It gets tons of hate, e.g. "If you have to write more than 10 lines, then use a real language," but I feel like those assertions are more socially-founded opinions than technically-backed arguments.

My basic thesis is that Shell as a programming language---with it's dynamic scope, focus on line-oriented text, and pipelines---is simply a different programming paradigm than languages like Perl, Python, whatever.

Obviously, if your mental model is BASIC and you try to write Python, then you encounter lots of friction and it's easy for the latter to feel hacky, bad and ugly. To enjoy and program Python well, it's probably best to shift your mental model. The same goes for Shell.

What is the Shell paradigm? I would argue that it's line-oriented pipelines. There is a ton to unpack in that, but a huge example where I see friction is overuse of variables in scripts. Trying to stuff data inside variables, with shell's paucity of data types is a recipe for irritation. However, if you instead organize all your data in a format that's sympathetic to line-oriented processing on stdin-stdout, then shell will work with you instead of against.

/2cents

Re: Shell script best practices, from a decade of scripting things

#183

Post by rroot »

1. If you have to start with a template, then shell script is not the right thing for whatever you're trying to accomplish.

2. Shell scripts are wonderful, but once they exceed a few lines (give or take 50), they've entered the fast track on becoming a maintenance headache and a liability.

Re: Shell script best practices, from a decade of scripting things

#184

Post by marklgr »

xelxebar wrote: post #182

Hands down, shell scripting is one of my all time favorite languages. It gets tons of hate, e.g. "If you have to write more than 10 lines, then use a real language," but I feel like those assertions are more socially-founded opinions than technically-backed arguments. My basic thesis is that Shell as a programming language---with it's dynamic scope, focus on line-oriented text, and pipelines---is simply a different p…

> "If you have to write more than 10 lines, then use a real language"

I swear, there should be a HN rule against those. It pollutes every single Shell discussions, bringing nothing to them and making it hard for others do discuss the real topic.

Re: Shell script best practices, from a decade of scripting things

#185

Post by coliveira »

I wrote a fair number of bash scripts, and the area where they're definitely weaker than using a mainstream programming language is debugging. If something bad goes on a large script, it is not only harder to figure out why, but sometimes the error may be in one of a dozen native UNIX commands that have nothing to do with bash. The interaction between the shell and these UNIX commands is the weak point in the process and you can spend a long time trying to figure out what is really going on.

Re: Shell script best practices, from a decade of scripting things

#186

Post by psychstudio »

xelxebar wrote: post #182

Hands down, shell scripting is one of my all time favorite languages. It gets tons of hate, e.g. "If you have to write more than 10 lines, then use a real language," but I feel like those assertions are more socially-founded opinions than technically-backed arguments. My basic thesis is that Shell as a programming language---with it's dynamic scope, focus on line-oriented text, and pipelines---is simply a different p…

Kindred spirit. I particularly love variable variables and exploit them often. Some would call it abuse I guess.

Re: Shell script best practices, from a decade of scripting things

#187

Post by coliveira »

hiepph wrote: post #165
Beltalowda wrote: post #19

> Use bash. Using zsh or fish or any other, will make it hard for others to understand / collaborate. Among all shells, bash strikes a good balance between portability and DX. I think fish is quite a bit different in terms of syntax and semantics (I'm not very familiar with it), but zsh is essentially the same as bash except without most of the needless footguns and awkwardness. zsh also has many more advanced featur…

I can't really stand Bash's arcane syntax, it drains my brain power (and time of consulting manual) every time I have to work with it. Switching to Fish has been a breath of fresh air for me. I think some people who want to use only Bash need to open their conservative mind. All of my personal shell scripts now are converted to Fish. If I want to run some POSIX-compatible script then I just use `bash scripts.sh` Of c…

This battle was lost a long time ago. Bash is the standard on most UNIX systems. If you change this reality, one might even start to try to think about writing in fish or some other new shell. But I will not even consider another shell for scripts that need to be run by other people.

Re: Shell script best practices, from a decade of scripting things

#188

Post by tuyiown »

More than one decade of shell script: bash is not shell, and talking about bash without version is suspicious.

I won't check with for version those tips applies, and continue writing POSIX shell as much as can. I might check which or those suggestions are POSIX, though.

Re: Shell script best practices, from a decade of scripting things

#189

Post by heresie-dabord »

HyperSane wrote: post #113
Beltalowda wrote: post #19

> Use bash. Using zsh or fish or any other, will make it hard for others to understand / collaborate. Among all shells, bash strikes a good balance between portability and DX. I think fish is quite a bit different in terms of syntax and semantics (I'm not very familiar with it), but zsh is essentially the same as bash except without most of the needless footguns and awkwardness. zsh also has many more advanced featur…

sh and bash feel pretty primitive after learning PowerShell.

"Primitive" seems like the wrong word to apply. A modern Python programmer (a common example) might think that anything not OOP is archaic. But OOP is just one design pattern. And... it can easily lead to over-engineering and delight in the pattern per se.

Bash, awk, grep et alii don't do OOP. But they are close to the data and are powerful.

Compaints about the notation of these tools (e.g. "line noise") are becoming silly now that we've seen the appearance of some Powershell and Python statements. Any non-trivial notation will have to make choices. ^_^

Re: Shell script best practices, from a decade of scripting things

#190

Post by ducktective »

xelxebar wrote: post #182

Hands down, shell scripting is one of my all time favorite languages. It gets tons of hate, e.g. "If you have to write more than 10 lines, then use a real language," but I feel like those assertions are more socially-founded opinions than technically-backed arguments. My basic thesis is that Shell as a programming language---with it's dynamic scope, focus on line-oriented text, and pipelines---is simply a different p…

>However, if you instead organize all your data in a format that's sympathetic to line-oriented processing on stdin-stdout, then shell will work with you instead of against.

Not even that is necessary. Just use structured data formats like json. If you are consuming some API that is not json but still structured, use `rq` to convert it to json. Then use `jq` to slice and dice through the data.

dmenu + fzf + jq + curl is my bread and butter in shell scripts.

However, I still haven't managed to find a way to do a bunch of tasks concurrently. No, xargs and parallel don't cut it. Just give me an opinionated way to do this that is easily inspectable, loggable and debuggable. Currenly I hack together functions in a `((job_i++ < max_jobs)) || wait -n` spaghetti.

Post reply on HN