I'll throw another one: If it is longer than a ~screen, throw it away and write it in bash is just not a good language at the best of days
Shell script best practices, from a decade of scripting things
211–220 of 500 posts
Re: Shell script best practices, from a decade of scripting things
#212Hands down, shell scripting is one of my all time favorite languages. It gets tons of hate, e.g. "If you have to write more than 10 lines, then use a real language," but I feel like those assertions are more socially-founded opinions than technically-backed arguments. My basic thesis is that Shell as a programming language---with it's dynamic scope, focus on line-oriented text, and pipelines---is simply a different p…
Shell and SQL make you 10x productive over any alternative. Nothing even comes close. I've seen people scrambling for 1 hours to write some data munging, then spend another 1 hour to run it through a thread pool to utilize those cores , while somebody comfortable is shell writes a parallelized one liner, rips through GBs of data, and delivers the answer in 15 minutes. What Python is to Java, Shell is to Python. It sp…
I'd add a working knowledge of regex to that. With a decent text editor + some fairly basic regex skills you can go a long way.
Re: Shell script best practices, from a decade of scripting things
#213Hands down, shell scripting is one of my all time favorite languages. It gets tons of hate, e.g. "If you have to write more than 10 lines, then use a real language," but I feel like those assertions are more socially-founded opinions than technically-backed arguments. My basic thesis is that Shell as a programming language---with it's dynamic scope, focus on line-oriented text, and pipelines---is simply a different p…
> Hands down, shell scripting is one of my all time favorite languages. It gets tons of hate, e.g. "If you have to write more than 10 lines, then use a real language," but I feel like those assertions are more socially-founded opinions than technically-backed arguments.
It is "opinion" based on debugging scripts made by people (which might be "you but few years ago") that don't know the full extent of death-traps that are put in the language. Or really writing anything more complex.
About only strong side of shell as a language is a pipe character. Everything else is less convenient at best, actively dangerous at worst.
Sure, "how to write something in a limited language" might be fun mental excercise but as someone sitting in ops space for good part of 15 years, it's just a burden.
Hell, I'd rather debug Perl script than Bash one...
Yeah, if it is few pipes and some minor post processing I'd use it too (pipe is the easiest way to do it out of all languages I've seen) but that's about it.
It is nice to write one-liners in cmdline but characteristic that make it nice there make it worse programming language. A bit like Perl in that matter
Re: Shell script best practices, from a decade of scripting things
#214Hands down, shell scripting is one of my all time favorite languages. It gets tons of hate, e.g. "If you have to write more than 10 lines, then use a real language," but I feel like those assertions are more socially-founded opinions than technically-backed arguments. My basic thesis is that Shell as a programming language---with it's dynamic scope, focus on line-oriented text, and pipelines---is simply a different p…
Shell and SQL make you 10x productive over any alternative. Nothing even comes close. I've seen people scrambling for 1 hours to write some data munging, then spend another 1 hour to run it through a thread pool to utilize those cores , while somebody comfortable is shell writes a parallelized one liner, rips through GBs of data, and delivers the answer in 15 minutes. What Python is to Java, Shell is to Python. It sp…
For simple scripting tasks, yes. I have had the opposite experience for more critical software engineering tasks (as in, coding integrated over time and people).
Language aside, the ecosystem and culture do not afford enough in way of testing, dependency management, feature flags, static analysis, legibility, and so on. The reason people say to keep shell programs short is because of these problems, it needs to be possible to rewrite shell programs on a whim. At least then, you can A/B test and deploy at that scope.
Re: Shell script best practices, from a decade of scripting things
#215> Use bash. Using zsh or fish or any other, will make it hard for others to understand / collaborate. Among all shells, bash strikes a good balance between portability and DX. I think fish is quite a bit different in terms of syntax and semantics (I'm not very familiar with it), but zsh is essentially the same as bash except without most of the needless footguns and awkwardness. zsh also has many more advanced featur…
> Most – though obviously far from all – scripts tend to be run in environments you control; portability is often overrated and not all that important (except when it is of course)
If you're at that spot, don't use shell in the first place but whatever other scripting language your team uses. Well, unless it's "pipe this to that to that", sh has no parallel here
Re: Shell script best practices, from a decade of scripting things
#216My biggest complaint about "idiomatic" shell scripting is the use of the [ and [[ operators. It gives the illusion that [ or [[ are part of the shell syntax when actually they're just programs / builtins / functions which communicate with the rest of the script the same way (most) other things interact -- setting exit status. Specifically this means if .. then .. fi works with any program not just [ [[ operators.
Traditional shell might be:
grep -q thing if
grep -q thing Re: Shell script best practices, from a decade of scripting things
#217> Use the .sh (or .bash) extension for your file. It may be fancy to not have an extension for your script, but unless your case explicitly depends on it, you’re probably just trying to do clever stuff. Clever stuff are hard to understand. I don't agree with this one. When I name my script without extension (btw, .sh is fine, .bash is ugly) I want my script to look just like any other command: as a user I do not care…
I honestly do it mostly coz IDEA is/was mighty stupid when it comes to detecting file types compared to Emacs... altho newer editions seemed to fix that problem for the most part
Re: Shell script best practices, from a decade of scripting things
#218Earlier quoted context omitted.
That's an excellent resource. Luckily, most commonly encountered scripting issues are with whitespace in filenames/variables and running a script through shellcheck will catch most (all?) of those problems. It's amazing how edge cases can make a simple command such as 'echo' break. (Top tip - use printf instead of echo)
echo has long been unreliable. Even the built-in echo in the shells were unreliable in SunOS, because the shell would look at the binaries in your PATH and try to figure out whether to emulate the BSD vs SysV (IIRC) version of echo and then change what echo would do. So much for writing a single script (with echo) that would work for all your users on the same host. This is why you'll see code like this: echo 'prompt…
echo is unreliable; I agree. Instead, I use "paranoid" printf with leading double dash:
prinf -- "fmt str here..." "$carefully" "$quoted" "$args"Re: Shell script best practices, from a decade of scripting things
#219What would be the justification for 'cd "$(dirname "$0")"'? Going to the scripts directory does not seem very helpful. If I don't care about the current directory, I might just go to '/' or a temporary directory, when I do care about it I better stay in it or interpreting relative command line arguments is going to get difficult. When symbolic links are involved, dirname will also give the wrong directory.
To be fair, it's common to want to be in the script directory for certain classes of scripts. For example, scripts which automate some tasks in a project, and are written for a project. But, more importantly, people will google for how to set cwd to the script directory more often then will google how to go to an absolute path. Having 'cd "$(dirname "$0")"' as reference in an article discussing best practices and the…
> To be fair, it's common to want to be in the script directory for certain classes of scripts. For example, scripts which automate some tasks in a project, and are written for a project
I think it would be more correct to just use vcs to get the root of a project (or fail if you can't), instead of essentially hardcoding path to the script.
For example if you put your script in helpers/ then someone else did a refactor and moved all of the stuff into cmd/helpers/, any relative reference you put into script is now invalid and your script is doing the wrong thing
Re: Shell script best practices, from a decade of scripting things
#220I favor POSIX and dash over bash, because POSIX is more portable. If a shell script needs any kind of functionality beyond POSIX, then that's a good time to upgrade to a higher-structure programming language. Here's my related list of shell script tactics: http://github.com/sixarm/unix-shell-script-tactics
Do you even ran your code in place where bash wasn't available? I held thought like you... 10 years ago but that really doesn't happen and if it does, rest of it probably won't work either...