Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Shell script best practices, from a decade of scripting things

sharats.me

Post reply on HN
Print view View original

151–160 of 500 posts

Re: Shell script best practices, from a decade of scripting things

#151

Post by GuB-42 »

HyperSane wrote: post #113
Beltalowda wrote: post #19

> Use bash. Using zsh or fish or any other, will make it hard for others to understand / collaborate. Among all shells, bash strikes a good balance between portability and DX. I think fish is quite a bit different in terms of syntax and semantics (I'm not very familiar with it), but zsh is essentially the same as bash except without most of the needless footguns and awkwardness. zsh also has many more advanced featur…

sh and bash feel pretty primitive after learning PowerShell.

I tried PowerShell, hated it. The idea of manipulating objects instead of text streams is interesting, and avoids most of the footguns sh/bash have, but you also lose in flexibility.

One reason is that there are thousands of command line tools in the UNIX ecosystem that process text streams and are designed to work with shells like bash. You have much less options when you are processing PowerShell objects.

Note: I think the first thing I tried to do in PowerShell was a script that scanned a directory recursively for files containing a CRC in their name, and then check it, or something along these lines. After several hours of trying, I simply couldn't do it while it was relatively straightforward in bash, even with spaces in file names.

And that's not that I like UNIX shell scripting, in fact I hate it, so many footguns, that's why I wanted to try PowerShell, but it didn't fit my needs.

Re: Shell script best practices, from a decade of scripting things

#153

Post by erlkonig »

eddyg wrote: post #74
bluetomcat wrote: post #46

Use the shell only if your script is mostly about calling other programs and filtering and redirecting their output. That's what the syntax of these languages is optimised for. As soon as you need any data manipulation (i.e. arrays, computation, etc.) it becomes a pain and Python is the much better fit.

And, if we’re being honest, makes Perl an even better fit. https://stackoverflow.blog/2022/07/06/why-perl-is-still-rele... https://stackoverflow.blog/2022/09/08/this-is-not-your-grand...

I've had Perl change its syntax on me and break my pet monitoring system too often to make me feel good about Perl. It hasn't been too hard to keep it running (20 years so far), but starting fresh I'd probably use Python.

Python is much cleaner code than Perl for the most part as well.

However, for anything that should run forever, make sure you have a copy of all of its source code AND its libraries AND the source code for it's compiler. Repo rot is a serious problem over time.

Re: Shell script best practices, from a decade of scripting things

#154

Post by Beltalowda »

orwin wrote: post #125
HyperSane wrote: post #113

Earlier quoted context omitted.

sh and bash feel pretty primitive after learning PowerShell.

I feel like powershell hides too much to be used regularly. I have a dozen of small shellscripts and aliases to do basically what PS help me to do when i work on windows (and some), but at least i know how it work behind. I had to work with Sencha/ExtJS early 2010. It was the same feeling. Yes, it is powerfull, but too much magic happen for something without a clear orientation (at the time, now it is used for data l…

> I had to work with Sencha/ExtJS early 2010. It was the same feeling. Yes, it is powerfull, but too much magic happen for something without a clear orientation (at the time, now it is used for data loaded frontend i think). PS i don't understand what it wants me to do.

Bit off-topic, but I worked with ExtJS around the same time, and I found it one of the most confusing development experiences I ever had. "It's so easy, just add this one property to this deeply nested object you're passing to this function!" Thinking back on it, it's a really good example of "simple vs. easy". It didn't help it gave not a peep if you got one of those data structures wrong (capitalisation typo, wrong location, etc.)

Plus in hindsight the whole idea of "OS semantics in your browser!" was never a good one to start with, although that wasn't as obvious to me at the time.

Re: Shell script best practices, from a decade of scripting things

#155

Post by nousermane »

Ultimatt wrote: post #149
nousermane wrote: post #146

About that "#!/usr/bin/env bash" business - are there any systems out there that have "/usr/bin/env", but do not have "/bin/bash"?

That you've asked this question means you don't understand the actual reason to do this. I might have my own bash in my home I use to run all my shell scripts, why are you ignoring my environment and going for the system shell? Unless you control the system or are writing a system script this is absolutely unexpected and bad behaviour. On macOS now that zsh is the main supported shell plenty of people run a modern Ba…

> plenty of people run a modern Bash out of their home

Ah, got it. Another failure mode. There is a /bin/bash, but it's an ancient, crummy thing, that is difficult to upgrade. MacOSX does this, so users paper this over by installing a private copy as ~/bin/bash. Thank you.

Re: Shell script best practices, from a decade of scripting things

#156

Post by sharken »

orwin wrote: post #125
HyperSane wrote: post #113

Earlier quoted context omitted.

sh and bash feel pretty primitive after learning PowerShell.

I feel like powershell hides too much to be used regularly. I have a dozen of small shellscripts and aliases to do basically what PS help me to do when i work on windows (and some), but at least i know how it work behind. I had to work with Sencha/ExtJS early 2010. It was the same feeling. Yes, it is powerfull, but too much magic happen for something without a clear orientation (at the time, now it is used for data l…

I recently replaced a bit of code to look up locked files for a file share with SMB cmdlets to do the same.

The performance difference was night and day.

The biggest issue with PowerShell is that PowerShell Core is not yet default on Windows 10/11 and Windows Server. That should be Microsofts highest priority for PowerShell.

Re: Shell script best practices, from a decade of scripting things

#157

Post by erlkonig »

ndsipa_pomu wrote: post #124
belter wrote: post #106

Maybe the discussion should start at: Can you even do anything safely in Bash? - https://mywiki.wooledge.org/BashPitfalls

That's an excellent resource. Luckily, most commonly encountered scripting issues are with whitespace in filenames/variables and running a script through shellcheck will catch most (all?) of those problems. It's amazing how edge cases can make a simple command such as 'echo' break. (Top tip - use printf instead of echo)

echo has long been unreliable. Even the built-in echo in the shells were unreliable in SunOS, because the shell would look at the binaries in your PATH and try to figure out whether to emulate the BSD vs SysV (IIRC) version of echo and then change what echo would do. So much for writing a single script (with echo) that would work for all your users on the same host.

This is why you'll see code like this: echo 'prompt: ' | tr -d '\012'

No other simple mechanism was portable at the time. Seriously portability-minded coders still use that line, because although the issue is finally dead in linux+bash (i.e. /bin/echo is enough like bash's builtin) - it's likely still broken in other Unixen out there.

Re: Shell script best practices, from a decade of scripting things

#158

Post by Beltalowda »

IYasha wrote: post #91
Beltalowda wrote: post #19

> Use bash. Using zsh or fish or any other, will make it hard for others to understand / collaborate. Among all shells, bash strikes a good balance between portability and DX. I think fish is quite a bit different in terms of syntax and semantics (I'm not very familiar with it), but zsh is essentially the same as bash except without most of the needless footguns and awkwardness. zsh also has many more advanced featur…

As much as I love ZSH in my daily life, in sripting I HATE it for not having the "==" operator! >:((

It works inside [[ ]], just not in [ ].

=name will expand to the entry in your PATH. e.g. =ls expands to /usr/bin/ls. So == expands to an executable named =, or rather, it tries to as you probably don't have = in your PATH.

[[ ]] disables expansions (e.g. [[ * = * ]] will work too) so it's not an issue there.

Re: Shell script best practices, from a decade of scripting things

#159

Post by erlkonig »

grumbel wrote: post #18

What would be the justification for 'cd "$(dirname "$0")"'? Going to the scripts directory does not seem very helpful. If I don't care about the current directory, I might just go to '/' or a temporary directory, when I do care about it I better stay in it or interpreting relative command line arguments is going to get difficult. When symbolic links are involved, dirname will also give the wrong directory.

No, this is pretty meaningless unless sourcing in function.sh or something. If your script is short lived and doesn't have a work area (read/writing files), don't bother with cd. If it's long lived, the best default would be "cd /" or "cd /tmp" - sadly, since bash seem to mmap the script, this still doesn't free up the filesystem for unmounting. Python is different, and "cd /" is a good default for a long-lived program.

Re: Shell script best practices, from a decade of scripting things

#160

Post by erlkonig »

wolletd wrote: post #88
wruza wrote: post #53

Earlier quoted context omitted.

And for certain classes of users, certainly. If I were in $prj/some/dir and called “../../script.sh foo”, I’d expect it to operate on $prj/some/dir/foo, not on $prj/foo. The latter would be a confusing practice, not even remotely the best one. people will google for how to set cwd to the script directory more often The answer should suggest setting $script_dir instead of chdir and refer to it when needed, explaining…

I can only agree with this. In my experience, everybody coming from a lifetime with windows has to learn that a "working directory" has a very real and everyday meaning in Linux. Windows software just isn't designed that way because it usually has GUIs and "Open file" dialogs and doesn't use the cwd mostly. Most shortcuts even execute software in their installation directory (because it's most compatible with develop…

My theory is that this same Windows experience is also how so many joined the command-with-filename-extension cargo cult. But unix and windows work differently, and the approach does NOT port.
Post reply on HN