Live data from Hacker News

Techniques I use to create a great user experience for shell scripts

nochlin.com

211–220 of 281 posts

Re: Techniques I use to create a great user experience for shell scripts

#211

Earlier quoted context omitted.

Yes. And it is only downhill from there. Now, show us `mycommand | sed 's/ugly/beautiful/g' | awk -F: '{print $2,$4}' 1> something.report 2> err.log` in Python.

import os os.system('mycommand | sed 's/ugly/beautiful/g' | awk -F: '{print $2,$4}' 1> something.report 2> err.log')

You forgot to point out all those "footguns" you avoided by writing in Python rather than bash...

Re: Techniques I use to create a great user experience for shell scripts

#212

I ask LLMs to modify the shell script to strictly follow Google’s Bash scripting guidelines[^1]. It adds niceties like `set -euo pipefail`, uses `[[…]]` instead of `[…]` in conditionals, and fences all but numeric variables with curly braces. Works great. [^1]: https://google.github.io/styleguide/shellguide.html

Why would you change a shell (sh?) script into a Bash script? And why would you change [[ into [ expressions, which are not Posix, as far as I remember? And why make the distinction for numeric variablesand not simply make the usage the same, consistent for everything? Does it also leave away the double quotes there? That even sounds dangerous, since numeric variables can contain filenames with spaces. Somehow whenev…

>That even sounds dangerous, since numeric variables can contain filenames with spaces.

Or filenames that contain the number zero :D

    #!/bin/sh
    #
    # Usage : popc_unchecked BINARY_STRING
    #
    #   Count number of 1s in BINARY_STRING.  Made to demonstrate a use of IFS that
    #   can bite you if you do not quote all the variables you don't want to split.
    
    len="${#1}"
    count() { printf '%s\n' "$((len + 1 - $#))"; }
    saved="${IFS}"
    IFS=0
    count 1${1}1
    IFS="${saved}"
    
    # PS: we do not run the code in a subshell because popcount needs to be highly
    # performant (≖ ᴗ ≖ )

Re: Techniques I use to create a great user experience for shell scripts

#213

Earlier quoted context omitted.

Yes. And it is only downhill from there. Now, show us `mycommand | sed 's/ugly/beautiful/g' | awk -F: '{print $2,$4}' 1> something.report 2> err.log` in Python.

import os os.system('mycommand | sed 's/ugly/beautiful/g' | awk -F: '{print $2,$4}' 1> something.report 2> err.log')

This has all of the purported problems of doing this directly in a shell language and zero advantages...

Re: Techniques I use to create a great user experience for shell scripts

#214

Earlier quoted context omitted.

The GP is pointing out [bad bash, good bash] and not [good bash, bad bash]. It was unclear to me at first as well. You two are in violent agreement.

No, they're not. The script they're critiquing uses #!/bin/bash, so they have to have been saying that #!/usr/bin/env bash is better.

They're definitely both critiquing the script in the OP for the same thing in the same way. They're in agreement with each other, not with the script in TFA

Re: Techniques I use to create a great user experience for shell scripts

#215
post #214

Earlier quoted context omitted.

No, they're not. The script they're critiquing uses #!/bin/bash, so they have to have been saying that #!/usr/bin/env bash is better.

They're definitely both critiquing the script in the OP for the same thing in the same way. They're in agreement with each other , not with the script in TFA

Oh, I also got confused! You're right, this is just a confusing subthread.

Re: Techniques I use to create a great user experience for shell scripts

#216
post #35

Every time I see a “good” bash script it reminds me of how incredibly primitive every shell is other than PowerShell. Validating parameters - a built in declarative feature! E.g.: ValidateNotNullOrEmpty. Showing progress — also built in, and doesn’t pollute the output stream so you can process returned text AND see progress at the same time. (Write-Progress) Error handling — Try { } Catch { } Finally { } works just l…

I also hate bash scripting, and as far as Unix shell go, bash is among the best. So many footguns... Dealing with filenames with spaces is a pain, and files that start with a '-', "rm -rf" in a script is a disaster waiting to happen unless you triple check everything (empty strings, are you in the correct directory, etc...), globs that don't match anything, etc... But interactively, I much prefer Unix shells over Pow…

> Dealing with filenames with spaces is a pain, and files that start with a '-',

Wait! The fact that arguments with a leading hyphen are interpreted as options is not bash's fault. It's ingrained in the convention of UNIX tools and there's nothing bash can do to mitigate it. You would have the same problem if you got rid of any shell and directly invoked commands from Python or C.

Re: Techniques I use to create a great user experience for shell scripts

#217

Earlier quoted context omitted.

Yes. And it is only downhill from there. Now, show us `mycommand | sed 's/ugly/beautiful/g' | awk -F: '{print $2,$4}' 1> something.report 2> err.log` in Python.

That looks like a snippet from a command session which is a perfectly great place to be using sh syntax. If it became unwieldy you’d turn it into a script: #!/bin/sh beautify() { sed -e ‘ s/ugly/beautiful/g …other stuff ‘ } select() { awk ‘ {print $2, $4} …other stuff ‘ } mycommand | beautify | select For me, now it’s starting to look like it could be safer to do these things in a real language.

I have a lot of scripts that started as me automating/documenting a manual process I would have executed interactively. The script format is more amenable to putting up guardrails. A few even did get complex enough that I either rewrote them from the ground up or translated them to a different language.

For me, the "line in the sand" is not so much whether something is "safer" in a different language. I often find this to be a bit of a straw-man that stands in for skill issues - though I won't argue that shell does have a deceptively higher barrier to entry. For me, it is whether or not I find myself wanting to write a more robust test suite, since that might be easier to accomplish with Ginkgo or pytest or `#include `.

Re: Techniques I use to create a great user experience for shell scripts

#218

Earlier quoted context omitted.

Yes. And it is only downhill from there. Now, show us `mycommand | sed 's/ugly/beautiful/g' | awk -F: '{print $2,$4}' 1> something.report 2> err.log` in Python.

Just ask chatgpt and you’ll get a script, probably makes some tests too if you ask for it.

I have not really been a fan of ChatGPT quality. But even if that were not an issue, it is kinda hard to ask ChatGPT to write a script and a test suite for something that falls under export control and/or ITAR, or even just plain old commercial restrictions.

Re: Techniques I use to create a great user experience for shell scripts

#219

Earlier quoted context omitted.

People say that (and the same for Python), but I just don’t get it – and I’m a huge fan of Python. With shell, I can take the same tools I’ve already been using as one-liners while fiddling around, and reuse them. There is no syntax mapping to do in my head. With any other language, I have to map the steps, and probably also add various modules (likely within stdlib, but still). That’s not nothing. I’ve rewritten a s…

Completely agree. I have a rule that if I’m using arrays I should move to Python, PHP, etc. It’s a nice red flag. Arrays in Bash are terrible and a sign that things are getting more complicated.

I use the same heuristic for when I should switch from shell to Python :-). Arrays (especially associative ones, at least for me) are a good indication that a more advanced language like Python might be more appropriate than shell.

Re: Techniques I use to create a great user experience for shell scripts

#220

Earlier quoted context omitted.

You mean import os os.rename(“src.txt”, “dest.txt”) ?

Yes. And it is only downhill from there. Now, show us `mycommand | sed 's/ugly/beautiful/g' | awk -F: '{print $2,$4}' 1> something.report 2> err.log` in Python.

I don’t think it’s fair to compare a workflow that is designed for sed/awk. It’s about 10 lines of python to run my command and capture stdout/stderr - the benefit of which is that I can actually read it. What happens if you want to retry a line if it fails?
Post reply on HN