Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hush, a modern shell scripting language

hush-shell.github.io

Post reply on HN
Print view View original

91–100 of 192 posts

Re: Hush, a modern shell scripting language

#91

Post by ilyash »

Hi. Author of Next Generation Shell. I share the line of thought with Hush. But then we have arrived at different solutions. Small comparison here - https://www.reddit.com/r/ProgrammingLanguages/comments/ubwiz...

In my probably biased view, NGS is even more domain specific (handling of exit codes, syntax for running+parsing, etc) and more concise.

Re: Hush, a modern shell scripting language

#93

Post by jiggawatts »

dale_glass wrote: post #76

If this wants to sell itself as a shell scripting language, it should very quickly advertise what is it that makes it superior to say, bash for typical shell scripting tasks. Shell scripts with bash are painful to the point that if I find myself writing more than around 10 lines of shell, I tend to stop and switch to Perl instead. But Perl hasn't been too popular lately and isn't ideal either, so I'm very much up for…

Most of those features are in PowerShell, especially version 7.

It uses strict ('reliable') argument passing, strong typing, etc..

It has a bazillion independent streams such as "Debug", "Verbose", "Warning", "Error", "Progress", "Output", "Information", etc...

It has "foreach -parallel", which is a fun way to make the CPU fan spin up.

> "Recurse through this directory, while ignoring .git and vim backup files". Read this file into an array, splitting by newline, in a single line of code. It's tiresome to implement that kind of thing in every script I write. At the very least it should be simple and comfortable.

    $lines = dir -Recurse -Exclude *.git, *.vim | Get-Content
Not exactly rocket science in most shells? In most shells you can make this a modular function that takes in a list of file names, but PowerShell's "dir" doesn't output file names. It outputs file objects, so then it is just one step to do a filter on something like extension (or length, or whatever):

    dir | ? Length -lt 1MB | ? Extension -in @( '.cs', '.rs', '.cpp' )
In PowerShell you can create functions that integrate with the object-oriented pipeline. You can pipe full objects into a function, and objects out. Object attributes can be mapped automatically to function arguments too. You can have begin/process/end blocks so you can do things like open a DB connection, pipe in data, then close the connection neatly without needing to do this all manually each time. Literally something as trivial as this:

    Get-Stuff | Out-DatabaseTable -TableName 'stuff_log' -Server '...'
Seriously. Give it a go. It works on Linux too, and it's more unix than unix.

Re: Hush, a modern shell scripting language

#94

Post by paskozdilar »

dale_glass wrote: post #76

If this wants to sell itself as a shell scripting language, it should very quickly advertise what is it that makes it superior to say, bash for typical shell scripting tasks. Shell scripts with bash are painful to the point that if I find myself writing more than around 10 lines of shell, I tend to stop and switch to Perl instead. But Perl hasn't been too popular lately and isn't ideal either, so I'm very much up for…

Bash is definitely painful to write - I write shell scripts a lot and I enjoy it (in a masochistic kind of way), and I wholeheartedly agree with most of the criticisms you've written. However, the greatest strength of Bash is its ubiquity - it is available on almost every modern Unix-like environment - and when combined with standard POSIX tools it can provide a-little-less-painful environment for writing reliable shell scripts.

In order to prove it, I will do my best to provide some kind of solution to each of the problems you have mentioned. Note that they are in no way perfect solutions, and all of them are just hacks that work around Bash's inherent clumsiness.

---

> 100% reliable argument passing

Wrapping a variable in double-quotes does this:

    var="1 2 3 4"
    for x in "$var"; do echo $x; done
For double-parsing (e.g. SSH commands) you can wrap the entire expression in single-quotes to maintain the original string until the second shell:

    ssh user@host 'var="1 2 3 4"; for x in "$var"; do echo $x; done'
> 100% reliable file iteration

The "find" command does exactly this, and it works for whatever weird characters in the filename you have.

    find ./directory/ -type f -exec SOME_COMMAND {} \;
> No length limits. If I'm processing 10K files, I don't want to run into the problem that the command line is too long.

Do not evaluate arguments directly in shell, use xargs to feed the arguments as standard input. E.g.:

    find ./directory/ -type f | xargs SOME_COMMAND
> Excellent path parsing. Such as filename, basename, canonicalization, finding the file extension and "find the relative path between A and B".

    dirname /foo/bar
    basename /foo/bar
    realpath ./symlink/to/foo/bar
    basename /foo/bar | sed 's/.*\.//'
    realpath --relative-to ./dirA/fileA ./dirB/fileB
> Good error handling and reporting

This is a tough one. Good error handling/reporting in Bash is extremely hard and requires discipline, but it's possible.

To exit immediately as soon as a command exits with non-zero exitcode:

    set -e
However, this does not play nice with temporary resources, so I personally prefer hand-written error checks:

    SOME_COMMAND || { echo "Some Command has failed" && exit 1; }
Error reporting depends on a scenario, but I generally have a pattern of doing checks like:

    if [ "$var" != "expected" ]
    then echo "Error: var different from expected" && exit 1
    fi
And it works fine most of the time.

> Easy capture of stdout and stderr, at the same time. Either together or individually, as needed.

To capture both stdout and stderr in a same stream:

    SOME_COMMAND |& SOME_FILTER
Capturing stdout and stderr in different streams is tough. The easiest way I can think of is to use a pipe and job control mechanisms:

    mkfifo pipe
    SOME_COMMAND 2>pipe | FILTER_1 &
* Excellent process management. We're in 2022, FFS. We have 128 core CPUs. A modern shell scripting language should make it trivial to do something like: take these 50000 files, and feed them all through imagemagick, using every core available, while being able to report progress, record each failure, and abort the entire thing if needed.

While bash is truly terrible when it comes to concurrency, I find GNU Parallel to be pretty satisfying for most concurrent shell-scripting:

    cat FILE.txt | parallel --timeout=30 SOME_COMMAND {} \|\| echo "Some Command failed with argument: {}"
For POSIX purity, xargs also can be used with "-L 1" argument that parses a single line per command iteration. For parallelism, there is also a "--max-procs" argument.

* Excellent error reporting. I don't want things failing with "Command failed, aborted". I want things to fail with "Command 'git checkout https://....' exited with return code 3, and here's for good measure the stdout and stderr even if I redirected them somewhere".

I find that adding "set -x" to the top of your shell, which prints each command with "+" prefix as it is expanded and executed, very useful for error tracking.

* Give me helpers for common situations. Eg, "Recurse through this directory, while ignoring .git and vim backup files". Read this file into an array, splitting by newline, in a single line of code. It's tiresome to implement that kind of thing in every script I write. At the very least it should be simple and comfortable.

There are helpers out there for most common situations - it's just that they are implemented as CLI tools, and not officially part of the shell. And in the scenario where you can't expect the availability of those unofficial tools, you can always write your own small library of commonly used Bash functions, and just copy-paste them into your script whenever you need them. It's ugly, but still possible.

Re: Hush, a modern shell scripting language

#95

Post by IshKebab »

dahfizz wrote: post #13
chaorace wrote: post #4

Is this really a shell scriping language? Hush isn't an interactive shell, nor does it compile to a common shell script. The only way to run these scripts is to install the hush interpreter and run the script through it. Isn't that just a normal scripting language? What's the real benefit of using this over Node or Python? I suppose the syntax is more aesthetically similar to shell scripts... but I don't exactly see…

System level stuff sucks in Python. Dealing with files, I/O, permissions, etc is a real pain. It easily takes 5x as long and as many loc to do the same thing as in bash. I can see the benefit of dropping to a command block to, say, run a command and filter the output with some | grep | awk | sort of whatever, and then seamlessly come back up to a more fully featured language to deal with that data.

It might take a few more lines of code to do stuff in a real programming language like Python (I would recommend Deno actually) but at least there's a decent chance it will actually work reliably.

Re: Hush, a modern shell scripting language

#96

Post by tym0 »

declnz wrote: post #70
jordemort wrote: post #21

Earlier quoted context omitted.

Shellcheck helps a lot with this, I won't write shell without it anymore: https://www.shellcheck.net/

Shellcheck is truly a game-changer, even for people who've been hacking Bash (etc) for 20+ years. Side note: it's also one of (or the ?) most popular Haskell projects out there: https://github.com/koalaman/shellcheck/

Pandoc might be more popular but I agree that Shellcheck is an absolute godsend.

Re: Hush, a modern shell scripting language

#97

Post by dale_glass »

jiggawatts wrote: post #93
dale_glass wrote: post #76

If this wants to sell itself as a shell scripting language, it should very quickly advertise what is it that makes it superior to say, bash for typical shell scripting tasks. Shell scripts with bash are painful to the point that if I find myself writing more than around 10 lines of shell, I tend to stop and switch to Perl instead. But Perl hasn't been too popular lately and isn't ideal either, so I'm very much up for…

Most of those features are in PowerShell, especially version 7. It uses strict ('reliable') argument passing, strong typing, etc.. It has a bazillion independent streams such as "Debug", "Verbose", "Warning", "Error", "Progress", "Output", "Information", etc... It has "foreach -parallel", which is a fun way to make the CPU fan spin up. > "Recurse through this directory, while ignoring .git and vim backup files". Read…

PowerShell is indeed excellent, but there's nothing wrong with it having a bit of competition.

Some of it is also less than ideal, but rather less than bash. Issues in powershell I can think of:

* It leaks environment variables. Set an env var, and it propagates to the shell from which it was called from!

* It can't comfortably import environment from batch files. Working with vcvars is an annoyance. You'd think somebody at Microsoft would have made built in support for that one.

* Argument passing in Windows is a bloody horror. PowerShell accepts commands encoded in base64 because it sucks so much!

* For some reason you can't quote the command itself. You can do:

    git checkout "blah"
But for some reason can't do:

    "C:\Program Files\Git\bin\git.exe" checkout "blah".
Granted, that can be worked around with & or Start-Process, but what a pointless annoyance.

That said, I've not really tried it on Linux yet, got to give it a go.

Re: Hush, a modern shell scripting language

#98

Post by hnlmorg »

stormbrew wrote: post #69

> Traditional shell scripting languages are notoriously limited I feel like people looking to replace shells and shell languages need to really think deep and hard about this if it's something they believe. Shell scripts are really anything but limited, and in fact most replacements are more limited (either by design or by accident), often imposing awkward control flow on you or making things that should be simple mu…

I couldn't agree with this statement more. As someone who has developed their own shell over the last decade I've been keeping a close eye on what other people have been building too. So often I see people writing shells that are inspired by programming languages so their syntax looks amazing in documents. But they always strike me as being hugely tedious for repetitive and often quite dull tasks. The kind of 5 minute jobs that you drop into an interactive REPL to solve an immediate problem. And then maybe save as a shell script later if you find you're doing that task frequently.

People look at the problems with error handling and maintainability (which, as you highlighted yourself, are very real) but they end up throwing the baby out with the bath water when trying to solve those problems.

Ultimately it doesn't matter how well a shell is for scripting if it still creates more trouble as an interactive shell compared to (for example) Bash. I say that because we already have Python, Ruby, Perl, node.js, Jua, and a plethora of other languages that can be used for scripting. We don't need more scripting languages. We need better shells.

Re: Hush, a modern shell scripting language

#99

Post by paskozdilar »

jiggawatts wrote: post #93
dale_glass wrote: post #76

If this wants to sell itself as a shell scripting language, it should very quickly advertise what is it that makes it superior to say, bash for typical shell scripting tasks. Shell scripts with bash are painful to the point that if I find myself writing more than around 10 lines of shell, I tend to stop and switch to Perl instead. But Perl hasn't been too popular lately and isn't ideal either, so I'm very much up for…

Most of those features are in PowerShell, especially version 7. It uses strict ('reliable') argument passing, strong typing, etc.. It has a bazillion independent streams such as "Debug", "Verbose", "Warning", "Error", "Progress", "Output", "Information", etc... It has "foreach -parallel", which is a fun way to make the CPU fan spin up. > "Recurse through this directory, while ignoring .git and vim backup files". Read…

> it's more unix than unix.

Would you please elaborate what exactly do you mean by this?

Re: Hush, a modern shell scripting language

#100

Post by darrenf »

dale_glass wrote: post #76

If this wants to sell itself as a shell scripting language, it should very quickly advertise what is it that makes it superior to say, bash for typical shell scripting tasks. Shell scripts with bash are painful to the point that if I find myself writing more than around 10 lines of shell, I tend to stop and switch to Perl instead. But Perl hasn't been too popular lately and isn't ideal either, so I'm very much up for…

In my (biased) opinion Perl comes close to ideal with judicious use of Path::Tiny and Capture::Tiny.
Post reply on HN