Hush, a modern shell scripting language

The weakness of the shell is that it did not emerge as an LR-parsed language that could easily be expressed with a concise yacc grammar. The "one true awk," for example, bundled a yacc grammar as part of the build until relatively recently. The complex grammar of the POSIX shell is ambiguous in the extreme in many situations.

The shell language was further constrained by POSIX that removed much Korn shell functionality, even though a public domain implementation of Korn existed. This was done to allow the shell parser to be very small, and Debian's Almquist shell compiles to under 100k on i386 (unlike bash, which trails it also in speed).

This new Hush shell looks a bit too wordy and reminiscent of javascript.

Anyone designing a new shell grammar should also design in such a way that Busybox could bundle this new interpreter, should popularity justify it.

attacking the problem of improving shell scripting while attempting to keep the syntax

(Oil author here) Slight correction, The project has two parts

(1) OSH (bin/osh) will run your existing scripts, so in that sense it does keep the syntax

(2) The Oil language (bin/oil) is an upgrade from OSH, but it can also be thought of as a clean slate language:

https://www.oilshell.org/release/latest/doc/oil-language-tou...

It's really the same interpreter with a few parse time and runtime options. There were surprisingly few compromises (probably the biggest one is the shell-like string literal syntax, which I mention.)

The biggest changes are that we take over ( ) and { } , so you can write stuff like:

    if (x > 0) {
      echo 'positive' | tac
    }

Rather than

    if test "$x" -gt 0; then
      echo 'positive' | tac
    fi

We add a Python-like expression language between () and surprisingly few things need to change to make it feel like a "new language".

Thanks for mentioning Oil :)

FWIW people are mentioning tons of alternative shells in this thread, and most of them are already listed here:

https://github.com/oilshell/oil/wiki/Alternative-Shells

(if not, feel free to add them)

Yes, exactly. I have a few slogans for this, but one is that the shell language should compose with the rest of the operating system, which is "processes and files". It's a situated language.

So I have been making the point that the "narrow waist" of shell is still processes and files. Whereas the narrow waist of Python and Lua is functions and compound data structures like dicts/lists (tables).

One thing that is not quite obvious until you design AND USE a shell language is that if you have these conflicting narrow waists, you have problems of composition, and you will offload this complexity onto your users.

From a very quick look at the hush docs, it is Lua with some shell-like stuff grafted on. I believe that will cause cause such problems in programs. They will be longer and harder to write. You will have more edge cases.

----

I hit this issue many times in Oil, and I wrote many posts tagged #software-architecture about it:

http://www.oilshell.org/blog/tags.html?tag=software-architec...

Unix Shell: Philosophy, Design, and FAQs http://www.oilshell.org/blog/2021/01/philosophy-design.html

makes several of these points.

This recent post

http://www.oilshell.org/blog/2022/03/backlog-arch.html

asks:

    - Should shells have two tiers?
      - Both external processes and internal "functions"? 
      - Both pipelines of bytes and pipelines of structured data?
    - Another one I'm working on now: Both exit codes and Python-like exceptions?

In Oil the answer is yes for the first 2, but a subtle point is that the former are PRIMARY and still the narrow waist; the latter are helpers. Dicts and Lists exist to help you deal with files (and produce JSON and TSV); but they're not the primary structures of the program.

I've also discussed this issue pretty extensively with the authors of the NGS and Elvish shells.

A related recent comment: https://old.reddit.com/r/ProgrammingLanguages/comments/ub5vi...

Having unsigned / signed and sync / async (what color is your function?) are similar problems of composition in language design. I call them "Perlis-Thompson" problems.

There is no easy answer but IMO it's vital to explicitly think about and address these problems. C has a flawed solution to the combinatorial explosion (implicit conversions), and it's caused problems and been lamented for ~50 years. There's also a good quote in that thread about why unsigned sizes in STL were a language design mistake.

Folks interested in this design space may enjoy reading about scsh (A Scheme Shell)

From the scsh paper (https://web.archive.org/web/20081010222846/http://www.scsh.n...):

> I have designed and implemented a Unix shell called scsh that is embedded inside Scheme. I had the following design goals and non-goals:

> The general systems architecture of Unix is cooperating computational agents that are realised as processes running in separate, protected address spaces, communicating via byte streams. The point of a shell language is to act as the glue to connect up these computational agents. That is the goal of scsh. I resisted the temptation to delve into other programming models. Perhaps cooperating lightweight threads communicating through shared memory is a better way to live, but it is not Unix. The goal here was not to come up with a better systems architecture, but simply to provide a better way to drive Unix. {Note Agenda}

> I wanted a programming language, not a command language, and I was unwilling to compromise the quality of the programming language to make it a better command language. I was not trying to replace use of the shell as an interactive command language. I was trying to provide a better alternative for writing shell scripts. So I did not focus on issues that might be important for a command language, such as job control, command history, or command-line editing. There are no write-only notational conveniences. I made no effort to hide the base Scheme syntax, even though an interactive user might find all the necessary parentheses irritating. (However, see section 12.)

> I wanted the result to fit naturally within Scheme. For example, this ruled out complex non-standard control-flow paradigms, such as awk's or sed's.