Live data from Hacker News

Hush, a modern shell scripting language

hush-shell.github.io

161–170 of 192 posts

Re: Hush, a modern shell scripting language

#161

Earlier quoted context omitted.

> We need better shells. Obviously. I don't think this is the most important missing part, though. I would say it differently: we need way, way better REPLs. IPython is an example of a REPL that's passable as a shell. It can run in a terminal and has a GUI version based on Qt, which allows displaying images inline. You can drop into a "real" shell with a single `!` character (you get pipes, output capture, and (Pytho…

The REPL is the shell. I even used the term "REPL" (and a couple of synonyms too) in my comment. So I do agree it's critical but that doesn't make the language irrelevant. Your point about how Python shells have had to create syntactic sugar for REPl usage is a good illustration of my point about how it matters a lot. Also you can render images in quite a few terminal emulators already. Some shells (mine included) sh…

> Also you can render images in quite a few terminal emulators already. Some shells (mine included) ship with hooks to autodetect which terminal emulator you're using and find the best method for rendering those images.

Same word, different concepts. In Smalltalk an 'image' is a memory dump of the VM state, suitable for reloading and resuming, and typically includes a complete development environment:

https://wiki.c2.com/?ImageBasedLanguage

Re: Hush, a modern shell scripting language

#162

Earlier quoted context omitted.

> In UNIX, however, it is usually a mess and data extraction using the PowerShell approach would almost never work due spurious characters appearing in the output of a command Where would they come from? > (for any reason, really) as the UNIX style is precisely this: «cobble things together, use a sledgehammer to make everything work and move on. If it ain't broken then don't fix it». This is why running the output t…

> Where would they come from? Logging components being the worst offenders immediately spring to me. Especially the ones that receive data points over a network in heterogenous environments. syslog running on a flavour ABC of UNIX receives an input from a locally running app that has a buffer overrun, the app has previously accepted a longer than permitted input and dumped the actual log entry + all trailing the garb…

> The log parser is now screwed. I can think of similar examples outside log parsers, too, such interoperability related between different system

Use a sane system, like journald that can dump logs in JSON and doesn't require you to parse dates by hand. It can also deal with binary content fine, and can store stuff like crash dumps in the log if you want to, and provides functionality to make log parsing easy.

In any case I don't think such a problem should happen in the Powershell model. If you have a log object, then your $log.Message can contain any arbitrary garbage you want, and so you can just go and put that in a database and have that work with no trouble.

> This is not guaranteed.

Of course, but there are better and worse ways of doing things. With the object way, you can extend stuff quite easily with a minimum of care. As opposed to the unix model where you have to think whether somebody, somewhere might be using cut on this and fixing a typo in a descriptive text that adds a character might break something.

> Anything that is tightly coupled with precisely defines things is going to break, there is no scripting solution possible, I am afraid.

Not the best example I admit, but I mean in the case of something like Get-NetIPAddress in powershell, so long the user is either looking for the specific thing they want, or ignores stuff they don't recognize, you very much could add a yet new addressing scheme without running into trouble.

Good design helps a lot there. If you make it clear that stuff has types, and that new types may appear in the future, it's easy for the end user to write code that either ignores anything it doesn't understand or gives a good error message when it runs into it.

Re: Hush, a modern shell scripting language

#163

Earlier quoted context omitted.

The REPL is the shell. I even used the term "REPL" (and a couple of synonyms too) in my comment. So I do agree it's critical but that doesn't make the language irrelevant. Your point about how Python shells have had to create syntactic sugar for REPl usage is a good illustration of my point about how it matters a lot. Also you can render images in quite a few terminal emulators already. Some shells (mine included) sh…

> Your point about how Python shells have had to create syntactic sugar for REPL usage is a good illustration of my point about how it matters a lot. Of course it matters. The fact that Python is hardly suited for one-liners means that, if you want to write one-liners in it, you'll have to first extend or change its syntax and stdlib, pouring time and effort into creating a DSL for the interactive use-case. Some lang…

> I would like to see a framework for creating rich REPLs that would be language agnostic, so that I could get a state of the art auto-completion dialog no matter which language I decided to make into a shell.

It's doable with existing tools. You have LSP to provide the syntactical framework and there's no shortage of alternatives to readline (I'd written my own[1] to use in murex[2], and open sourced that).

[1] https://github.com/lmorg/readline

[2] https://murex.rocks

The problem you still face is that a good shell will offer autocompletion suggestions for strings that aren't language keywords or function names. eg

- file names; and there's a lot of hidden logic in how to do this. Do you build in fzf-like support, just include fzf wholesale but increase your dependency tree, or go for basic path completion. Do you check metadata (eg hidden files and system files on Windows), include dot-prefixed files on Linux / UNIX, etc. How do you know when to return paths, or paths and files, or even know not to return disk items at all? (see next point)

- flags for existing CLI tools (assuming you want compatibility with existing tools). Fish and murex will parse man pages to populate suggestions, others rely entirely on the community to write autocompletion scripts.

- Are you including variables in your completion of strings. And if so are you reading the variables to spot if it's a path and then following that path. eg `cd $HOME/[tab]` should then return items inside a your home directory even though you've not actually specified your home directory as a string. That means the shell needs to expand the variables to see if it's a valid path. But that's a shell decision rather than a language feature.

Some of these lists might take a while to populate so you then have another problem. Do you delay the autocompletion list (bad UX because it slows the user down) or provide the autocompletion sooner. And if the latter, how do you do that without:

1. changing the items under what you're about to select causing you to accidentally select the wrong option

2. communicate that there are update clearly

3. ensure the UI is consistent when slower loading entries might not fit the same dimensions as the space allocated for the list (if you dynamically size your completions to fit the screen real estate)

4. ensure that there's still something present while you're lazy loading the rest of the suggestions; and that those early entries on the completion list are worthwhile and accurate

5. what about sorting the list? Alphabetical? By feature? etc

The REPL in murex was inspired by IDEs so I've spent a lot of time trying to consider how to provide the best UX around autocompletion. One thing I've learnt is that it's a lot harder to get right than it seems on the surface.

Re: Hush, a modern shell scripting language

#164
post #153
post #147

Earlier quoted context omitted.

While being extremely small is a worthy goal, I suppose the aim of Hush is to make writing larger shell scripts easier and less error-prone. It's more for the niche of Perl of old than of minimal shells like ash. For a very limited device, a very limited shell like that in Busybox is sufficient, because it likely does not need large shell scripts, or a lot of interactive work. Looking at [1], current Hush is under 70…

The whole of Busybox is a megabyte. If Hush is 700k, then there are many, many places that it cannot go. As such, it cannot be a standard default shell. Use on smaller embedded platforms will not be possible. That is the trade.

zsh is also rather large, but it does not prevent zsh from being useful and popular. I'm not going to run it on my router, though.

OTOH I'm not going to run Busybox as the main shell on my laptop. Right tool for the job.

Re: Hush, a modern shell scripting language

#165

Earlier quoted context omitted.

The REPL is the shell. I even used the term "REPL" (and a couple of synonyms too) in my comment. So I do agree it's critical but that doesn't make the language irrelevant. Your point about how Python shells have had to create syntactic sugar for REPl usage is a good illustration of my point about how it matters a lot. Also you can render images in quite a few terminal emulators already. Some shells (mine included) sh…

> Also you can render images in quite a few terminal emulators already. Some shells (mine included) ship with hooks to autodetect which terminal emulator you're using and find the best method for rendering those images. Same word, different concepts. In Smalltalk an 'image' is a memory dump of the VM state, suitable for reloading and resuming, and typically includes a complete development environment: https://wiki.c2…

TBH I used the word "image" in both context: once as a synonym for picture, and the other time meaning Smalltalk image. I think GP was responding to the former use in the quoted text.

But, it's actually worth highlighting: GNU Smalltalk is image-based, but has NO GUI by default. So while the image does contain the state of your whole program at the point of dumping it, it does not include the whole IDE together with it. GNU Smalltalk is such a wonder, it's so sad it's not being actively developed :(

Re: Hush, a modern shell scripting language

#166
post #164
post #153

Earlier quoted context omitted.

The whole of Busybox is a megabyte. If Hush is 700k, then there are many, many places that it cannot go. As such, it cannot be a standard default shell. Use on smaller embedded platforms will not be possible. That is the trade.

zsh is also rather large, but it does not prevent zsh from being useful and popular. I'm not going to run it on my router, though. OTOH I'm not going to run Busybox as the main shell on my laptop. Right tool for the job.

The requirement, as set by POSIX, was to run the shell in Xenix on an 80286, which only supported a 64k text segment.

There are few tools that can meet this requirement, and the limit is as relevant now as the day it was first published. Any aspiring shell must run within this limit to fully pervade POSIX. Ignoring this requirement greatly reduces applicable scope.

Korn '88 could do it, but the code was not pretty.

Any POSIX shell replacement has to hit that target.

Re: Hush, a modern shell scripting language

#167
post #60

Earlier quoted context omitted.

Thirty lines? Unholy one liners are not limited to shell scripting: from subprocess import check_output sh = lambda script: check_output(script, shell=True, text=True) ips = set(line.split()[-1] for line in sh('last -a').splitlines() if line and 'tmux' not in line) The first two lines are pure overhead. The last line is the equivalent of a shell script one liner but now has all the advantages of a language that suppo…

> The first two lines are pure overhead. In 200 bytes half of which is overhead. That's not a great start. > The last line is the equivalent of a shell script one liner It's slower and requires much much more typing, and anything I want to add isn't going to go in the right place. It has a gross hack to support "tmux", and produces other spurious output (bugs). I don't want my X sessions or other ptys; I have reverse…

Your shell thing is honestly fine. The emoji thing was a joke, sorry that wasn’t clear.

The thing is with 1 IP lookup in N addresses a grep is also fine, but with M IP lookups the set is O(1) instead of O(N) and that makes a difference.

It’s not just theoretical. Bash slowness adds up fast and you will need to upgrade to a real language anyway.

Example: IPv6 addresses need to be truncated to /64 but last prints them with 0 truncation. Good luck doing that in bash. You’ll need a fully fledged IPv6 address parser, if it’s code that has to survive.

You make a fine point, honestly, about what you can get away with in shell scripting. The bigger picture is that literally nothing ever survives the sticky fingers if many engineers over time if it is a shell script. It gets worse and worse until it is ported, at which point it begins a new life as maintainable code.

Bash is a shell. Not a programming language.

Re: Hush, a modern shell scripting language

#168
post #154

Earlier quoted context omitted.

I want a shell language that wonderfully encompasses both interactive usage and the creation of easy, safe, effective shell scripts. I've come to the conclusion I can want that all I want, but it's not possible. Shell scripts and interactive usage have too many fundamentally opposing forces in them for one language to bridge the gap. You can be great at one or the other, and you can be bad at both, but you can't be g…

> but you can't be great at both My latest discovery - Raku - does try, and with not bad results. Also, Scala and Ammonite work quite well. Clojure and Babashka, too. > There simply isn't such a thing, even in theory. That's a very strong claim. What makes you think it's impossible to design such a language? The fact that none exists currently doesn't mean that it can't exist. > arguing seriously and with some merit…

"What makes you think it's impossible to design such a language?"

1. An interactive shell bases its error handling on the human being right there to handle the error. A programming languages bases its error handling on a human not being right there to handle it. This is a fundamental difference, and a language must privilege one or the other. Shell languages currently privilege the first, which is a non-trivial element of why they're dangerous to program in.

2. An interactive shell prioritizes keystrokes, because a human is right there in the moment tapping the keyboard trying to do something. A programming language won't necessarily ignore this, but it's definitely down the list of priorities.

3. An interactive shell bases its expectations of what state the user has in their head as being based on the live session state and the user's experience. A programming language bases its expectations of what state the user has based on the parameters to the functions and other language environment stuff. "cd" is really quite dangerous in shell scripts, especially once you start feeding it variables, but generally safe in interactive use because you see where you are (unless your PS1 is too bare, but you should fix that). Yes, obviously, we've all still messed that up at some point but I don't think it's necessarily above the baseline for any other human messup.

4. An interactive shell expects you to base your grounding of what's going to happen when you hit enter on the contents of the line when you hit enter. Again, yes, there are human messups, but that's the model. You don't wrap a whole bunch of defensive programming around every command line you type, you just use your eyes and your own understanding. A programming language does expect defensive programming. Defaults that make sense in one case are incredibly annoying or dangerous in the other. In my shell scripts, I want an empty (or perhaps "undefined") variable to be an error. In my interactive use, I may like a button to expand them all in place so I can see what I'm actually running but don't want it to necessarily be an error. And that's just one example of the sort of differences I'd expect. Another is string handling; there is a reason that almost all programming languages require all strings to be delimited via quotes (and the ones that don't generally consider it a mistake), and almost all shells do not. I wouldn't be surprised this criterion alone could be used as a razor to separate the categories with very high accuracy.

I suspect that were I to take the time to examine the 6 things you cite as things that are putatively both good scripting languages and good shell languages that I could tell you which category I put them in quite easily and why I wouldn't consider using them for the other thing, but that's rather more than I can take on for an HN post.

Re: Hush, a modern shell scripting language

#169
post #166
post #164

Earlier quoted context omitted.

zsh is also rather large, but it does not prevent zsh from being useful and popular. I'm not going to run it on my router, though. OTOH I'm not going to run Busybox as the main shell on my laptop. Right tool for the job.

The requirement, as set by POSIX, was to run the shell in Xenix on an 80286, which only supported a 64k text segment. There are few tools that can meet this requirement, and the limit is as relevant now as the day it was first published. Any aspiring shell must run within this limit to fully pervade POSIX. Ignoring this requirement greatly reduces applicable scope. Korn '88 could do it, but the code was not pretty. A…

> the limit is as relevant now as the day it was first published

Can't say if it's ironic or serious.

Re: Hush, a modern shell scripting language

#170
post #127

Earlier quoted context omitted.

My experience is different. For me, Python has a good balance of readability, access to system functions and consistent, predictable interfaces. I'm not saying you're wrong, just that "it depends". I'm pretty adequate at writing BASH, but it's not my "daily driver" programming language, so I have to still try things out, google a lot of things that aren't easily obvious, and excessive trial and error. But as a Java p…

You’re comparing Python and Bash as programming languages, not as shells. Python’s great as a programming language, but it’s a terrible shell. Likewise, Bash is a terrible programming language, but as a shell is pretty good and is probably the most used shell. The point parent was making is that it’s far more difficult in Python to run a process, capture the output, and grep something from it, than it is in Bash. In…

Sure, but the conversation had already evolved to "scripting languages" as opposed to command lines.
Post reply on HN