Marcel the Shell

That's really no different to the advice given for GNU coreutils:

    man grep | grep -- --recursive
           -r, --recursive

This looks a lot like an idea I am having for a shell written in GNU Guile. MI am still at the very beginning and POC phase. My ideas are:

(1) making it possible to have any procedure that follows a certain interface to be usable as a "command" in the shell

(2) have a fallback function, that gives you a procedure for a standard shell command like 'ls' (or reimplement your own!)

(3) have piped commands automatically use lightweight processes (guile-fibers) to automatically parallelize work, if possible

(4) pass structured data instead of strings, unless the structured data is actually a string or a fallback command of standard shell is used (although perhaps one could write functions parsing that string output into structured data and have structured data at the next step ...)

(5) provide composition concepts, that make it easy to compose commands or pipe their outputs into other commands

(6) have readable arguments for things like input or output redirection

(7) no need to be POSIX compatible, I just want a useful tool for myself and anyone, who might want to try it at some point. Don't want to get mired in arcane stuff.

Well, many things to consider, probably tons of work hiding somewhere under the hood of all this. I also noticed there are issues with parallelizing things, if I want to keep the shell state local to a command. Previous commands in a pipeline would have to communicate shell state updates down the pipeline. So one needs to distinguish between communication of a shell state update and more input for the next command. Concurrency issues might ensue. I think I will be glad, if I can get some subset of my goals to work.

This is a pretty brilliant idea. Ideally, someone from BSD-land and GNU-land should meet and come up with some sane JSON schemas.

Is that so? Because the way I see it, there are a bazillion different tools, plus decades of accrued knowledge only related to massaging data into text and back to structures. How many human brain cycles are wasted to data conversion on the Terminal?

Text is a lousy abstraction. It may have worked well 50 years ago when computing was limited to researchers on American universities, but the digital world back then has little in common with ours.

Acting like extending the shell to anything beyond plain text would require switching to fancy new serialisation formats every other year is a red herring. The only thing it takes is admitting a shell isn’t used on typewriters but modern computers, where humans use high-resolution screens and programs can exchange structured data. Everything else follows, and is well suited for standardisation.

But that will never happen as long as grumpy sysadmins insist that cobbling together grep and sed in some arcane incantation is the only thing we ever need.

Text does work, but text data exchange formats come in and out of vogue just like binary. And when any structure is needed, we typically get some ad-hoc text format that doesn't interoperate. JSON has made things better here, but it's a bit of a verbose eyesore for humans.

I think the lack a structured (text based) format is a definite shortcoming in UNIX. Whitespace delimited lists of strings do get you surprisingly far (although some things are a total mess, e.g. filenames with spaces). But a simple more structured format (e.g. stripped down YAML with maybe non-prefixed lines being a list/iterator of strings for backward compatibility) would make the shell a lot more powerful.

You should be able to include binary in these, but make it render nicely in the terminal (e.g. like browser devtools do).

This doesn't also exclude having binary. And binary data is piped though programs all the time, but it makes including other data painful. Also don't see any harm in being able to show images in the terminal when wanted.