Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cicada – Unix shell written in Rust

github.com

Post reply on HN
Print view View original

41–50 of 168 posts

Re: Cicada – Unix shell written in Rust

#41

Post by vidarh »

vog wrote: post #24
vidarh wrote: post #23

Earlier quoted context omitted.

I lean towards wanting all my tools to support at least one common structured format (and JSON generally would seem the best supported) these days. It'd be great if there was a "standard" environment variable to toggle JSON input/output on for apps. If you did that, then e.g. a "structured" shell could just default to turning that on, and try detecting JSON on output and offering according additional functionality. T…

Nice idea! Maybe this issue is similar to that of command completion, but could also result in a similar mess. Every command has its "-h" or "--help", showing the syntax, available options and so on. But since even that is not really standardized, separate command-line completion rules are written for every command. And for every shell.

I still miss AmigaOS, where command options parsing was largely standardised ca. 1987 via ReadArgs() (barring the occasional poor/direct port from other OS's) [1]. It wasn't perfect (it only provided a very basic usage summary), but it was typed, and it was there.

[1] http://www.pjhutchison.org/tutorial/cmdline_arrgs.html

Re: Cicada – Unix shell written in Rust

#42

Post by vidarh »

ori_b wrote: post #32
vidarh wrote: post #23

Earlier quoted context omitted.

I lean towards wanting all my tools to support at least one common structured format (and JSON generally would seem the best supported) these days. It'd be great if there was a "standard" environment variable to toggle JSON input/output on for apps. If you did that, then e.g. a "structured" shell could just default to turning that on, and try detecting JSON on output and offering according additional functionality. T…

> It'd be great if there was a "standard" environment variable to toggle JSON input/output on for apps. Look at FreeBsd's libxo. It's supported by most of the base system.

That looks very interesting, though I actually think that standardising on the command-line/env variable API is more important than an implementation. It's the moment you can "automagically" "upgrade" a pipe to contain structured data for apps that support it that you get the most value.

But given that it's there, I'll certainly consider it when writing tools, and consider supporting the command-line option/env var even if/when I don't use the implementation...

Re: Cicada – Unix shell written in Rust

#43

Post by panic »

ori_b wrote: post #35
panic wrote: post #34

Earlier quoted context omitted.

If you use bytes that are invalid in UTF-8 (e.g. 0xF5-0xFF) as delimiters / structure characters, you can use text-only tools to do structured operations on UTF-8 strings without ever having to escape anything -- the structural "characters" you would need to escape can never appear in the encoded bytes.

The Unix shell operates on binary streams, not on text.

Ah, I thought this was about structured text (like JSON) vs. plain text -- binary content is a bigger issue!

Re: Cicada – Unix shell written in Rust

#45

Post by int_19h »

vog wrote: post #12
zdkl wrote: post #7

Earlier quoted context omitted.

My structured objects aren't your structured objects. In that use case why not use a dedicated program (like say ae python/js/clojure interpreter) to handle more complex pipes and keep the shell level primitves... Well, primitive. Text is compatible with all systems, past and present, and can be used to model more complex objects. Let's not add features to the core on a "why not" please.

The so-called "plain text formats" are also just representations of complex objects. Often these formats are ad-hoc and hard to parse correctly. Multiple text files in a file system hierarchy (e.g. /etc) are also just nested structures. So in principle, treating those as complex object structures is the right way to go. Also, I believe this is the idea behind D-Bus and similar modern Unix developments. However, what'…

Powershell does two things right: it uses structured output, and it separates producing the data from rendering the data.

It also does one thing wrong: it uses objects (i.e. the stuff that carries behavior, not just state). This ties it to a particular object model, and the framework that supports that model.

What's really needed is something simple that's data-centric, like JSON, but with a complete toolchain to define schemas and perform transformations, like XML (but without the warts and overengineering).

Re: Cicada – Unix shell written in Rust

#46

Post by drothlis »

ori_b wrote: post #32
vidarh wrote: post #23

Earlier quoted context omitted.

I lean towards wanting all my tools to support at least one common structured format (and JSON generally would seem the best supported) these days. It'd be great if there was a "standard" environment variable to toggle JSON input/output on for apps. If you did that, then e.g. a "structured" shell could just default to turning that on, and try detecting JSON on output and offering according additional functionality. T…

> It'd be great if there was a "standard" environment variable to toggle JSON input/output on for apps. Look at FreeBsd's libxo. It's supported by most of the base system.

It looks like libxo does the output in structured format, but what about input? (For chaining commands together with pipes). Did the FreeBsd porting work[1] implement the input side with libxo, independently of libxo, or not at all?

(Not to diminish libxo --it looks pretty cool, and I didn't know about it before-- just curious.)

[1]: https://wiki.freebsd.org/LibXo

Re: Cicada – Unix shell written in Rust

#47

Post by jstimpfle »

skissane wrote: post #27
vog wrote: post #24

Earlier quoted context omitted.

Nice idea! Maybe this issue is similar to that of command completion, but could also result in a similar mess. Every command has its "-h" or "--help", showing the syntax, available options and so on. But since even that is not really standardized, separate command-line completion rules are written for every command. And for every shell.

Define a standard format for describing command syntax. I'd suggest basing it on JSON, but you could use XML or Protobuf or Thrift or ASN.1 or Sexprs or whatever. Embed in executables a section containing the command syntax description object. Now when I type in a command, the shell finds the executable on the path, opens it, locates the command description syntax section (if present), and if found uses that to provi…

There isn't a 1:1 mapping of binaries to documentation or completion scripts. That idea won't work out.

"Defining a standard format" hasn't been a successful practice in the Unix world. Many standards consist essentially of the bare minimum that everybody can agree with.

Unix command-line interfaces are already structured (as a list of strings), and more structure would be hard to support at the binary level. There are too many possibilities of doing that, and most CLI programs wouldn't even need that.

Re: Cicada – Unix shell written in Rust

#48

Post by Klasiaster »

Escaping and passing strings as arguments (without losing e.g. " characters) is often a problem. It would be so nice if this would be solved in a shell the way it works for function calls in propper programming languages like Python or Rust (i.e. various ways to specify strings with special characters and a good format method for string composition) with Haskell-like function call syntax.

Re: Cicada – Unix shell written in Rust

#49

Post by laumars »

brandur wrote: post #3

Amazing work! Between this, Alacritty [1], and coreutils [2], we're getting pretty close to a plausible all-Rust CLI stack. While Cicada is pretty clearly modeled on the "old" generation of shells (sh, Bash, Zsh, etc.), one has to wonder what a more modern shell might look to address some of the problems of its predecessors like pipes that are essentially text-only, poor composability (see `cut` or `xargs`), and terr…

The approach I took to my shell was to pass JSON objects about as the primary data type but fall back to text streams when the data doesn't look like JSON.

The problem with passing objects around though is it can cause issues with the parallel nature of pipes. ie the programs in a chain of pipes can run concurrently since the data is being passed is just a stream. But with objects you need to read the object in its entirety to ensure it is a valid object. This means each process is waiting for the previous process to finish outputting it's object before the next can start processing it. While this is less of an issue with smaller chunks of data in a chain, it would become really noticeable really quickly as your data scales up.

I'm sure will be workarounds for the above problem but my project is still young so there are other bugs I'm more focused on (for me I'm more interested in getting an IDE-level auto-complete implemented so writing one liners in the Shell is as natural as writing your favourite language in your preferred development tool.

Re: Cicada – Unix shell written in Rust

#50

Post by IshKebab »

Klasiaster wrote: post #48

Escaping and passing strings as arguments (without losing e.g. " characters) is often a problem. It would be so nice if this would be solved in a shell the way it works for function calls in propper programming languages like Python or Rust (i.e. various ways to specify strings with special characters and a good format method for string composition) with Haskell-like function call syntax.

And lists, and maths, and error handling, and ...

To fix all those things you wouldn't end up with a traditional Unix shell, which I think would be a great project, but there are clearly two mutually exclusive projects to do:

1. A Unix-like shell (this)

2. A robust shell suitable for writing scripts

Post reply on HN