Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cicada – Unix shell written in Rust

github.com

Post reply on HN
Print view View original

81–90 of 168 posts

Re: Cicada – Unix shell written in Rust

#81

Post by majewsky »

jstimpfle wrote: post #73
yorwba wrote: post #62

Earlier quoted context omitted.

If the binary is using one of a few common libraries, e.g. GNU getopt, to read its command-line parameters, it should be possible to extract somewhat useful completions automatically. Then you are essentially relying on the de-facto standard of not rolling your own argument parsing, as opposed to some product of a standards committee.

Getopt is just barely flexible enough to be applicable to many projects while still encoding some conventions (dash-prefixed options and optional arguments). Still it's too opinionated to be applicable to all programs. And it's by far not structured enough to support automated completions. Every programmer goes through this phase of trying to abstract what can't be abstracted. Essentially for non-trivial programs you…

> By the way, even handcoded completions are, often enough, more confusing than helpful to me (and not because they are handcoded). I've disabled them entirely.

Huh? Can you give some examples? I'm on zsh, and except for broken auto-generated shell completions like that provided by ripgrep, I've found zsh completions to be incredibly competent. Much more competent than myself usually.

Re: Cicada – Unix shell written in Rust

#82

Post by zokier »

brandur wrote: post #3

Amazing work! Between this, Alacritty [1], and coreutils [2], we're getting pretty close to a plausible all-Rust CLI stack. While Cicada is pretty clearly modeled on the "old" generation of shells (sh, Bash, Zsh, etc.), one has to wonder what a more modern shell might look to address some of the problems of its predecessors like pipes that are essentially text-only, poor composability (see `cut` or `xargs`), and terr…

What sort of features do you feel that shell needs to support structured data? I've toyed with this idea, and in my view the shell itself has relatively minor role here. What is needed are utilities that produce and process structured data, and a terminal that can display it. But shell, it just glues the pieces together and doesn't really need to be aware of the data and the structure of it.

Re: Cicada – Unix shell written in Rust

#83

Post by sbmassey »

IshKebab wrote: post #63
ue_ wrote: post #60

Earlier quoted context omitted.

Why shouldn't it? The shell is arguably the most convenient way to interact with your system's programs, seconded only by something like Perl. That level of interaction ought to mean you can write functions such as for common tasks. Or should I now have to use Perl for that?

Because fragile shell scripts have been an endless source of security vulnerabilities and bugs. I would probably use Python, Go or Powershell.

You can use functions outside of shell scripts - they can serve as a better kind of alias, or as a way of modifying the current shell without having to type the '.' before the script.

Re: Cicada – Unix shell written in Rust

#84

Post by brandur »

zokier wrote: post #82
brandur wrote: post #3

Amazing work! Between this, Alacritty [1], and coreutils [2], we're getting pretty close to a plausible all-Rust CLI stack. While Cicada is pretty clearly modeled on the "old" generation of shells (sh, Bash, Zsh, etc.), one has to wonder what a more modern shell might look to address some of the problems of its predecessors like pipes that are essentially text-only, poor composability (see `cut` or `xargs`), and terr…

What sort of features do you feel that shell needs to support structured data? I've toyed with this idea, and in my view the shell itself has relatively minor role here. What is needed are utilities that produce and process structured data, and a terminal that can display it. But shell, it just glues the pieces together and doesn't really need to be aware of the data and the structure of it.

Here's a very simple example that I'd consider the holy grail. Say I want run `ls -lh` on a directory:

    -rw-r--r--   1 brandur  staff   6.5K Mar 25 15:37 Makefile
    -rw-r--r--   1 brandur  staff    63B Jul 10  2016 Procfile
    -rw-r--r--   1 brandur  staff   1.6K Mar 11 07:20 README.md
    drwxr-xr-x   4 brandur  staff   136B Oct 16  2016 assets/
    drwxr-xr-x   4 brandur  staff   136B Jul 15  2016 atom/
    drwxr-xr-x   4 brandur  staff   136B Apr 26  2016 cmd/
I'm interested in getting the date fields out of this structure. Currently that's possible with some disgusting use of `awk` or maybe `cut`.

What if I could do something like `ls -lh | select 6` (select the 6th column of data):

    Mar 25 15:37
    Jul 10  2016
    Mar 11 07:20
    Oct 16  2016
    Jul 15  2016
    Apr 26  2016
It doesn't matter that the date comes in three parts and would act as three separate tokens for `awk` and `cut` because in my structured object backend, it's all just one logical timestamp.

Now say I want to sort these dates and pick the earliest. Once again, shells make this extremely difficult. Because the dates are not lexigraphically orderable, I'd have to use some fancy parsing, or try to step back to `ls` and have it somehow order by date.

Imagine if I could instead to something like `ls -lh | select 6 | sort | first`:

    Apr 26  2016
In this case, although the shell is still displaying a string to me that's not lexigraphically orderable, the data is being passed around in the backend in a structured way and these timestamps are actually real timestamps. It's then trivial for the next utility to apply ordering.

The part that the shell would have to provide (as opposed to the programs) is a "smart pipe" that understands more than just a basic text stream. It would also have to know when there is no pipe connected, and print as pretty strings when it knows the output is going to `STDOUT`.

Re: Cicada – Unix shell written in Rust

#85

Post by sirn »

gkya wrote: post #74
sirn wrote: post #22

Earlier quoted context omitted.

This matches my experience trying to switch the login shell from bash to fish: a surprisingly amount of things failed. macOS users won't have this problem, as the login shell is only started when Terminal/iTerm is started, but in Linux pretty much every process rely on the fact that your $SHELL is POSIX-complaint. It seems like the way to go to use non-POSIX shell as the default shell without changing login shell, is…

How come? Having your user's shell sth. non-compliant should not affect other scripts, as they are supposed to specify what interpreter they wanna use via shebang lines; and as long as programs are run with the correct variant of the exec* functions they should be fine.

I meant the login shell, the one that get executed during console login (i.e. the "-bash" process).

The login shell need to source /etc/profile and /etc/profile.d during logins to populate environment variables. Some apps that expect those environment variable to be present wrote to /etc/profile.d with an expectation that a POSIX-compatible shell and will read the directory (by source-ing those files). The environment variable getting set there ranging from PATH, LANG, to application/distro-specific like MOZ_PLUGIN_PATH.

So when you changed the login shell to something that no longer reads /etc/profile.d (like fish, which actually say it's "interactive shell" in the man page) then application that rely on those variables will start to behave in some funny way, which was the point I was trying to make.

Re: Cicada – Unix shell written in Rust

#86

Post by jstimpfle »

majewsky wrote: post #81
jstimpfle wrote: post #73

Earlier quoted context omitted.

Getopt is just barely flexible enough to be applicable to many projects while still encoding some conventions (dash-prefixed options and optional arguments). Still it's too opinionated to be applicable to all programs. And it's by far not structured enough to support automated completions. Every programmer goes through this phase of trying to abstract what can't be abstracted. Essentially for non-trivial programs you…

> By the way, even handcoded completions are, often enough, more confusing than helpful to me (and not because they are handcoded). I've disabled them entirely. Huh? Can you give some examples? I'm on zsh, and except for broken auto-generated shell completions like that provided by ripgrep, I've found zsh completions to be incredibly competent. Much more competent than myself usually.

I only know bash, but I doubt zsh can be so much better (yes, I know, they have colorful suggestions and better interactivity for selection).

The context sensitivity is just very confusing. Sometimes completion hangs, maybe because an SSH connection is made in the background, or the completion is just not very efficient.

Then sometimes I have a typo and get the weirdest suggestions and it's much harder to track down the error statically than just running the program and get a nice error message.

Then sometimes I am more intimate with the complexities of the command-line invocation than the completion script can appreciate, and get no completion, when all I need is just a stupid local file completion. This results in the (untrue) assumption a file wasn't there, which easily leads to further confusion.

No, thanks. I know how to use the tools that I use. Dear completion, just help me with the files on my disk (which is the fast-changing variable in the game) and let me type commands as I see fit.

Re: Cicada – Unix shell written in Rust

#87

Post by lihaoyi »

brandur wrote: post #84
zokier wrote: post #82

Earlier quoted context omitted.

What sort of features do you feel that shell needs to support structured data? I've toyed with this idea, and in my view the shell itself has relatively minor role here. What is needed are utilities that produce and process structured data, and a terminal that can display it. But shell, it just glues the pieces together and doesn't really need to be aware of the data and the structure of it.

Here's a very simple example that I'd consider the holy grail. Say I want run `ls -lh` on a directory: -rw-r--r-- 1 brandur staff 6.5K Mar 25 15:37 Makefile -rw-r--r-- 1 brandur staff 63B Jul 10 2016 Procfile -rw-r--r-- 1 brandur staff 1.6K Mar 11 07:20 README.md drwxr-xr-x 4 brandur staff 136B Oct 16 2016 assets/ drwxr-xr-x 4 brandur staff 136B Jul 15 2016 atom/ drwxr-xr-x 4 brandur staff 136B Apr 26 2016 cmd/ I'm i…

Not quite a "Unix" shell, but the [Ammonite Scala Shell](http://ammonite.io/#Ammonite-Shell) lets you do this trivially:

    lihaoyi Ammonite$ amm
    @ import ammonite.ops._
    import ammonite.ops._
    @ ls! pwd
    res1: LsSeq =
    ".git"              'LICENSE            'ci                 'project            'sshd
    ".gitignore"        'amm                'integration        'readme             'target
    ".idea"             "appveyor.yml"      "internals-docs"    "readme.md"         'terminal
    ".travis.yml"       "build.sbt"         'ops                'shell
    
    @ ls! pwd | (_.mtime)
    res2: Seq[java.nio.file.attribute.FileTime] = List(
      2017-06-21T12:24:41Z,
      2017-06-11T13:48:45Z,
      2017-06-21T12:29:05Z,
      2017-06-11T13:48:45Z,
      2017-05-01T03:41:14Z,
      2017-06-18T14:06:33Z,
      2017-06-11T13:48:45Z,
      2017-06-18T08:01:20Z,
      2017-06-18T12:34:00Z,
      2017-06-19T06:05:51Z,
      2017-06-18T09:06:07Z,
      2017-06-18T14:06:33Z,
      2017-06-18T14:06:15Z,
      2017-06-18T14:07:20Z,
      2017-06-11T13:48:45Z,
      2017-06-18T14:06:33Z,
      2017-06-18T14:06:33Z,
      2017-06-19T06:06:03Z,
      2017-06-18T14:06:33Z
    )
    @ ls! pwd | (_.mtime) sorted
    res3: Seq[java.nio.file.attribute.FileTime] = List(
      2017-05-01T03:41:14Z,
      2017-06-11T13:48:45Z,
      2017-06-11T13:48:45Z,
      2017-06-11T13:48:45Z,
      2017-06-11T13:48:45Z,
      2017-06-18T08:01:20Z,
      2017-06-18T09:06:07Z,
      2017-06-18T12:34:00Z,
      2017-06-18T14:06:15Z,
      2017-06-18T14:06:33Z,
      2017-06-18T14:06:33Z,
      2017-06-18T14:06:33Z,
      2017-06-18T14:06:33Z,
      2017-06-18T14:06:33Z,
      2017-06-18T14:07:20Z,
      2017-06-19T06:05:51Z,
      2017-06-19T06:06:03Z,
      2017-06-21T12:24:41Z,
      2017-06-21T12:29:05Z
    )
    @ ls! pwd | (_.mtime) min
    res4: java.nio.file.attribute.FileTime = 2017-05-01T03:41:14Z
    @ ls! pwd | (_.mtime) max
    res5: java.nio.file.attribute.FileTime = 2017-06-21T12:29:05Z

    @ ls! pwd maxBy (_.mtime)
    res6: Path = root/'Users/'lihaoyi/'Dropbox/'Github/'Ammonite/".idea"
Ammonite certainly has problems around JVM memory usage and startup times, but it has a sweet spot for this sort of not-quite-trivial filesystem operations that are a pain to do in Bash but too small to be worth futzing with a Python script

Re: Cicada – Unix shell written in Rust

#88

Post by naasking »

brandur wrote: post #3

Amazing work! Between this, Alacritty [1], and coreutils [2], we're getting pretty close to a plausible all-Rust CLI stack. While Cicada is pretty clearly modeled on the "old" generation of shells (sh, Bash, Zsh, etc.), one has to wonder what a more modern shell might look to address some of the problems of its predecessors like pipes that are essentially text-only, poor composability (see `cut` or `xargs`), and terr…

A modern shell with properly structured data streams would look like caml-shcaml:

http://users.eecs.northwestern.edu/~jesse/pubs/caml-shcaml/

PowerShell definitely made a lot of mistakes they should have known needed fixing. Like the need for signed scripts.

Re: Cicada – Unix shell written in Rust

#89

Post by arca_vorago »

I just have to say, while I respect the work and have been looking forward to nix tools being ported to rust, the first thing I do is see if it's GPL or not and if it's not it immediately loses points (not all of them), because I have been working hard to free up my stacks. I wish people would consider making core system tools like this GPL instead of MIT/BSD styles. Tivoization is a real threat to the freedom of users and devs. (which is what mit license enables, despite it being listed as "gpl compat")

For example, I forced myself to learn screen really well because even though I like tmux and it has some features lacking in screen, I would rather support a tool I know is more aligned with freedom for the user. Same with i3 vs awesome, etc.

An interesting side benefit of this is I have noticed the complexity of my stack has been reduced because of this selectiveness.

Re: Cicada – Unix shell written in Rust

#90

Post by subnixr »

codesnik wrote: post #39
vidarh wrote: post #23

Earlier quoted context omitted.

I lean towards wanting all my tools to support at least one common structured format (and JSON generally would seem the best supported) these days. It'd be great if there was a "standard" environment variable to toggle JSON input/output on for apps. If you did that, then e.g. a "structured" shell could just default to turning that on, and try detecting JSON on output and offering according additional functionality. T…

jq is awesome. "json lines" data format (where each record is json without literal newlines, and newline separates records) nicely upgrades unix pipes to structured output, so you can mix old unix tools, like head, tail etc with jq. maybe what's needed is some set of simple json wrappers around core utils to increase adoption.

> "json lines" data format (where each record is json without literal newlines, and newline separates records)

Good point, but jq handles this already. If the payload is an array of object, simply `jq -c '.[]'` to get an object per line

Post reply on HN