Live data from Hacker News

The Beauty of Unix Pipelines

prithu.xyz

61–70 of 388 posts

Re: The Beauty of Unix Pipelines

#61

Earlier quoted context omitted.

> (1) everything is text And lists are space-separated. Unless you want them to be newline-separated, or NUL-separated, which is controlled by an option that may or may not be present for the command you're invoking, and is spelled completely differently for each program. Or maybe you just quote spaces somehow, and good luck figuring out who is responsible for inserting quotes and who is responsible for removing them…

God help you if your paths have spaces in them.

Just quote paths? shrug

Re: The Beauty of Unix Pipelines

#62
post #4

I use pipelines as much as the next guy but every time I see post praise how awesome they are, I'm reminded of the Unix Hater's Handbook. Their take on pipelines is pretty spot on too. http://web.mit.edu/~simsong/www/ugh.pdf

Hum... Those other abstractions on that section are great and everything, but it really sucks to write them interactively. Pipelines also lack some concept similar to exceptions, but it would also suck to handle those interactively.

  set -o pipefail

Re: The Beauty of Unix Pipelines

#65
post #57
post #4

I use pipelines as much as the next guy but every time I see post praise how awesome they are, I'm reminded of the Unix Hater's Handbook. Their take on pipelines is pretty spot on too. http://web.mit.edu/~simsong/www/ugh.pdf

I think of the Unix Hater's Handbook as a kind of loving roast to Unix, that hackers of the time understood to be humorous (you know, how people complain about the tools they use every day, much like people would later complain endlessly about Windows) and which was widely misunderstood later to be a real scathing attack. It hasn't aged very well, either. "Even today, the X server turns fast computers into dumb termi…

No, I think it's mostly just as angry and bitter as it sounds, coming from people who backed the wrong horse (ITS, Lisp Machines, the various things Xerox was utterly determined to kill off... ) and got extremely pissy about Unix being the last OS concept standing outside of, like, VMS or IBM mainframe crap, neither of which they'd see as improvements.

Re: The Beauty of Unix Pipelines

#66
If you eval this simple pipeline

     file -b `echo $PATH:|sed 's/:/\/* /g'`|cut -d\  -f-2|sort|uniq -c|sort -n
it prints a histogram of the types of all the programs on your path (e.g., whether they are shell, python, perl scripts or executable binaries). How can you ever write such a cute thing in e.g., python or, god forbid, java?

Re: The Beauty of Unix Pipelines

#67
post #27

Pipes are wonderful! In my opinion you can’t extol them by themselves. One has to bask in a fuller set of features that are so much greater than the sum of their parts, to feel the warmth of Unix: (1) everything is text (2) everything (ish) is a file (3) including pipes and fds (4) every piece of software is accessible as a file, invoked at the command line (5) ...with local arguments (6) ...and persistent globals in…

How’s it uncool? Just not “ mobile”?

My complaints with unix (as someone running linux on every device, starting to dip my toe into freebsd on a vps); apologies for lack of editing:

> everything is text

I'd often like to send something structured between processes without needing both sides to have to roll their own de/serialization of the domain types; in practice I end up using sockets + some thrown-together HTTP+JSON or TCP+JSON thing instead of pipes for any data that's not CSV-friendly

> everything (ish) is a file

> including pipes and fds

To my mind, this is much less elegant when most of these things don't support seeking, and fnctls have to exist.

It'd be nicer if there were something to declare interfaces like Varlink [0, 1], and a shell that allowed composing pipelines out of them nicely.

> every piece of software is accessible as a file, invoked at the command line

> ...with local arguments

> ...and persistent global in the environment

Sure, mostly fine; serialization still a wart for arguments + env vars, but one I run into much less

> and common signalling facility

As in Unix signals? tbh those seem ugly too; sigwinch, sighup, etc ought to be connected to stdin in some way; it'd be nice if there were a more general way to send arbitrary data to processes as an event

> also nice if some files have magic properties like /dev/random or /proc or /dev/null

userspace programs can't really extend these though, unless they expose FUSE filesystems, which sounds terrible and nobody does

also, this results in things like [2]...

> every program starts with 3 streams, stdin/stdout for work and stderr for out of band errors

iow, things get trickier once I need more than one input one one output. :)

I also prefer something like syslog over stderr, but again this is an argument for more structured things.

[0]: https://varlink.org/

[1]: ideally with sum type support though, and maybe full dependent types like https://dhall-lang.org/

[2]: https://www.phoronix.com/scan.php?page=news_item&px=UEFI-rm-...

Re: The Beauty of Unix Pipelines

#68

Unix pipelines are indeed beautiful, especially when you consider its similarity to Haskell's monadic I/O: http://okmij.org/ftp/Computation/monadic-shell.html Unix pipelines actually helped me make sense of Haskell's monad.

Could you be more specific? I don't get it.

In Unix:

    a; b    # Execute command b after a. The result of a is not used by b.
In Haskell:

    a >> b  # Run function b after a. The result of a is not used by b.
In Unix:

    a | b    # Execute command b after a. The result of a is used by b.
In Haskell:

    a >>= b  # Run function b after a. The result of a is used by b.

Re: The Beauty of Unix Pipelines

#69

Earlier quoted context omitted.

> (1) everything is text And lists are space-separated. Unless you want them to be newline-separated, or NUL-separated, which is controlled by an option that may or may not be present for the command you're invoking, and is spelled completely differently for each program. Or maybe you just quote spaces somehow, and good luck figuring out who is responsible for inserting quotes and who is responsible for removing them…

To criticize sh semantics without acknowledging that C was always there when you needed something serious is a bit short sighted. There are two uses of the Unix “api”: [A] Long lived tools for other people to use. [B] Short lived tools one throws together oneself. The fact that most things work most of the time is why the shell works so well for B, and why it is indeed a poor choice for the sort of stable tools desig…

There's a certain irony in responding to criticism of that which you're extolling by saying not to use it.

And the only reason I might be pushed down that path is because the task I'm working on happens to involve filenames with spaces in them (without those spaces, the code would work fine!), because spaces are a reasonable thing to put in a filename unless you're on a Unix system.

Post reply on HN