Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Beauty of Unix Pipelines

prithu.xyz

Post reply on HN
Print view View original

101–110 of 388 posts

Re: The Beauty of Unix Pipelines

#101

Post by zamalek »

cowmix wrote: post #4

I use pipelines as much as the next guy but every time I see post praise how awesome they are, I'm reminded of the Unix Hater's Handbook. Their take on pipelines is pretty spot on too. http://web.mit.edu/~simsong/www/ugh.pdf

I am firmly in the "ugh" camp. I strongly suspect that the fawning that occurs over pipelines is because of sentimentality more than practicality. Extracting information from text using a regular expression is fragile. Writing fragile code brings me absolutely no joy as a programmer - unless I am trying to flex my regex skills.

If you really look at how pipelines are typically used is: lines are analogous to objects and [whatever the executable happens to use] delimiters are analogous to fields. Piping bare objects ("plain old shell objects") makes far more sense, involves far less typing and is far less fragile.

Re: The Beauty of Unix Pipelines

#102

Post by [deleted] »

gorgoiler wrote: post #21

Pipes are wonderful! In my opinion you can’t extol them by themselves. One has to bask in a fuller set of features that are so much greater than the sum of their parts, to feel the warmth of Unix: (1) everything is text (2) everything (ish) is a file (3) including pipes and fds (4) every piece of software is accessible as a file, invoked at the command line (5) ...with local arguments (6) ...and persistent globals in…

[deleted]

Re: The Beauty of Unix Pipelines

#103

Post by 1MachineElf »

nerdponx wrote: post #60
gorgoiler wrote: post #47

Earlier quoted context omitted.

To criticize sh semantics without acknowledging that C was always there when you needed something serious is a bit short sighted. There are two uses of the Unix “api”: [A] Long lived tools for other people to use. [B] Short lived tools one throws together oneself. The fact that most things work most of the time is why the shell works so well for B, and why it is indeed a poor choice for the sort of stable tools desig…

C was always there when you needed something serious There is a world of stuff in between "I need relatively low-level memory management" and "I need a script to just glue some shit together". For that we have Python and Perl and Ruby and Go, or even Rust.

Going with the same theme, C itself was an innovation meant to fill the space between Assembler (in this context, "A") and B[0].

[0] https://en.wikipedia.org/wiki/B_(programming_language)

Re: The Beauty of Unix Pipelines

#104

Post by ryandrake »

gorgoiler wrote: post #21

Pipes are wonderful! In my opinion you can’t extol them by themselves. One has to bask in a fuller set of features that are so much greater than the sum of their parts, to feel the warmth of Unix: (1) everything is text (2) everything (ish) is a file (3) including pipes and fds (4) every piece of software is accessible as a file, invoked at the command line (5) ...with local arguments (6) ...and persistent globals in…

> (2) everything (ish) is a file

I'm not all that knowledgable about Unix history, but one thing that has always puzzled me was that for whatever reason network connections (generally) aren't files. While I can do:

  cat
to read from a serial device, I've always wondered why I can't do something like:

  bind /tmp/mysocket 12.34.56.78 80
  cat
It is weird that so many things in Unix are somehow twisted into files (/dev/fb0??), but network connections, to my knowledge, never managed to go that way. I know we have netcat but it's not the same.

Re: The Beauty of Unix Pipelines

#105

Post by viraptor »

gorgoiler wrote: post #47
jcranmer wrote: post #38

Earlier quoted context omitted.

> (1) everything is text And lists are space-separated. Unless you want them to be newline-separated, or NUL-separated, which is controlled by an option that may or may not be present for the command you're invoking, and is spelled completely differently for each program. Or maybe you just quote spaces somehow, and good luck figuring out who is responsible for inserting quotes and who is responsible for removing them…

To criticize sh semantics without acknowledging that C was always there when you needed something serious is a bit short sighted. There are two uses of the Unix “api”: [A] Long lived tools for other people to use. [B] Short lived tools one throws together oneself. The fact that most things work most of the time is why the shell works so well for B, and why it is indeed a poor choice for the sort of stable tools desig…

This doesn't always hold in Linux though. Some proc entries are available only via text entries you have to parse whether that's in sh or C. There's simply no public structured interface.

Re: The Beauty of Unix Pipelines

#106

Post by falafel »

mongol wrote: post #98
russellbeattie wrote: post #82

I think there's an interesting inflection point between piping different utilities together to get something done, and just whipping up a script to do the same thing instead. First I'll use the command line to, say, grab a file from a URL, parse, sort and format it. If I find myself doing the same commands a lot, I'll make a .sh file and pop the commands in there. But then there's that next step, which is where Bash…

I have never come across a NodeJS script instead of a shell script. Can you suggest an example to look at that shows the advantages of this?

There are plenty of CLI tools built on Node: https://github.com/sindresorhus/awesome-nodejs#command-line-...

Re: The Beauty of Unix Pipelines

#107

Post by ma2rten »

ianai wrote: post #27
gorgoiler wrote: post #21

Pipes are wonderful! In my opinion you can’t extol them by themselves. One has to bask in a fuller set of features that are so much greater than the sum of their parts, to feel the warmth of Unix: (1) everything is text (2) everything (ish) is a file (3) including pipes and fds (4) every piece of software is accessible as a file, invoked at the command line (5) ...with local arguments (6) ...and persistent globals in…

How’s it uncool? Just not “ mobile”?

I was also skeptical of that statement. I did a google trends search and indeed there seems to be a slow decline in command-line related searches:

https://trends.google.com/trends/explore?date=all&q=bash,awk...

Re: The Beauty of Unix Pipelines

#108

Post by nojito »

jpxw wrote: post #83
lordleft wrote: post #28

Earlier quoted context omitted.

A little bit of grep / awk goes very, very far.

grep, awk, sed, cut, sort, uniq and join are the Swiss Army knife of working with tabulated data on the command line.

And yet they are absolutely useless if columns get swapped silently.

Re: The Beauty of Unix Pipelines

#109

Post by fao_ »

dataflow wrote: post #85
bityard wrote: post #61

Earlier quoted context omitted.

Just quote paths? shrug

Try doing that in e.g. Make

Try finding out which registry key you need to edit to get something to happen.

Some things across different systems have pitfalls as a result of their implementation, or some features have pitfalls. There are half a dozen things I can point to on windows that are worse than the "spaces in filenames". Hell, windows doesn't even support ISO 8601 dates, or times to be properly placed in files. It restricts symbols much more than Linux / Unix

Re: The Beauty of Unix Pipelines

#110

Post by imglorp »

enedil wrote: post #97
imglorp wrote: post #45

Earlier quoted context omitted.

More Lego rules: (7) and common signalling facility (8) also nice if some files have magic properties like /dev/random or /proc or /dev/null (9) every program starts with 3 streams, stdin/stdout for work and stderr for out of band errors

(9) every program starts with 3 streams, stdin/stdout for work and stderr for out of band errors This is false - if a process closes stdin/out/err, its children won't have these files open.

That's correct, which is why I said _starts_ with the streams. The child can do what it wants with them after that.
Post reply on HN