Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Beauty of Unix Pipelines

prithu.xyz

Post reply on HN
Print view View original

41–50 of 388 posts

Re: The Beauty of Unix Pipelines

#41

Post by goldenrhino »

sleepybrett wrote: post #32
Aaronstotle wrote: post #10

Earlier quoted context omitted.

I wouldn't be so quick to say it's an alt-right joke. Plenty of my friends will describe something/someone as a Chad and it has nothing to do with incel/alt-right cultures. Are those kind of phrases thrown out in those circles? Yes, but that's more of the general meme/internet lingo as opposed to subscribing to an ideology.

'Chad' originated in the incel subculture, be better.

No, it really didn’t. Even if it did, it has since transcended into something more general that many people are familiar with. Ending your statement with “be better” is condescending and seems purposefully antagonistic. How about you lead by example?

Re: The Beauty of Unix Pipelines

#42

Post by carlmr »

gorgoiler wrote: post #33
ianai wrote: post #27

Earlier quoted context omitted.

How’s it uncool? Just not “ mobile”?

Exactly. “ Linux, ew !” A big part of teaching computer science to children is breaking this obsession with approaching a computer from the top down — the old ICT ways, and the love of apps — and learning that it is a machine under your own control the understanding of which is entirely tractable from the bottom up. Unlike the natural sciences, computer science (like math) is entirely man made, to its advantage. No m…

>No microscopes, test tubes, rock hammers, or magnets required to investigate its phenomena.

They're full of magnets, and with electron microscope you can analyze what's going on in the hardware.

Re: The Beauty of Unix Pipelines

#43

Post by AnimalMuppet »

CalmStorm wrote: post #25

Unix pipelines are indeed beautiful, especially when you consider its similarity to Haskell's monadic I/O: http://okmij.org/ftp/Computation/monadic-shell.html Unix pipelines actually helped me make sense of Haskell's monad.

Could you be more specific? I don't get it.

Re: The Beauty of Unix Pipelines

#44

Post by hi41 »

Thank you! I did not know you get to a "sql like group by" using uniq -c. That's so cool! I think I used to pipe it to awk and count using an array and then display but your method is far better than mine.

Re: The Beauty of Unix Pipelines

#45

Post by imglorp »

gorgoiler wrote: post #21

Pipes are wonderful! In my opinion you can’t extol them by themselves. One has to bask in a fuller set of features that are so much greater than the sum of their parts, to feel the warmth of Unix: (1) everything is text (2) everything (ish) is a file (3) including pipes and fds (4) every piece of software is accessible as a file, invoked at the command line (5) ...with local arguments (6) ...and persistent globals in…

More Lego rules:

(7) and common signalling facility

(8) also nice if some files have magic properties like /dev/random or /proc or /dev/null

(9) every program starts with 3 streams, stdin/stdout for work and stderr for out of band errors

Re: The Beauty of Unix Pipelines

#46

Post by hnlmorg »

jcranmer wrote: post #38
gorgoiler wrote: post #21

Pipes are wonderful! In my opinion you can’t extol them by themselves. One has to bask in a fuller set of features that are so much greater than the sum of their parts, to feel the warmth of Unix: (1) everything is text (2) everything (ish) is a file (3) including pipes and fds (4) every piece of software is accessible as a file, invoked at the command line (5) ...with local arguments (6) ...and persistent globals in…

> (1) everything is text And lists are space-separated. Unless you want them to be newline-separated, or NUL-separated, which is controlled by an option that may or may not be present for the command you're invoking, and is spelled completely differently for each program. Or maybe you just quote spaces somehow, and good luck figuring out who is responsible for inserting quotes and who is responsible for removing them…

That's a POSIX shell thing rather than a Unix pipeline thing. Some non-POSIX shells don't have this problem while still passing data long Unix pipes

Source: I wrote a shell and it solves a great many of the space / quoting problems with POSIX shells.

Re: The Beauty of Unix Pipelines

#47

Post by gorgoiler »

jcranmer wrote: post #38
gorgoiler wrote: post #21

Pipes are wonderful! In my opinion you can’t extol them by themselves. One has to bask in a fuller set of features that are so much greater than the sum of their parts, to feel the warmth of Unix: (1) everything is text (2) everything (ish) is a file (3) including pipes and fds (4) every piece of software is accessible as a file, invoked at the command line (5) ...with local arguments (6) ...and persistent globals in…

> (1) everything is text And lists are space-separated. Unless you want them to be newline-separated, or NUL-separated, which is controlled by an option that may or may not be present for the command you're invoking, and is spelled completely differently for each program. Or maybe you just quote spaces somehow, and good luck figuring out who is responsible for inserting quotes and who is responsible for removing them…

To criticize sh semantics without acknowledging that C was always there when you needed something serious is a bit short sighted.

There are two uses of the Unix “api”:

[A] Long lived tools for other people to use.

[B] Short lived tools one throws together oneself.

The fact that most things work most of the time is why the shell works so well for B, and why it is indeed a poor choice for the sort of stable tools designed for others to use, in A.

The ubiquity of the C APIs of course solved [A] use cases in the past, when it was unconscionable to operate a system without cc(1). It’s part of why they get first class treatment in the Unix man pages, as old fashioned as that seems nowadays.

Re: The Beauty of Unix Pipelines

#48

Post by cuddlybacon »

cowmix wrote: post #4

I use pipelines as much as the next guy but every time I see post praise how awesome they are, I'm reminded of the Unix Hater's Handbook. Their take on pipelines is pretty spot on too. http://web.mit.edu/~simsong/www/ugh.pdf

I mostly like what they wrote about pipes. I think the example of bloating they talked about in ls at the start of the shell programming section is a good example: if pipelines are so great, why have so many unix utilities felt the need to bloat?

I think it a result of there being just a bit too much friction in building a pipeline. A good portion tends to be massaging text formats. The standard unix commands for doing that tend to have infamously bad readability.

Fish Shell seems to be making this better by making a string which has a syntax that makes it clear what it is doing: http://fishshell.com/docs/current/cmds/string.html I use fish shell, and I can usually read and often write text manipulations with the string command without needing to consult the docs.

Nushell seems to take a different approach: add structure to command output. By doing that, it seems that a bunch of stuff that is super finicky in the more traditional shells ends up being simple and easy commands with one clear job in nushell. I have never tried it, but it does seem to be movement in the correct direction.

Re: The Beauty of Unix Pipelines

#49

Post by junke »

Aaronstotle wrote: post #10
dodwer wrote: post #8

Earlier quoted context omitted.

It's an alt-right in-joke related to the incel subculture.

I wouldn't be so quick to say it's an alt-right joke. Plenty of my friends will describe something/someone as a Chad and it has nothing to do with incel/alt-right cultures. Are those kind of phrases thrown out in those circles? Yes, but that's more of the general meme/internet lingo as opposed to subscribing to an ideology.

This is brogrammer speak (e.g. http://new_words.enacademic.com/662/brogrammer).

Re: The Beauty of Unix Pipelines

#50

Post by every »

young_unixer wrote: post #23
salmo wrote: post #20

Earlier quoted context omitted.

I think people can go through a few stages of their shell-foo. The first involves a lot of single commands and temporary files. The second uses pipes, but only tacks on commands with no refactoring. The third would recognize that all the grep and cut should just be awk, that you can redirect the cumulative output of a control statement, that subprocesses and coroutines are your friend. We should all aspire to this. T…

My most complex use of the shell is defining aliases in .bashrc and have never felt the need to go further. Do you recommend learning all of that for someone like me? If so, what resource do you recommend?

You can get a lot of mileage out of aliases, especially compound ones. But eventually you may run up against something they don't handle very well. For that a simple shell script might be the next logical step. And fortunately, they aren't that much of a leap...
Post reply on HN