Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Beauty of Unix Pipelines

prithu.xyz

Post reply on HN
Print view View original

171–180 of 388 posts

Re: The Beauty of Unix Pipelines

#171

Post by jandrese »

zamalek wrote: post #158
shakna wrote: post #150

Earlier quoted context omitted.

If you're having to extract your data using a regex, then the data probably isn't well-formed enough for a shell pipeline. It's doable, but a bad idea. Regex should not be the first hammer you reach for, because it's a scalpel. I recently wanted cpu cores + 1. That could be a single regex. But this is more maintainable, and readable: echo '1 + '"$(grep 'cpu cores' /proc/cpuinfo | tail -n1 | awk -F ':' '{print $2}')"…

Would anyone be able to approach this, without knowledge of /proc/cpuinfo, and understand what `$2` refers to? > Regex should not be the first hammer you reach for, because it's a scalpel. You used a regex tool: grep.

It's plaintext, you could just look at it. That's the beauty of text based systems.

Re: The Beauty of Unix Pipelines

#172

Post by buzzkillington »

hnlmorg wrote: post #46
jcranmer wrote: post #38

Earlier quoted context omitted.

> (1) everything is text And lists are space-separated. Unless you want them to be newline-separated, or NUL-separated, which is controlled by an option that may or may not be present for the command you're invoking, and is spelled completely differently for each program. Or maybe you just quote spaces somehow, and good luck figuring out who is responsible for inserting quotes and who is responsible for removing them…

That's a POSIX shell thing rather than a Unix pipeline thing. Some non-POSIX shells don't have this problem while still passing data long Unix pipes Source: I wrote a shell and it solves a great many of the space / quoting problems with POSIX shells.

Same, I've built toy shells [0] that used xml, s-expressions and ascii delimited text [1]. The last was closest to what unix pipes should be like. Of course it breaks ALL posix tools, but it felt like a shell that finally works as you'd expect it to work.

[0] Really just input + exec + pipes.

[1] https://en.wikipedia.org/wiki/Delimiter#ASCII_delimited_text

Re: The Beauty of Unix Pipelines

#173

Post by jandrese »

nojito wrote: post #70

Debugging Linux pipelines is not a fun experience. This is one clear area where Powershell with its object model has got it right.

I don't find it too bad most of the time. If the pipeline isn't working you can cut it off at any juncture and watch standard out to see whats going on.

You can even redirect some half processed data to a file so you don't have to re-run the first half of the pipe over and over while you work out what's going wrong in the tail end.

Re: The Beauty of Unix Pipelines

#174

Post by a4wtaw4tawtawt »

adamnemecek wrote: post #74

Unix pipes are the a 1970's construct, the same way bell bottom pants are. It's a construct that doesn't take into account the problems and scale of today's computing. Unicode? Hope your pipes process it fine. Video buffers? High perf? Fuggetaboutit. Piping the output of ls to idk what? Nice, I'll put it on the fridge.

Shhh, you're upsetting them!

Re: The Beauty of Unix Pipelines

#175

Post by fomine3 »

jcranmer wrote: post #38
gorgoiler wrote: post #21

Pipes are wonderful! In my opinion you can’t extol them by themselves. One has to bask in a fuller set of features that are so much greater than the sum of their parts, to feel the warmth of Unix: (1) everything is text (2) everything (ish) is a file (3) including pipes and fds (4) every piece of software is accessible as a file, invoked at the command line (5) ...with local arguments (6) ...and persistent globals in…

> (1) everything is text And lists are space-separated. Unless you want them to be newline-separated, or NUL-separated, which is controlled by an option that may or may not be present for the command you're invoking, and is spelled completely differently for each program. Or maybe you just quote spaces somehow, and good luck figuring out who is responsible for inserting quotes and who is responsible for removing them…

The fact that filename can contain anything except NULL/slash is really pain. I often write a shell script that treats LF as separator but I know it's not good.

Re: The Beauty of Unix Pipelines

#176

Post by klyrs »

1MachineElf wrote: post #103
nerdponx wrote: post #60

Earlier quoted context omitted.

C was always there when you needed something serious There is a world of stuff in between "I need relatively low-level memory management" and "I need a script to just glue some shit together". For that we have Python and Perl and Ruby and Go, or even Rust.

Going with the same theme, C itself was an innovation meant to fill the space between Assembler (in this context, "A") and B[0]. [0] https://en.wikipedia.org/wiki/B_(programming_language)

That isn't really supported by the article you linked -- it describes C as an extension of B multiple times.

Re: The Beauty of Unix Pipelines

#177

Post by yjftsjthsd-h »

uryga wrote: post #146
kccqzy wrote: post #99

Earlier quoted context omitted.

It replaces the executable in the current process with a different executable. It's kind of like spawning a new process with an executable, except it's the same PID, and file descriptors without CLOEXEC remain open in their original state.

would `become` be a good alternate name for it?

That's loosely correct.

Re: The Beauty of Unix Pipelines

#178

Post by yjftsjthsd-h »

carlmr wrote: post #42
gorgoiler wrote: post #33

Earlier quoted context omitted.

Exactly. “ Linux, ew !” A big part of teaching computer science to children is breaking this obsession with approaching a computer from the top down — the old ICT ways, and the love of apps — and learning that it is a machine under your own control the understanding of which is entirely tractable from the bottom up. Unlike the natural sciences, computer science (like math) is entirely man made, to its advantage. No m…

>No microscopes, test tubes, rock hammers, or magnets required to investigate its phenomena. They're full of magnets, and with electron microscope you can analyze what's going on in the hardware.

Yeah, but it's not required for a lot of investigation.

Re: The Beauty of Unix Pipelines

#179

Post by yjftsjthsd-h »

blacksqr wrote: post #127
anthk wrote: post #96

Earlier quoted context omitted.

- Python as a shell would be the worst crapware ever. Whitespace syntax, no proper file autocompletion, no good pipes, no nothing. Even Tclsh with readline would be better than a Python shell. - By mixing C and C++ you look like a clueless youngster. - /usr/local is to handle non-base/non-packages stuff so you don't trash your system. Under OpenBSD, /usr/local is for packages, everything else should be under /opt or…

> Even Tclsh with readline would be better than a Python shell In fact, I think Tcl would make an excellent sh replacement.

So, wish(1)?

Re: The Beauty of Unix Pipelines

#180

Post by abnry »

dvirsky wrote: post #135
gorgoiler wrote: post #21

Pipes are wonderful! In my opinion you can’t extol them by themselves. One has to bask in a fuller set of features that are so much greater than the sum of their parts, to feel the warmth of Unix: (1) everything is text (2) everything (ish) is a file (3) including pipes and fds (4) every piece of software is accessible as a file, invoked at the command line (5) ...with local arguments (6) ...and persistent globals in…

> Unix is seriously uncool with young people at the moment. Those damn kids with their loud rockn'roll music and their Windows machines. Back in my day we had he vocal stylings of Dean Martin and the verbal stylings of Linus Torvalds let me tell ya. Seriously though, I'm actually seeing younger engineers really taking the time to learn how to do shell magic, using vim, etc. It's like the generation of programmers who…

I guess I am one of the young kids who think the unix command line is wicked cool. It makes the user experience on my laptop feel so much more powerful.
Post reply on HN