Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Beauty of Unix Pipelines

prithu.xyz

Post reply on HN
Print view View original

131–140 of 388 posts

Re: The Beauty of Unix Pipelines

#131

Post by dredmorbius »

jrumbut wrote: post #84
laumars wrote: post #30

Earlier quoted context omitted.

> everything is text Everything is a byte stream. Usually that means text but sometimes it doesn't. Which means you can do fun stuff like: - copy file systems over a network: https://docs.oracle.com/cd/E18752_01/html/819-5461/gbchx.htm... - stream a file into gzip - backup or restore an SD card using `cat`

Also dd, which may be the disk destroyer but is also a great tool for binary file miracles. See one here: https://unix.stackexchange.com/questions/6852/best-way-to-re...

"data definition", from MVS JCL.

https://news.ycombinator.com/user?id=dredmorbius

Re: The Beauty of Unix Pipelines

#132

Post by viraptor »

coliveira wrote: post #77
jcranmer wrote: post #69

Earlier quoted context omitted.

There's a certain irony in responding to criticism of that which you're extolling by saying not to use it. And the only reason I might be pushed down that path is because the task I'm working on happens to involve filenames with spaces in them (without those spaces, the code would work fine!), because spaces are a reasonable thing to put in a filename unless you're on a Unix system.

The shell works with spaces, you just need to be careful to quote every file name. The advantage of using file names without spaces is that you can avoid the quotes.

Please don't ever assume you can avoid spaces and quotes. It's just a time bomb.

It's like people saying you don't need to escape SQL values because they come from constants. Yes, they do... today.

It's not just quoting either. It's setting the separator value and reverting it correctly. It's making sure you're still correct when you're in a function in an undefined state. It's a lot of overhead in larger projects.

Re: The Beauty of Unix Pipelines

#133

Post by hu3 »

fogetti wrote: post #128

I am sorry for playing the devil's advocate. I also think that pipes are extremely useful and a very strong paradigm, and I use them daily in my work. Also it is not an accident that it is fundamental and integral part of powershell too. But is this really HN top page worthy? I have seen this horse beaten to death for decades now. These kind of articles have been around since the very beginning of the internet. Am I…

Perhaps it's useful to reinforce pipes' utility to those who are not aware.

As for being top most article, if I had to guess it's because people feel related to the tool's power and enjoy sharing anecdotes of it.

Re: The Beauty of Unix Pipelines

#134

Post by DSMan195276 »

imglorp wrote: post #110
enedil wrote: post #97

Earlier quoted context omitted.

(9) every program starts with 3 streams, stdin/stdout for work and stderr for out of band errors This is false - if a process closes stdin/out/err, its children won't have these files open.

That's correct, which is why I said _starts_ with the streams. The child can do what it wants with them after that.

His point was that if you close stdin/stdout/stderr and then fork a child, the new child will not have all of those streams when it starts - the three streams are a convention, not something enforced by the kernel when creating a new process.

That said, it's still a bit of a pedantic point, you can expect to always have those three fds in the same way you can expect `argv[0]` to represent the name of your executable (Which also isn't enforced by the kernel).

Re: The Beauty of Unix Pipelines

#135

Post by dvirsky »

gorgoiler wrote: post #21

Pipes are wonderful! In my opinion you can’t extol them by themselves. One has to bask in a fuller set of features that are so much greater than the sum of their parts, to feel the warmth of Unix: (1) everything is text (2) everything (ish) is a file (3) including pipes and fds (4) every piece of software is accessible as a file, invoked at the command line (5) ...with local arguments (6) ...and persistent globals in…

> Unix is seriously uncool with young people at the moment.

Those damn kids with their loud rockn'roll music and their Windows machines. Back in my day we had he vocal stylings of Dean Martin and the verbal stylings of Linus Torvalds let me tell ya.

Seriously though, I'm actually seeing younger engineers really taking the time to learn how to do shell magic, using vim, etc. It's like the generation of programmers who started up until the late 90s used those by default, people who like me started 15-20 years ago grew up on IDEs and GUIs, but I've seen a lot of 20-30 something devs who are really into the Unix way of doing things. The really cool kids are in fact doing it.

Re: The Beauty of Unix Pipelines

#136

Post by julianeon »

cowmix wrote: post #4

I use pipelines as much as the next guy but every time I see post praise how awesome they are, I'm reminded of the Unix Hater's Handbook. Their take on pipelines is pretty spot on too. http://web.mit.edu/~simsong/www/ugh.pdf

I love UNIX but I think on this point I'm with the haters.

Practically speaking, as an individual, I don't have the needs of the DMV. For my personal use I'm combing through pip-squeaky data file downloads and CSV's.

So even though using Python or some other language causes a gigantic performance hit, it's just the difference between 0.002s and 0.02 seconds: a unit of time/inefficiency so small I can't ever perceive it. So I might also well use a language to do my processing, because at my level it's easier to understand and practically the same speed.

Re: The Beauty of Unix Pipelines

#137

Post by flukus »

ssivark wrote: post #75
gorgoiler wrote: post #21

Pipes are wonderful! In my opinion you can’t extol them by themselves. One has to bask in a fuller set of features that are so much greater than the sum of their parts, to feel the warmth of Unix: (1) everything is text (2) everything (ish) is a file (3) including pipes and fds (4) every piece of software is accessible as a file, invoked at the command line (5) ...with local arguments (6) ...and persistent globals in…

Serialized bytestreams do compose better than graphical applications. But that is setting a very low bar. For example, allowing passing around dicts/maps/json (and possibly other data structures) would already be a massive improvement. You know what might be even better — passing around objects you could interact with by passing messages (gasp! cue, Alan Kay). While posix and streams are nice (if you squint, files lo…

> For example, allowing passing around dicts/maps/json (and possibly other data structures) would already be a massive improvement.

It is allowed, you can pass around data in whatever encoding you desire. Not many do though because text is so useful for humans.

> You know what might be even better — passing around objects you could interact with by passing messages (gasp! cue, Alan Kay)

That's a daemon/service and basic unix utils make it easy to create, a shell script reading from a FIFO file fits this definition of object, the message passing is writing to the file and the object is passed around with the filename. Unix is basically a fulfillment of Alan Kay's idea of OO.

Re: The Beauty of Unix Pipelines

#138

Post by spijdar »

ryandrake wrote: post #104
gorgoiler wrote: post #21

Pipes are wonderful! In my opinion you can’t extol them by themselves. One has to bask in a fuller set of features that are so much greater than the sum of their parts, to feel the warmth of Unix: (1) everything is text (2) everything (ish) is a file (3) including pipes and fds (4) every piece of software is accessible as a file, invoked at the command line (5) ...with local arguments (6) ...and persistent globals in…

> (2) everything (ish) is a file I'm not all that knowledgable about Unix history, but one thing that has always puzzled me was that for whatever reason network connections (generally) aren't files. While I can do: cat to read from a serial device, I've always wondered why I can't do something like: bind /tmp/mysocket 12.34.56.78 80 cat It is weird that so many things in Unix are somehow twisted into files (/dev/fb0?…

The original authors of "UNIX" considered their creation to be fatally flawed, and created a successor OS called "Plan 9", partially to address this specific deficit, among others like the perceived tacked-on nature of GUIs on *nix.

That's a whole rabbit hole to go down, but Plan 9 networking is much more like your hypothetical example [0]. Additionally, things like the framebuffer and devices are also exposed as pure files, whereas on Linux and most unix-like systems such devices are just endpoints for making ioctl() calls.

[0] http://doc.cat-v.org/plan_9/4th_edition/papers/net/

Re: The Beauty of Unix Pipelines

#139

Post by cobalt »

ssivark wrote: post #75
gorgoiler wrote: post #21

Pipes are wonderful! In my opinion you can’t extol them by themselves. One has to bask in a fuller set of features that are so much greater than the sum of their parts, to feel the warmth of Unix: (1) everything is text (2) everything (ish) is a file (3) including pipes and fds (4) every piece of software is accessible as a file, invoked at the command line (5) ...with local arguments (6) ...and persistent globals in…

Serialized bytestreams do compose better than graphical applications. But that is setting a very low bar. For example, allowing passing around dicts/maps/json (and possibly other data structures) would already be a massive improvement. You know what might be even better — passing around objects you could interact with by passing messages (gasp! cue, Alan Kay). While posix and streams are nice (if you squint, files lo…

try powershell, its on linux too

Re: The Beauty of Unix Pipelines

#140

Post by bananabreakfast »

anthk wrote: post #95
gorgoiler wrote: post #53

Earlier quoted context omitted.

By “God” you may mean “libc”... ...if it was the 1970s. Nowadays we use Python etc for tools where it’s less acceptable for them to fall apart, though everything falls apart at some point, spaces or otherwise.

That's Perl role, not Python. You are like 20 years late, kid. Have fun with that shitty whitespace syntax. At least Perl can be refactorized.

Don't you mean Raku? lol
Post reply on HN