Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Mighty Named Pipe

vincebuffalo.com

Post reply on HN
Print view View original

31–40 of 99 posts

Re: The Mighty Named Pipe

#31

Post by lazyjones »

a3n wrote: post #22
mhax wrote: post #3

I've used *nix for ~15 years and never used a named pipe or process substitution before. Great to know about!

Named pipes have been rare for me, but simple process substitution is every day. Very often I do something like this in quick succession. Command line editing makes this trivial. $ find . -name "*blarg*.cpp" # Some output that looks like what I'm looking for. # Run the same find again in a process, and grep for something. $ grep -i "blooey" $(find . -name "*blarg*.cpp") # Yep, those are the files I'm looking for, so…

> Command line editing makes this trivial.

I'm lazy, so I typically do this (2nd step) to avoid the extra key strokes necessary for editing:

  $ grep -i "blooey" $(!!)
Also very useful for avoiding editing in order to do something else with the same argument as in the previous command: !$, i.e.:

  $ foo myfile
  $ bla !$

Re: The Mighty Named Pipe

#32

Post by reitanqild »

icebraining wrote: post #26
jbnicolai wrote: post #25

Earlier quoted context omitted.

You could just use a pipe here though, which would also make it more easy to read. e.g.: $ find . -name '*blarg*.cpp' | grep -li blooey | vi -

Your version searches for blooey in the filenames, not in the files themselves.

And, to try to be helpful, - seems it lacks an xargs (or similar construct)

Re: The Mighty Named Pipe

#34

Post by baschism »

AFAIK process substitution is a bash-ism (not part of POSIX spec for /bin/sh). I recently had to go with the slightly less wieldy named pipes in a dash environment and put the pipe setup, command execution and teardown in a script.

Re: The Mighty Named Pipe

#35

Post by ajuc »

amelius wrote: post #9

If you like pipes, then you will love lazy evaluation. It is unfortunate, though, that Unix doesn't support that (operations can block when "writing" only, not when "nobody is reading").

I don't know, I love pipes and I'm on the fence regarding strict vs lazy.

BTW: when is nobody reading in pipes? There's always implicit

    &> stdout
added.

EDIT: oh, right, named pipes.

Re: The Mighty Named Pipe

#36

Post by tne »

a3n wrote: post #22
mhax wrote: post #3

I've used *nix for ~15 years and never used a named pipe or process substitution before. Great to know about!

Named pipes have been rare for me, but simple process substitution is every day. Very often I do something like this in quick succession. Command line editing makes this trivial. $ find . -name "*blarg*.cpp" # Some output that looks like what I'm looking for. # Run the same find again in a process, and grep for something. $ grep -i "blooey" $(find . -name "*blarg*.cpp") # Yep, those are the files I'm looking for, so…

Same here, except I typically use $(!!) to re-run the previous command. I find it faster than command-line editing.

    $ find . -name "*blarg*.cpp"
    $ grep -i "blooey" $(!!)
    $ vim $(!! -l)
Granted, you can only append new arguments and using the other ! commands will often be less practical than editing. Still, it's amazing how frequently this is sufficient.

I've always thought it'd be nice if there was a `set` option or something similar that would make bash record command lines and cache output automatically in implicit variables, so that it doesn't re-run the commands. The semantics are definitely different and you wouldn't want this enabled at all times, but for certain kinds of sessions it would be very handy.

EDIT: lazyjones beat me to it.

Re: The Mighty Named Pipe

#37

Post by masklinn »

mhax wrote: post #3

I've used *nix for ~15 years and never used a named pipe or process substitution before. Great to know about!

Beware though, process substitution is not POSIX and not supported in all shells. It isn't in pdksh or ash/dash for instance.

It's a ksh93 extension adopted by bash and zsh.

Re: The Mighty Named Pipe

#38

Post by vog »

tne wrote: post #36
a3n wrote: post #22

Earlier quoted context omitted.

Named pipes have been rare for me, but simple process substitution is every day. Very often I do something like this in quick succession. Command line editing makes this trivial. $ find . -name "*blarg*.cpp" # Some output that looks like what I'm looking for. # Run the same find again in a process, and grep for something. $ grep -i "blooey" $(find . -name "*blarg*.cpp") # Yep, those are the files I'm looking for, so…

Same here, except I typically use $(!!) to re-run the previous command. I find it faster than command-line editing. $ find . -name "*blarg*.cpp" $ grep -i "blooey" $(!!) $ vim $(!! -l) Granted, you can only append new arguments and using the other ! commands will often be less practical than editing. Still, it's amazing how frequently this is sufficient. I've always thought it'd be nice if there was a `set` option or…

Since "!!" are replaced when you hit the "up" arrow key (i.e. jump to previous command), you can go really wild with them:

https://oeis.org/A228162

Re: The Mighty Named Pipe

#39

Post by falcolas »

amelius wrote: post #9

If you like pipes, then you will love lazy evaluation. It is unfortunate, though, that Unix doesn't support that (operations can block when "writing" only, not when "nobody is reading").

If nobody is reading, you will eventually fill the pipe buffer (which is about 4k), and the writing will stop. It's a bigger queue than most of us would expect when compared to generator expressions, but it can and does create back pressure while making reads efficient.

Re: The Mighty Named Pipe

#40

Post by noselasd »

larsf wrote: post #33

Pipes are probably the original instantiation of dataflow processing (dating back to the 1960s). I gave a tech talk on some of the frameworks: https://www.youtube.com/watch?v=3oaelUXh7sE And my company creates a cool dataflow platform - https://composableanalytics.com

http://doc.cat-v.org/unix/pipes/ . And there's a bit more about how pipes came to be in unix here: http://cm.bell-labs.com/who/dmr/hist.html
Post reply on HN