# avoid temporary files when some program needs two inputs:
join -e0 -o0,1.1,2.1 -a1 -a2 -j2 -t$'\t' \
The Mighty Named Pipe
11–20 of 99 posts
Re: The Mighty Named Pipe
#12Nice article. Really easy to follow introduction. I only discovered process substitution a few months ago but it's already become a frequently used tool in my kit. One thing that I find a little annoying about unix commands sometimes is how hard it can be to google for them. ' Unless you know to look for "Process Substitution" it can be hard to find information on these things. And that's once you even know these thi…
man pages! $ man bash / Drops you right into the Process Substitution section.
/
For those wondering, man, which uses less as a pager, has vi-like key bindings."/This is the origin of the regex literal syntax in most programming languages that have them. It was first introduced by Ken Thompson in the "ed" text editor.
Re: The Mighty Named Pipe
#13Nice article. Really easy to follow introduction. I only discovered process substitution a few months ago but it's already become a frequently used tool in my kit. One thing that I find a little annoying about unix commands sometimes is how hard it can be to google for them. ' Unless you know to look for "Process Substitution" it can be hard to find information on these things. And that's once you even know these thi…
man pages! $ man bash / Drops you right into the Process Substitution section.
Re: The Mighty Named Pipe
#14Re: The Mighty Named Pipe
#15How does the > process substitution differ from simply piping the output with | ? For example (from Wikipedia) tee >(wc -l >&2) bigfile.gz vs tee bigfile.gz
a | b
you connect stdout (fd #1) of a to stdin (fd #0) of b. Technically, the shell process will create a pipe, which is two filedescriptors connected back to back. It then will fork two times (create two copies of itself) where it replaces standard output (filedescriptor 1) of the first copy by one end of the pipe and replaces standard input (filedescriptor 0) of the second copy by the other end of the pipe. Then the first copy will replace itself (exec) by a, the second copy will replace itself (exec) by b. Everything that a writes to stdout will appear on stdin of b.But nothing prevents the shell from replacing any other filedescriptor by pipes. And when you create a subprocess by writing "" (symlink to /proc/self/fd/ which itself is a symlik to /proc//fd/), so that's what's replaced as a "name" to refer to the substituted process on "a"'s command line:
Try this:
$ echo $$
12345 # ( sort ) >( sort ) >( sort ) otherfile | sort
and in a second terminal $ pstree 12345 #
If your system doesn't support the convenient /proc/self/fd/ shortcut, the shell might decide not to create a pipe, but rather create temporary fifos in /tmp and use those to connect the filedescriptors.http://man7.org/linux/man-pages/man2/pipe.2.html
http://linux.die.net/man/2/dup
You can watch the syscalls as they are made:
$ strace -fe fork,pipe,close,dup,dup2,execve bash -c 'tee Re: The Mighty Named Pipe
#16Re: The Mighty Named Pipe
#17Once you discover # avoid temporary files when some program needs two inputs: join -e0 -o0,1.1,2.1 -a1 -a2 -j2 -t$'\t' \
Something wonderful I found out the other day: Bash executes scripts as it parses them, so you can do all kinds of awful things. For starters,
bash
will have bash execute an infinite script that looks like echo hello
echo hello
echo hello
...
without trying to load the whole thing first.After that, you can move onto having a script append to itself and whatever other dreadful things you can think of.
Re: The Mighty Named Pipe
#18Re: The Mighty Named Pipe
#19Pipes are very cool and useful, but it's hard for me to understand this common worship of something like that. Yes, it's useful and elegant, but is it really the best thing since Jesus Christ?
Re: The Mighty Named Pipe
#20Once you discover # avoid temporary files when some program needs two inputs: join -e0 -o0,1.1,2.1 -a1 -a2 -j2 -t$'\t' \
> # gawk doesn't care if it's given a regular file or the output fd of some process: Something wonderful I found out the other day: Bash executes scripts as it parses them, so you can do all kinds of awful things. For starters, bash will have bash execute an infinite script that looks like echo hello echo hello echo hello ... without trying to load the whole thing first. After that, you can move onto having a script…