One surprising fact about Linux pipes I stumbled across 4 years ago is that using a pipe can create indeterministic behavior: https://www.gibney.org/the_output_of_linux_pipes_can_be_inde...
How fast are Linux pipes anyway? (2022)
41–50 of 116 posts
Re: How fast are Linux pipes anyway? (2022)
#42One surprising fact about Linux pipes I stumbled across 4 years ago is that using a pipe can create indeterministic behavior: https://www.gibney.org/the_output_of_linux_pipes_can_be_inde...
Not surprising, the pipe you've created doesn't transport any of the data you've echoed. (echo red; echo green 1>&2) | echo blue This creates two subshells separated by the pipe | symbol. A subshell is a child process of the current shell, and as such it inherits important properties of the current shell, notably including the open file descriptor table. Since they are child processes, both subshells run concurrently…
(The data "runs" in serial.)
Re: How fast are Linux pipes anyway? (2022)
#43Earlier quoted context omitted.
I have to agree... I really like the side-notes to get more details/explanation. You can skip the side-notes to keep reading and stay on the main story, but get what normally would be included in parenthesis or otherwise as an in-line comment.... Best of both worlds here I think. If I actively maintained a blog, I'd probably steal this design! :-)
Is there some standard css/html way of pushing side notes or pics into the first column if viewing width is too small? That would be the best of both worlds!
Re: How fast are Linux pipes anyway? (2022)
#44One surprising fact about Linux pipes I stumbled across 4 years ago is that using a pipe can create indeterministic behavior: https://www.gibney.org/the_output_of_linux_pipes_can_be_inde...
PS: Precision of language to avoid confusion: "Indeterministic" is a philosophy term, while the CS term is "nondeterministic".
0. https://blog.superpat.com/zero-copy-in-linux-with-sendfile-a...
Re: How fast are Linux pipes anyway? (2022)
#45I could only see it because of my dark mode extension, otherwise I guarantee I wouldn't have caught it.
Re: How fast are Linux pipes anyway? (2022)
#46Anyone see the stonks image hidden quite well behind the first table? I could only see it because of my dark mode extension, otherwise I guarantee I wouldn't have caught it.
Re: How fast are Linux pipes anyway? (2022)
#47https://github.com/nathants/s4/blob/master/examples/nyc_taxi...
Re: How fast are Linux pipes anyway? (2022)
#48Earlier quoted context omitted.
Not surprising, the pipe you've created doesn't transport any of the data you've echoed. (echo red; echo green 1>&2) | echo blue This creates two subshells separated by the pipe | symbol. A subshell is a child process of the current shell, and as such it inherits important properties of the current shell, notably including the open file descriptor table. Since they are child processes, both subshells run concurrently…
Thank you for taking the time to write this very detailed and lucid explanation.
If you want a program to read from stdin and write to stdout, you can use the `cat`, e.g. `echo a | cat` will print “a”.
Lastly, be aware that `echo` is usually a shell builtin that functions like `print`. I’m not sure of all the ways that it might behave differently, but something to be aware of (that it’s not a child process like `cat`).
Re: How fast are Linux pipes anyway? (2022)
#49Other main takeaway, it’s a bit sad that the naive implementation everybody will write, is 20x slower than what is possible.
Exceptionally written article btw.
Re: How fast are Linux pipes anyway? (2022)
#50One surprising fact about Linux pipes I stumbled across 4 years ago is that using a pipe can create indeterministic behavior: https://www.gibney.org/the_output_of_linux_pipes_can_be_inde...
Not surprising, the pipe you've created doesn't transport any of the data you've echoed. (echo red; echo green 1>&2) | echo blue This creates two subshells separated by the pipe | symbol. A subshell is a child process of the current shell, and as such it inherits important properties of the current shell, notably including the open file descriptor table. Since they are child processes, both subshells run concurrently…
Yes the pipe runs two subcommands in parallel but that is not why the blogpost is interesting (or its author surprised). It's because 'echo red' is supposed to block, thus introducing synchronization between the two branches of the pipe, yet it doesn't!
And I must confess, when reading the command my first though was: "Ok so that first echo will die with a SIGPIPE and stderr will be all about the broken pipe." And I was wrong, because of that small buffer.
I wonder what other unices do allow a write to a broken pipe to complete successfully?