Does anyone have a working link to Gary Bernhardt's The Unix Chainsaw, as mentioned in the article?
I found a high quality copy I had downloaded: http://sikhnerd.com/downloaded_vids/02-gary-bernhardt.mp4 (703M)
The Mighty Named Pipe
61–70 of 99 posts
Re: The Mighty Named Pipe
#62Nice article. Really easy to follow introduction. I only discovered process substitution a few months ago but it's already become a frequently used tool in my kit. One thing that I find a little annoying about unix commands sometimes is how hard it can be to google for them. ' Unless you know to look for "Process Substitution" it can be hard to find information on these things. And that's once you even know these thi…
I often use SymbolHound in these cases - it's a search engine with support for special characters; for example: http://symbolhound.com/?q=%3C%28%29
Re: The Mighty Named Pipe
#63Earlier quoted context omitted.
No, that's not why you were down voted. You were down voted because you were condescending to the people who enjoy working with *nix.
It really is a question that I've been having for a long time. I didn't just say that to piss people off. I guess that's the risk you run of coming across when you try to insert yourself in a conversation where the other participants have already agreed on a set of shared opinions - this is great - and you try to question that common assumption/opinion. I have honestly been questioning my own understanding of pipe ,…
I don't think it's that you have a differing opinion. Those are great and most people would be OK with that. I really believe you got the down votes because what you said came off as condescending.
If you had said what you just said in your follow up, I think you'd actually have gotten some up votes.
Re: The Mighty Named Pipe
#64Nice article. Really easy to follow introduction. I only discovered process substitution a few months ago but it's already become a frequently used tool in my kit. One thing that I find a little annoying about unix commands sometimes is how hard it can be to google for them. ' Unless you know to look for "Process Substitution" it can be hard to find information on these things. And that's once you even know these thi…
A bit OT but I don't understand why Google doesn't supply a way to do strict searches where everything you input is interpreted literally.
I wish they'd supply that too, but they do seem to have gotten better at interpreting literally when it makes sense in context. I've been learning C# and have found, for example, that searches with the term "C#" return the appropriate resources when in the past I'd have probably seen results for C.
Re: The Mighty Named Pipe
#65Earlier quoted context omitted.
man pages! $ man bash / Drops you right into the Process Substitution section.
Ahhhh..haaa...ha....DOH! I've never even thought of looking at the manpage for bash before. Thanks, you've just made my life better.
If you're interested in stock Posix shell rather than bashish, the dash man page is a whole lot shorter and easier to follow, makes a great concise reference.
Re: The Mighty Named Pipe
#66Earlier quoted context omitted.
I guess you found another issue with Unix. The user does not care in general how something is performed, just that it is performed correctly and with good performance. I guess an OS should be functional at its interface to the user, and only imperative deep down to keep things running efficiently. However, note that this hypothetical functional layer on top also would ensure efficiency, as it enables lazy evaluation.…
>The user does not care in general how something is performed, just that it is performed correctly and with good performance. This is the crux of the matter. With BASH scripting the user does care how a task is preformed as that task maybe system administration, involve sensitive system components, OR sensitive data. Lazy evaluation is great for binary/cpu level optimization. But passing system administration tasks t…
Well, in any case, the problem can be resolved by adding a kernel-level api function that allows one to wait (block) until results are requested from the other end of the pipe.
Re: The Mighty Named Pipe
#67Earlier quoted context omitted.
A bit OT but I don't understand why Google doesn't supply a way to do strict searches where everything you input is interpreted literally.
I wish they'd supply that too, but they do seem to have gotten better at interpreting literally when it makes sense in context. I've been learning C# and have found, for example, that searches with the term "C#" return the appropriate resources when in the past I'd have probably seen results for C.
Google handles some constructs with punctuation as atomic tokens as special cases. C# and C++ are examples. A# through G# also return appropriate results, for the musical notes. H# and onward through the alphabet do not.
.NET is another example. Google will ignore a prepended dot on most words, but .NET is handled specially as an atomic token. I would bet this is a product of human curation, not of algorithms that have somehow identified .NET as a semantic token.
Searching for punctuation in a general case is hard, though. You wouldn't want a search for Lisp to fail to match pages with (Lisp). We often forget that the pages are tokenized and indexed, that Google and the other search engines aren't a byte-for-byte scan across the entire web.
I was recently trying to understand the difference between the <%# and <%= server tags in ASP.NET. Google couldn't even interpret those as tokens to search for. It took me a long time to figure out the former's true name as the data-bind operator in order to search for that and find the MS docs.
Re: The Mighty Named Pipe
#68Nice article. Really easy to follow introduction. I only discovered process substitution a few months ago but it's already become a frequently used tool in my kit. One thing that I find a little annoying about unix commands sometimes is how hard it can be to google for them. ' Unless you know to look for "Process Substitution" it can be hard to find information on these things. And that's once you even know these thi…
Be aware that process substitution (and named pipes) can bite you in the arse in some situations --- for example, if the program expects to be able to seek in the file. Pipes don't support this and the program will see it as an I/O error. This'd be fine if programs just errored out cleanly but they frequently don't check that seeking succeeds. unzip treats a named pipe as a corrupt zipfile, for example:
$ unzip Re: The Mighty Named Pipe
#69Anybody know of a way to increase the buffer size of pipes? I've experienced cases where piping a really fast program to a slow one caused them both to go slower as the OS pauses first program writing when pipe buffer is full. This seemed to ruin the caching for the first program and caused them both to be slower even though normally pipes are faster as you're not touching disk.
Re: The Mighty Named Pipe
#70Anybody know of a way to increase the buffer size of pipes? I've experienced cases where piping a really fast program to a slow one caused them both to go slower as the OS pauses first program writing when pipe buffer is full. This seemed to ruin the caching for the first program and caused them both to be slower even though normally pipes are faster as you're not touching disk.
Both mbuffer and pv by default contain fairly large in-memory buffers for pipe data, and accept parameters for particularly large buffers.
http://www.maier-komor.de/mbuffer.html http://www.ivarch.com/programs/pv.shtml