Earlier quoted context omitted.
Monads are effectively pipes; the monad controls how data flows through the functions you put into the monad, but the functions individually are like individual programs in a pipe.
I'm not sure I agree with this. Function composition is more directly comparable to pipes, whereas I tend to think of monads as collapsing structure (i.e. `join :: m (m a) -> m a`)
For the Love of Pipes
231–240 of 323 posts
Re: For the Love of Pipes
#232I’m surprised JessFraz who is employed by Microsoft doesn’t talk about powershell pipes at all. Powershell pipes are an extension over Unix pipes. Rather than just being able to pipe a stream of bytes, powershell can pipe a stream of objects. It makes working with pipes so much fun. In Unix you have to cut, awk and do all sorts of parsing to get some field out of `ls`. In poweshell, ls outputs stream of file objects…
She is actually employed at GitHub now. https://twitter.com/jessfraz
Re: For the Love of Pipes
#233I'm probably nitpicking, but if you're using cat to pipe a single file into the sdtin of another program, you most likely don't need the cat in the first place, you can just redirect the file to the process' stdin. Unless, of course, you're actually concatenating multiple files or maybe a file and stdin together. Disclaimer: I do cat-piping myself quite a bit out of habit, so I'm not trying to look down at the author…
In fact, I don't like people optimizing shell scripts for performance. I mean, shell scripts are slow by design and if you need something fast, you choose the wrong technology in the first place. Instead, shell script should be optimized for readability and portability and I think it is much easier to understand something like 'read | change >write' than 'change write'. So I like to write pipelines like this: cat foo…
For example, you can write this pipeline as:
grep '^x' foo.txt \
| sed 's/a/b/g' \
| awk '{print $2}' \
| wc -l > bar.txt
This is by no means scientific, but I've got a LaTeX document open right now. A quick `time` says: $ time grep 'what' AoC.tex
real 0m0.045s
user 0m0.000s
sys 0m0.000s
$ time cat AoC.tex | grep what
real 0m0.092s
user 0m0.000s
sys 0m0.047s
Anecdotally, I've witnessed small pipelines that absolutely make sense totally thrash a system because of inappropriate uses of `cat`. When you `cat` a file, the OS must (1) `fork` and `exec`, (2) copy the file to `cat`'s memory, (3) copy the contents of `cat`'s memory to the pipe, and (4) copy the contents of the pipe to `grep`'s memory. That's a whole lot of copying for large files -- especially when the first command grep in the sequence usually performs some major kind of reduction on the input data!Re: For the Love of Pipes
#234Pipes are awesome and infuriating. Sometimes they work great -- being able to dump from MySQL into gzip sending across the wire via ssh into gunzip and into my local MySQL without ever touching a file feels nothing short of magic... although the command/incantation to do so took quite a while to finally get right. But far too often they inexplicably fail. For example, I had an issue last year where piping curl to bun…
> Any given combination of pipe tools, there's a kind of random chance they'll actually work in the end or not. While this may be your experience, the mechanism of FIFO pipes in Unix (which is filehandles and buffers, basically), is an old one that is both elegant and robust; it doesn't "randomly" fail due to unreliability of the core algorithm or components. In 20 years, I never had an init script or bash command fa…
One issue I did used to have (before I discovered ‘-o pipefail’[1]) was the annoyance that if an earlier command in a pipeline failed, all the other commands in the pipeline still ran albeit with no data or garbage data being piped to them.
[1] https://stackoverflow.com/questions/1550933/catching-error-c...
Re: For the Love of Pipes
#235I’m surprised JessFraz who is employed by Microsoft doesn’t talk about powershell pipes at all. Powershell pipes are an extension over Unix pipes. Rather than just being able to pipe a stream of bytes, powershell can pipe a stream of objects. It makes working with pipes so much fun. In Unix you have to cut, awk and do all sorts of parsing to get some field out of `ls`. In poweshell, ls outputs stream of file objects…
She is actually employed at GitHub now. https://twitter.com/jessfraz
Re: For the Love of Pipes
#236pipe junkies might like to know about the following tools: * vipe (part of https://joeyh.name/code/moreutils/ - lets you edit text part way through a complex series of piped commands) * pv ( http://www.ivarch.com/programs/pv.shtml - lets you visualise the flow of data through a pipe)
Yes! I love pv. Besides that and tee, can anyone else suggest some more general pipe tools?
* pee: tee standard input to pipes (`pee "some-command" "another-command"`)
* sponge: soak up standard input and write to a file
Though in zsh and bash you can create pee using tee: `tee >(some-command) >(another-command) >/dev/(null`
Re: For the Love of Pipes
#237I’m surprised JessFraz who is employed by Microsoft doesn’t talk about powershell pipes at all. Powershell pipes are an extension over Unix pipes. Rather than just being able to pipe a stream of bytes, powershell can pipe a stream of objects. It makes working with pipes so much fun. In Unix you have to cut, awk and do all sorts of parsing to get some field out of `ls`. In poweshell, ls outputs stream of file objects…
I'm guessing you've mentioned using `ls` as a simple, first-thing-that-comes-to-mind example, which is cool. I just wanted to point out that if a person is piping ls's output, there are probably other, far better alternatives, such as (per the link below) `find` and globs:
https://unix.stackexchange.com/a/247973
That has been my experience, at least.
Re: For the Love of Pipes
#238Earlier quoted context omitted.
I would hate to see the day HN allowed any way to bold sections of text. It's way more restful purveying a page of uniformly restrained text.
How is that meaningfully different to italics in that regard?
Re: For the Love of Pipes
#239(I love pipes too btw)
Re: For the Love of Pipes
#240Earlier quoted context omitted.
In your original command, how can 'input' be a program with no arguments?
Oh, damn. You're exactly right. OK, to save some of my face, this will work: grep 'foo' ... at least in zsh and probably bash.
input | grep foo | sed ...