Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

For the Love of Pipes

blog.jessfraz.com

Post reply on HN
Print view View original

121–130 of 323 posts

Re: For the Love of Pipes

#121

Post by duckerude »

msravi wrote: post #104
duckerude wrote: post #20

Earlier quoted context omitted.

Many functional languages have |> for piping, but chained method calls are also a lot like pipelines. Data goes from left to right. This javascript expression: [1, 2, 3].map(n => n + 1).join(',').length Is basically like this shell command: seq 3 | awk '{ print $1 + 1 }' | tr '\n' , | wc -c (the shell version gives 6 instead of 5 because of a trailing newline, but close enough)

In Julia, this would be: 1:3 |> x->x+1 |> x->join(x,",") |> length

Small correction (or it won't run on my system):

  1:3 |> x -> map(y -> y+1, x) |> x -> join(x, ",") |> length
All those anonymous functions seem a bit distracting, though https://github.com/JuliaLang/julia/pull/24990 could help with that.

Re: For the Love of Pipes

#122

Post by korla »

koboll wrote: post #113
glenneroo wrote: post #77

Earlier quoted context omitted.

Why is this or OP's even necessary? The bullet points are copied directly from the article.

I'm sure I'm not the only one who opens the comments to skim them and quickly vet whether the article is worth reading or not

You are not. I made a chrome plugin to find the HN-discussions for an article thinking I'd use it primarily after I'd read an article, but I find that I more often than not use it as a benchmark whether I should spend the time to read it or not.

Re: For the Love of Pipes

#123

Post by arendtio »

yaakushi wrote: post #85

I'm probably nitpicking, but if you're using cat to pipe a single file into the sdtin of another program, you most likely don't need the cat in the first place, you can just redirect the file to the process' stdin. Unless, of course, you're actually concatenating multiple files or maybe a file and stdin together. Disclaimer: I do cat-piping myself quite a bit out of habit, so I'm not trying to look down at the author…

In fact, I don't like people optimizing shell scripts for performance. I mean, shell scripts are slow by design and if you need something fast, you choose the wrong technology in the first place.

Instead, shell script should be optimized for readability and portability and I think it is much easier to understand something like 'read | change >write' than 'change write'. So I like to write pipelines like this:

  cat foo.txt \
    | grep '^x' \
    | sed 's/a/b/g' \
    | awk '{print $2}' \
    | wc -l >bar.txt
It might be not the most efficient processing method, but I think it is quite readable.

For those who disagree with me: You might find the pure-bash-bible [1] valuable. While I admire their passion for shell scripts, I think they are optimizing to the wrong end. I would be more a fan of something along the lines of 'readable-POSIX-shell-bible' ;-)

[1]: https://github.com/dylanaraps/pure-bash-bible

Re: For the Love of Pipes

#124

Post by da_chicken »

laumars wrote: post #97
AnIdiotOnTheNet wrote: post #84

Earlier quoted context omitted.

Yes. The compromise of just using an untyped byte stream in a single linear pipeline was a fair tradeoff in the 70s, but it is nearly 2020 and we can do better.

We have done better. The shell I'm writing is typed and I know I'm not the only person to do this (eg Powershell). The issue here is really more with POSIX compatibility but if you're willing to step away from that then you might find an alternative that better suits your needs. Thankfully switching shells is as painless as switching text editors.

> Thankfully switching shells is as painless as switching text editors.

So, somewhere between, "That wasn't as bad as I feared," and, "Sweet Jesus, what fresh new hell have I found myself in"?

Re: For the Love of Pipes

#126

Post by jerf »

alkonaut wrote: post #82

I love the idea of simple things that can be connected in any way. I'm not so much a fan of "everything is a soup of bytes with unspecified encoding and unknown formatting". It's an abstraction that held up quite well, but its starting to show its age.

I fully agree... and yet... everyone who has tried to "fix" this has failed, at least in the sense of "attaining anything like shell's size and reach". Many have succeeded in the sense of producing working code that fixes this in some sense.

Powershell's probably the closest to success, because it could be pushed out unilaterally. Without that I'm not sure it would have gotten very far, not because it's bad, but again because nobody else seems to have gotten very far....

Re: For the Love of Pipes

#128

Post by roelb »

Fully agree, pipes are awesome, only downside is the potential duplicate serialization/deserialization overhead.

Streams in most decent languages closely adhere to this idea.

I especially like how node does it, in my opinion one of the best things in node. Where you can simply create cli programs that have backpressure the same as you would work with binary/file streams, while also supporting object streams.

    process.stdin.pipe(byline()).pipe(through2(transformFunction)).pipe(process.stdout)

Re: For the Love of Pipes

#129

Post by [deleted] »

alias_neo wrote: post #2

Anyone else disappointed that this wasn't some in-depth, Docker related pipe usage post? For those of us that know who Jess is, the post is a little lacking. I went there to read about some cool things she's doing with pipe, but was disappointed at a post that contained nothing.

[deleted]

Re: For the Love of Pipes

#130

Post by [deleted] »

arendtio wrote: post #123
yaakushi wrote: post #85

I'm probably nitpicking, but if you're using cat to pipe a single file into the sdtin of another program, you most likely don't need the cat in the first place, you can just redirect the file to the process' stdin. Unless, of course, you're actually concatenating multiple files or maybe a file and stdin together. Disclaimer: I do cat-piping myself quite a bit out of habit, so I'm not trying to look down at the author…

In fact, I don't like people optimizing shell scripts for performance. I mean, shell scripts are slow by design and if you need something fast, you choose the wrong technology in the first place. Instead, shell script should be optimized for readability and portability and I think it is much easier to understand something like 'read | change >write' than 'change write'. So I like to write pipelines like this: cat foo…

[deleted]
Post reply on HN