For the Love of Pipes • Hacker News Board

Re: For the Love of Pipes

#81

Post by OJFord » Tue, Jan 22, 2019, 1:03 PM UTC

duckerude wrote: post #20

benj111 wrote: post #10

Why isn't the pipe a construct that has caught on in 'proper' languages?

Many functional languages have |> for piping, but chained method calls are also a lot like pipelines. Data goes from left to right. This javascript expression: [1, 2, 3].map(n => n + 1).join(',').length Is basically like this shell command: seq 3 | awk '{ print $1 + 1 }' | tr '\n' , | wc -c (the shell version gives 6 instead of 5 because of a trailing newline, but close enough)

But each successive 'command' is a method on what's constructed so far; not an entirely different command to which we delegate processing of what we have so far.

The Python:

   length(','.join(map(lambda n: n+1, range(1, 4)))

is a bit closer, but the order's now reversed, and then jumbled by the map/lambda. (Though I suppose arguably awk does that too.)

Re: For the Love of Pipes

#82

Post by alkonaut » Tue, Jan 22, 2019, 1:05 PM UTC

I love the idea of simple things that can be connected in any way. I'm not so much a fan of "everything is a soup of bytes with unspecified encoding and unknown formatting".

It's an abstraction that held up quite well, but its starting to show its age.

Re: For the Love of Pipes

#83

Post by pdkl95 » Tue, Jan 22, 2019, 1:07 PM UTC

mothsonasloth wrote: post #51

[Quote] The Unix philosophy is documented by Doug McIlroy as: Make each program do one thing well. To do a new job, build afresh rather than complicate old programs by adding new “features”. Expect the output of every program to become the input to another, as yet unknown, program. Don’t clutter output with extraneous information. Avoid stringently columnar or binary input formats. Don’t insist on interactive input.…

> The Unix philosophy is documented by Doug McIlroy as

TaoUP has a longer discussion[1] of the Unix philosophy, which includes Rob Pike's and Ken Thompson's comments on the philosophy.

[1] http://www.catb.org/esr/writings/taoup/html/ch01s06.html

"Those who don't understand Unix are condemned to reinvent it, poorly." (Henry Spencer)

Re: For the Love of Pipes

#84

Post by AnIdiotOnTheNet » Tue, Jan 22, 2019, 1:08 PM UTC

jimktrains2 wrote: post #71

pytester wrote: post #62

Earlier quoted context omitted.

The core critique - that everything is stringly typed - still holds pretty well though. >The receiving and sending processes must use a stream of bytes. Any object more complex than a byte cannot be sent until the object is first transmuted into a string of bytes that the receiving end knows how to reassemble. This means that you can’t send an object and the code for the class definition necessary to implement the ob…

To be fair, the same critisim could be used for a socket? I think the issue is that some people want pipes to be something magical that connects their software, not a dumb connection between them.

Yes. The compromise of just using an untyped byte stream in a single linear pipeline was a fair tradeoff in the 70s, but it is nearly 2020 and we can do better.

Re: For the Love of Pipes

#85

Post by yaakushi » Tue, Jan 22, 2019, 1:10 PM UTC

I'm probably nitpicking, but if you're using cat to pipe a single file into the sdtin of another program, you most likely don't need the cat in the first place, you can just redirect the file to the process' stdin. Unless, of course, you're actually concatenating multiple files or maybe a file and stdin together.

Disclaimer: I do cat-piping myself quite a bit out of habit, so I'm not trying to look down at the author or anything like that! :)

Re: For the Love of Pipes

#86

Post by kingosticks » Tue, Jan 22, 2019, 1:15 PM UTC

MichaelMoser123 wrote: post #35

kingosticks wrote: post #32

Earlier quoted context omitted.

While I do agree they can sometimes be tricky to use, Gstreamer pipelines are also it's best feature. Is there a better way?

ffmpeg took a different approach (sort of like a single program/system with many command line options). Some say its easier to use (at least easier than gst-launch)

I find the ffmpeg options dizzying whereas the single syntax for gst-launch and parse_bin_from_description is pretty neat. But then I guess you've still got a selection of randomly named properties to discover and correctly set.

Re: For the Love of Pipes

#87

Post by AnIdiotOnTheNet » Tue, Jan 22, 2019, 1:16 PM UTC

MichaelMoser123 wrote: post #22

I think the Unix pipeline works because this it has this moldable and expressive text substance that is exchanged. Gstreamer also has pipelines, but the result in my opinion is quite awkward because events of many types are exchanged. Windows Powershell also has a pipeline where objects are exchanged, but it also somehow failed to become a huge success. i think the unix pipeline concept doesn't quite scale to other d…

PowerShell is a huge success, just not in the webdev echo chamber because it isn't exactly like the same old UNIX tools everyone is familiar with.

Re: For the Love of Pipes

#88

Post by AnIdiotOnTheNet » Tue, Jan 22, 2019, 1:17 PM UTC

agumonkey wrote: post #39

has there been proposals/research on having non linear / branching pipe syntax ? a |1 b | c | d, |1 e aka piping a to both b,c,d and e branches ?

I don't think the commandline is well suited to constructing or reading anything more complicated than a linear pipe. What you'd want for that kind of workflow is a 2d layout.

Re: For the Love of Pipes

#89

Post by amdavidson » Tue, Jan 22, 2019, 1:21 PM UTC

OliverJones wrote: post #64

bagrow wrote: post #33

Earlier quoted context omitted.

> When was the last time your Unix workstation was as useful as a Macintosh? Some of that discussion has not aged well :)

MacOs is layered on a UNIX-like OS. You can use pipes in your command windows.

The Unix Haters Handbook was published in 1994, when System 7 was decidedly not unix-like.

Re: For the Love of Pipes

#90

Post by DavidWoof » Tue, Jan 22, 2019, 1:22 PM UTC

OliverJones wrote: post #64

bagrow wrote: post #33

Earlier quoted context omitted.

> When was the last time your Unix workstation was as useful as a Macintosh? Some of that discussion has not aged well :)

MacOs is layered on a UNIX-like OS. You can use pipes in your command windows.

This comment makes me feel really old.

MacOs wasn't always layered on unix, and the unix-haters' handbook predates the switch to the unix-based MacOs X.