Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

For the Love of Pipes

blog.jessfraz.com

Post reply on HN
Print view View original

71–80 of 323 posts

Re: For the Love of Pipes

#71

Post by jimktrains2 »

pytester wrote: post #62
bagrow wrote: post #33

Earlier quoted context omitted.

> When was the last time your Unix workstation was as useful as a Macintosh? Some of that discussion has not aged well :)

The core critique - that everything is stringly typed - still holds pretty well though. >The receiving and sending processes must use a stream of bytes. Any object more complex than a byte cannot be sent until the object is first transmuted into a string of bytes that the receiving end knows how to reassemble. This means that you can’t send an object and the code for the class definition necessary to implement the ob…

To be fair, the same critisim could be used for a socket? I think the issue is that some people want pipes to be something magical that connects their software, not a dumb connection between them.

Re: For the Love of Pipes

#73

Post by majewsky »

benj111 wrote: post #45
jgtrosh wrote: post #24

Earlier quoted context omitted.

Well you can't read either from /dev/null, and I don't think that's just a question of permissions. I'm pretty sure it's impossible to get /dev/null to behave like an executable.

Interesting question. You could write a executable that accepts piped input and throws it away. When would it exit though? Would it exit successfully at the end of the input stream? That sounds sensible. That would be behaving like an executable wouldn't it?

> When would it exit though? Would it exit successfully at the end of the input stream?

A process that attempts to read from a closed pipe receives SIGPIPE. The default disposition for SIGPIPE is to terminate the program (similar to SIGTERM or SIGINT). So yeah, assuming that the previous program in the pipeline closes its stdout at some point (either explicitly, or implicitly by just exiting), then our program would die of SIGPIPE the when it tries to read() from stdin and the pipe's buffer has been depleted.

However, our program could also set SIGPIPE to ignored and ignore the EPIPE errors that read() would return in that case. In that case, it could run indefinitely. But at this point, you're way past normal behavior.

Re: For the Love of Pipes

#75

Post by kitd »

jacquesm wrote: post #65
lincpa wrote: post #58

The Pure Function Pipeline Data Flow https://github.com/linpengcheng/PurefunctionPipelineDataflow Using the input and output characteristics of pure functions, pure functions are used as pipelines. Dataflow is formed by a series of pure functions in series. A dataflow code block as a function, equivalent to an integrated circuit element (or board)。 A complete integrated system is formed by serial or parallel dataflow…

Yes, there is a strong connection between pipes and functional programming, they all transform what passes through them and should not retain state when implemented properly.

They also both rely on very generic data types as input and output. Which is why I was surprised to see the warning about avoiding columnar data. Tabular data is a basic way of expressing data and their relationships.

Re: For the Love of Pipes

#76

Post by leni536 »

agumonkey wrote: post #39

has there been proposals/research on having non linear / branching pipe syntax ? a |1 b | c | d, |1 e aka piping a to both b,c,d and e branches ?

I've seen an idea of a generic utility that does this on moreutils' main page[1]. Search for 'connect'.

[1] https://joeyh.name/code/moreutils/

Re: For the Love of Pipes

#77

Post by glenneroo »

quietbritishjim wrote: post #54
mothsonasloth wrote: post #51

[Quote] The Unix philosophy is documented by Doug McIlroy as: Make each program do one thing well. To do a new job, build afresh rather than complicate old programs by adding new “features”. Expect the output of every program to become the input to another, as yet unknown, program. Don’t clutter output with extraneous information. Avoid stringently columnar or binary input formats. Don’t insist on interactive input.…

Reformatted to be readable: > Make each program do one thing well. To do a new job, build afresh rather than complicate old programs by adding new “features”. > Expect the output of every program to become the input to another, as yet unknown, program. Don’t clutter output with extraneous information. Avoid stringently columnar or binary input formats. Don’t insist on interactive input. > Design and build software, e…

Why is this or OP's even necessary? The bullet points are copied directly from the article.

Re: For the Love of Pipes

#78

Post by jarpineh »

avodonosov wrote: post #38
jarpineh wrote: post #18

I recently came across Ramda CLI's interactive mode [1] It essentially hijacks pipe's input and output into browser where you can play with the Ramda command. Then you just close browser tab and Ramda CLI applies your changed code in the pipe, resuming its operation. Now I'm thinking all kinds ways I use pipe that I could "tee" through a browser app. I can use browser for interactive JSON manipulation, visualization…

You can just store the first pipeline results in a file, edit it, then use the file as an input for the second pipeline.

Well, yes, but this kind defeats transient nature of data moving through pipe. Testing and debugging and operating on pipe based processing benefits from close feedback cycle. I’d rather keep that as much as possible.

Re: For the Love of Pipes

#79

Post by [deleted] »

alias_neo wrote: post #2

Anyone else disappointed that this wasn't some in-depth, Docker related pipe usage post? For those of us that know who Jess is, the post is a little lacking. I went there to read about some cool things she's doing with pipe, but was disappointed at a post that contained nothing.

[deleted]

Re: For the Love of Pipes

#80

Post by barrkel »

chriswarbo wrote: post #70
barrkel wrote: post #26

Earlier quoted context omitted.

Monads are effectively pipes; the monad controls how data flows through the functions you put into the monad, but the functions individually are like individual programs in a pipe.

I'm not sure I agree with this. Function composition is more directly comparable to pipes, whereas I tend to think of monads as collapsing structure (i.e. `join :: m (m a) -> m a`)

Some of the most common monads (especially lists and other collection-like things) feel very much like pipes.

    list.select { |x|  x.foo > 10 }.map { |x| x.bar }...
Etc.

I wouldn't make the same argument about the IO monad, which I think more in terms of a functional program which evaluates to an imperative program. But most monads are not like the IO monad, in my experience at least.

Post reply on HN