Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What Unix pipelines got right and how we can do better

programmingsimplicity.substack.com

Post reply on HN
Print view View original

11–20 of 22 posts

Re: What Unix pipelines got right and how we can do better

#11

Post by kazinator »

Unix pipelines got something right by being a syntactic sugar for chaining pure function application. It's easy to get excited when you don't understand this.

For instance sqrt(sin(cos(theta))) can be notated Pipeline syntax implemented in functional languages expands into chained function invocation.

Everything follows from that: what we know about combining functions applies to pipes.

> When cat writes to stdout, it doesn't block waiting for grep to process that data.

That says nothing more than that nested function invocations admit non-strict evaluation strategies. E.g. the argument of a function need not be reduced to a value before it is passed to another, which can proceed with a calculation which depends on that result before obtaining it.

When you expand the actual data dependencies into a tree, it's obvious to see what can be done in parallel.

Re: What Unix pipelines got right and how we can do better

#12

Post by hnlmorg »

> The lack of fan-out makes it awkward to express combinations where one sender feeds many receivers. In 1970, avoiding garbage collection was a practical necessity, but today garbage collection is available in most programming workflows and fan-out could be implemented much more easily through message copying rather than consumption.

Fanout has precisely zero dependency on GC. For example ‘tee’ has been around for decades and it can copy io streams just fine.

There has been some effort to built fanout shells too. With a discussion in HN earlier this month on one called dgsh https://news.ycombinator.com/item?id=45425298

Edit: I agree with other comments that this feels like AI slop

Re: What Unix pipelines got right and how we can do better

#13

Post by Joker_vD »

userbinator wrote: post #8

When cat writes to stdout, it doesn't block waiting for grep to process that data. It will certainly do that if the buffer is full. prevents the implicit blocking No, that's exactly the case of implicit blocking mentioned above. Does anyone else find this article rather AI-ish? The extreme verbosity and repetitiveness, the use of dashes, and "The limitation isn't conceptual—it's syntactic" are notable artifacts.

If anything, the pre-pipe style of

    prog1 -input input_file -output tmp1_file
    prog2 -input tmp1_file -output tmp2_file && del tmp1_file
    prog3 -input tmp2_file -output tmp1_file && del tmp2_file
    ...
    progN -input tmpX_file -output output_file && del tmpX_file
is more in line with the author's claimed benefits of the pipes than the piped style itself. The process isolation is absolute, they are separated not just in space, but in time as well, entirely!

Re: What Unix pipelines got right and how we can do better

#14

Post by 1718627440 »

userbinator wrote: post #8

When cat writes to stdout, it doesn't block waiting for grep to process that data. It will certainly do that if the buffer is full. prevents the implicit blocking No, that's exactly the case of implicit blocking mentioned above. Does anyone else find this article rather AI-ish? The extreme verbosity and repetitiveness, the use of dashes, and "The limitation isn't conceptual—it's syntactic" are notable artifacts.

> It will certainly do that if the buffer is full.

You can consider that an OS/resource specific limitation, rather than a limitation in the concept.

Re: What Unix pipelines got right and how we can do better

#16

Post by geysersam »

userbinator wrote: post #8

When cat writes to stdout, it doesn't block waiting for grep to process that data. It will certainly do that if the buffer is full. prevents the implicit blocking No, that's exactly the case of implicit blocking mentioned above. Does anyone else find this article rather AI-ish? The extreme verbosity and repetitiveness, the use of dashes, and "The limitation isn't conceptual—it's syntactic" are notable artifacts.

> Does anyone else find this article rather AI-ish?

After reading the whole thing, yes! Specifically it feels incoherent in the way AI text often is. It starts by praising unix pipes for their simple design and the explicit tradeoffs they make, and then proceeds explaining how we could and should make the complete opposite set of tradeoffs.

Re: What Unix pipelines got right and how we can do better

#17

Post by Joker_vD »

1718627440 wrote: post #14
userbinator wrote: post #8

When cat writes to stdout, it doesn't block waiting for grep to process that data. It will certainly do that if the buffer is full. prevents the implicit blocking No, that's exactly the case of implicit blocking mentioned above. Does anyone else find this article rather AI-ish? The extreme verbosity and repetitiveness, the use of dashes, and "The limitation isn't conceptual—it's syntactic" are notable artifacts.

> It will certainly do that if the buffer is full. You can consider that an OS/resource specific limitation, rather than a limitation in the concept.

Nah. Having built-in automatic backpressure is one of the most underappreciated things about the UNIX pipes.

Re: What Unix pipelines got right and how we can do better

#18

Post by 1718627440 »

geysersam wrote: post #16
userbinator wrote: post #8

When cat writes to stdout, it doesn't block waiting for grep to process that data. It will certainly do that if the buffer is full. prevents the implicit blocking No, that's exactly the case of implicit blocking mentioned above. Does anyone else find this article rather AI-ish? The extreme verbosity and repetitiveness, the use of dashes, and "The limitation isn't conceptual—it's syntactic" are notable artifacts.

> Does anyone else find this article rather AI-ish? After reading the whole thing, yes! Specifically it feels incoherent in the way AI text often is. It starts by praising unix pipes for their simple design and the explicit tradeoffs they make, and then proceeds explaining how we could and should make the complete opposite set of tradeoffs.

Also the headings are just sprinkled at intervals and don't really fit the text.

Re: What Unix pipelines got right and how we can do better

#19

Post by 1718627440 »

Joker_vD wrote: post #17
1718627440 wrote: post #14

Earlier quoted context omitted.

> It will certainly do that if the buffer is full. You can consider that an OS/resource specific limitation, rather than a limitation in the concept.

Nah. Having built-in automatic backpressure is one of the most underappreciated things about the UNIX pipes.

Fully agree. This is still a representation of the available resources.

Re: What Unix pipelines got right and how we can do better

#20

Post by kej »

geysersam wrote: post #16
userbinator wrote: post #8

When cat writes to stdout, it doesn't block waiting for grep to process that data. It will certainly do that if the buffer is full. prevents the implicit blocking No, that's exactly the case of implicit blocking mentioned above. Does anyone else find this article rather AI-ish? The extreme verbosity and repetitiveness, the use of dashes, and "The limitation isn't conceptual—it's syntactic" are notable artifacts.

> Does anyone else find this article rather AI-ish? After reading the whole thing, yes! Specifically it feels incoherent in the way AI text often is. It starts by praising unix pipes for their simple design and the explicit tradeoffs they make, and then proceeds explaining how we could and should make the complete opposite set of tradeoffs.

That would explain the strangeness of the recent spherical cows article from the same site, as well.
Post reply on HN