Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pipelining might be my favorite programming language feature

herecomesthemoon.net

Post reply on HN
Print view View original

311–320 of 360 posts

Re: Pipelining might be my favorite programming language feature

#311

Post by bjoli »

axblount wrote: post #209
invalidator wrote: post #205

The author keeps calling it "pipelining", but I think the right term is "method chaining". Compare with a simple pipeline in bash: grep needle Each of those components executes in parallel, with the intermediate results streaming between them. You get a similar effect with coroutines. Compare Ruby: data = File.readlines("haystack.txt") .map(&:strip) .grep(/needle/) .map { |i| i.gsub('foo', 'bar') } .map { |i| File.re…

Syntactic sugar can sometimes fool us into thinking the underlying process is more efficient or streamlined. As a new programmer, I probably would have assumed that "storing" `data` at each step would be more expensive.

Reading this, I am so happy that my first language was a scheme where I could see the result of the first optimization passes.

This helped me quickly develop a sense for how code is optimized and what code is eventually executed.

Re: Pipelining might be my favorite programming language feature

#312

Post by AndyKluger »

bjourne wrote: post #292

In concatenative languages with an implicit stack (Factor) that expression would read: iter [ alive? ] filter [ id>> ] map collect The beauty of this is that everything can be evaluated strictly left-to-right. Every single symbol. "Pipelines" in other languages are never fully left-to-right evaluated. For example, ".filter(|w| w.alive)" in the author's example requires one to switch from postfix to infix evaluation t…

Thank you!

Because I love to practice and demonstrate Factor, this is working code for that example:

  "f1.txt" "f2.txt" [ utf8 file-lines [ string>number ] map ] bi@ vdot

Re: Pipelining might be my favorite programming language feature

#313

Post by mvieira38 »

bnchrch wrote: post #24

I'm personally someone who advocates for languages to keep their feature set small and shoot to achieve a finished feature set quickly. However. I would be lying if I didn't secretly wish that all languages adopted the `|>` syntax from Elixir. ``` params |> Map.get("user") |> create_user() |> notify_admin() ```

R has a lovely toolkit for data science using this syntax, called the tidyverse. My favorite dev experience, it's so easy to just write code

Re: Pipelining might be my favorite programming language feature

#314

Post by mvieira38 »

SimonDorfman wrote: post #2

The tidyverse folks in R have been using that for a while: https://magrittr.tidyverse.org/reference/pipe.html

R + tidyverse is the gold standard for working with data quickly in a readable and maintainable way, IMO. It's just absolutely seamless. Shoutout to tidyverts (https://tidyverts.org/) for working with time series, too

Re: Pipelining might be my favorite programming language feature

#315

Post by gregabbott »

Cyykratahk wrote: post #36
bnchrch wrote: post #24

I'm personally someone who advocates for languages to keep their feature set small and shoot to achieve a finished feature set quickly. However. I would be lying if I didn't secretly wish that all languages adopted the `|>` syntax from Elixir. ``` params |> Map.get("user") |> create_user() |> notify_admin() ```

We might be able to cross one more language off your wishlist soon, Javascript is on the way to getting a pipeline operator, the proposal is currently at Stage 2 https://github.com/tc39/proposal-pipeline-operator I'm very excited for it.

A while ago, I wondered how close you could get to a pipeline operator using existing JavaScript features. In case anyone might like to have a look, I wrote a proof-of-concept function called "Chute" [1]. It chains function and method calls in a dot-notation style like the basic example below.

  chute(7)        // setup a chute and give it a seed value
  .toString       // call methods of the current data (parens optional)
  .parseInt       // send the current data through global native Fns
  .do(x=>[x])     // through a chain of one or more local / inline Fns
  .JSON.stringify // through nested global functions (native / custom)
  .JSON.parse
  .do(x=>x[0])
  .log            // through built in Chute methods
  .add_one        // global custom Fns (e.g. const add_one=x=>x+1)
  ()              // end a chute with '()' and get the result
[1] https://chute.pages.dev/ | https://github.com/gregabbott/chute

Re: Pipelining might be my favorite programming language feature

#316

Post by AndyKluger »

layer8 wrote: post #112

The one thing that I don’t like about pipelining (whether using a pipe operator or method chaining), is that assigning the result to a variable goes in the wrong direction, so to speak. There should be an equivalent of the shell’s `>` for piping into a variable as the final step. Of course, if the variable is being declared at the same time, whatever the concrete syntax is would still require some getting used to, be…

FWIW in Factor you can set dynamic variables with

  "coolvalue" thisisthevar set
or if you use the `variables` vocab, alternately:

  "coolvalue" set: thisisthevar
and lexical variables are set with

  "coolvalue" :> thisisthevar

Re: Pipelining might be my favorite programming language feature

#317

Post by AndyKluger »

dpc_01234 wrote: post #162

I think there's a language syntax to be invented that would make everything suffix/pipeline-based. Stack based languages are kind of there, but I don't think exactly the same thing. BTW. For people complaining about debug-ability of it: https://doc.rust-lang.org/std/iter/trait.Iterator.html#metho... etc.

What do you think stack based languages like Factor miss in this regard?

Re: Pipelining might be my favorite programming language feature

#318

Post by matt_kantor »

trealira wrote: post #197
the_sleaze_ wrote: post #89

Earlier quoted context omitted.

In my eyes newlines don't solve what I feel to be the issue. Reader needs to recognize reading from left->right to right->left. Of course this really only matters when you're 25 minutes into critical downtime and a bug is hiding somewhere in these method chains. Anything that is surprising needs to go. IMHO it would be better to set intermediate variables with dead simple names instead of newlines. fn get_ids(data: V…

> Reader needs to recognize reading from left->right to right->left. Yeah, I agree. The problem is that you have to keep track of nesting in the middle of the expression and then unnest it at the end, which is taxing. So, I also think it could also read better written like this, with the arguments reversed, so you don't have to read it both ways: fn get_ids(data: Vec ) -> Vec { collect( map(|w| w.id, filter |w| w.ali…

I've been prototyping a programming language[0] with Haskell-like function conventions (all functions are unary and the "primary" parameter comes last). I recently added syntax to allow applying any "binary" function using infix notation, with `a f b` being the same as `f(b)(a)`[1]. Argument order is swapped compared to Haskell's infix notation (where `a f b` would desugar to `f(a)(b)`).

Along with the `|>` operator (which is itself just a function that's conventionally infixed), this turns out to be really nice for flexibility/reusability. All of these programs do the same thing:

  1 - 2 - 3 + 4

  1
    |> -(2)
    |> -(3)
    |> +(4)

  +(4)(
    -(3)(
      -(2)(1)
    )
  )
It was extremely satisfying to discover that with this encoding, `|>` is simply an identity function!

[0]: https://github.com/mkantor/please-lang-prototype

[1]: In reality variable dereferencing uses a sigil, but I'm omitting it from this comment to keep the examples focused.

Re: Pipelining might be my favorite programming language feature

#319

Post by nine_k »

invalidator wrote: post #205

The author keeps calling it "pipelining", but I think the right term is "method chaining". Compare with a simple pipeline in bash: grep needle Each of those components executes in parallel, with the intermediate results streaming between them. You get a similar effect with coroutines. Compare Ruby: data = File.readlines("haystack.txt") .map(&:strip) .grep(/needle/) .map { |i| i.gsub('foo', 'bar') } .map { |i| File.re…

In Python, such steps like map() and filter() would execute concurrently, without large intermediate arrays. It lacks the chaining syntax for them, too.

Java streams are the closest equivalent, both by the concurrent execution model, and syntactically. And yes, the Java debugger can show you the state of the intermediate streams.

Re: Pipelining might be my favorite programming language feature

#320

Post by immibis »

We had this - it was called variables. You could do:

x = iter(data);

y = filter(x, w=>w.isAlive);

z = map(y, w=>w.id);

return collect(z);

It doesn't need new syntax, but to implement this with the existing syntax you do have to figure out what the intermediate objects are, but you also have that problem with "pipelining" unless it compiles the whole chain into a single thing a la Linq.

Post reply on HN