Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pipelining might be my favorite programming language feature

herecomesthemoon.net

Post reply on HN
Print view View original

331–340 of 360 posts

Re: Pipelining might be my favorite programming language feature

#331

Post by jlkuester7 »

veidr wrote: post #262
refactor_master wrote: post #240

Earlier quoted context omitted.

> Despite being clean and readable, I don't tend to do it any more, because it's harder to debug. More often these days, I write things like this: data = File.readlines("haystack.txt") data = data.map(&:strip) data = data.grep(/needle/) data = data.map { |i| i.gsub('foo', 'bar') } data = data.map { |i| File.readlines(i).count } Hard disagree. It's less readable, the intend is unclear (where does it end?), and the var…

The person you are quoting already conceded that is less readable, but that the ability to set a breakpoint easily (without having to stop the process and modify the code) is more important. I myself agree, and find myself doing that too, especially in frontend code that executes in a browser. Debuggability is much more important than marginally-better readability, for production code.

> Debuggability is much more important than marginally-better readability, for production code.

I find this take surprising. I guess it depends on how much weight you give to "marginally-better", but IMHO readability is the single most important factor when it comes to writing code in most code-bases. You write code once, it may need to be debugged (by yourself or others) on rare occasions. However anytime anyone needs to understand the code (to update it, debug it, or just make changes in adjacent code) they will have to read it. In a shared code-base your code will be read many more times than it will be updated/debugged.

Re: Pipelining might be my favorite programming language feature

#332

Post by AlchemistCamp »

freehorse wrote: post #206
MaxBarraclough wrote: post #136

Earlier quoted context omitted.

With the disclaimer that I don't know Elixir and haven't programmed with the pipeline operator before: I don't like that special () syntax. That syntax denotes application of the function without passing any arguments, but the whole point here is that an argument is being passed. It seems clearer to me to just put the pipeline operator and the name of the function that it's being used with. I don't see how it's uncle…

I am also confused with such syntax of "passing as first argument" pipes. Having to write `x |> foo` instead of `x |> foo()` does not solve much, because you have the same lack of clarity if you need to pass a second argument. Ie `x |> foo(y)` in this case means `foo(x,y)`, but if `foo(y)` actually gives you a function to apply to `x` prob you should write `x |> foo(y)()` or `x |> (foo(y))()` then as I understand it?…

It would be silly to use a pipeline for x |> foo(). What's nice is being able to write:

    def main_loop(%Game{} = game) do
      game
      |> get_move()
      |> play_move()
      |> win_check()
      |> end_turn()
    end
instead of the much harder to read:

    def main_loop(%Game{} = game)
        end_turn(win_check(play_move(get_move(game))))
    end
For an example with multiple parameters, this pipeline:

    schema
    |> order_by(^constraint)
    |> Repo.all()
    |> Repo.preload(preload_opts)
would be identical to this:

    Repo.preload(Repo.all(order_by(schema, ^constraint)), preload_opts)
To address your question above,

> if `foo(y)` actually gives you a function to apply to `x` prob you should write `x |> foo(y)()`

If foo(y) returned a function, then to call it with x, you would have to write foo(y).(x) or x |> foo(y).(), so the syntax around calling the anonymous function isn't affected by the pipe. Also, you're not generally going to be using pipelines with functions that return functions so much as with functions that return data which is then consumed as the first argument by the next function in the pipeline. See my previous comment on this thread for more on that point.

There's no inconsistency or ambiguity in the pipeline operator's behavior. It's just syntactic sugar that's handy for making your code easier to read.

Re: Pipelining might be my favorite programming language feature

#333

Post by _heimdall »

alganet wrote: post #325
_heimdall wrote: post #306

Earlier quoted context omitted.

I may have just misunderstood the OP. It sounded to me like describing the benefits specifically of transducers, but if it was OOP and more just about piping operators or chaining the term wouldn't fit.

Yes, you totally misunderstood.

Yep not sure how I totally misread it here. Looking back they're describing currying.

I've used languages and libraries that call it piping, ramda has a .pipe() method for example. Don't think I've ever seen it called pipelining but I see how you could get there.

Re: Pipelining might be my favorite programming language feature

#334

Post by maleldil »

nine_k wrote: post #319
invalidator wrote: post #205

The author keeps calling it "pipelining", but I think the right term is "method chaining". Compare with a simple pipeline in bash: grep needle Each of those components executes in parallel, with the intermediate results streaming between them. You get a similar effect with coroutines. Compare Ruby: data = File.readlines("haystack.txt") .map(&:strip) .grep(/needle/) .map { |i| i.gsub('foo', 'bar') } .map { |i| File.re…

In Python, such steps like map() and filter() would execute concurrently, without large intermediate arrays. It lacks the chaining syntax for them, too. Java streams are the closest equivalent, both by the concurrent execution model, and syntactically. And yes, the Java debugger can show you the state of the intermediate streams.

> would execute concurrently

Iterators are not (necessarily) concurrent. I believe you mean lazily.

Re: Pipelining might be my favorite programming language feature

#335

Post by nine_k »

maleldil wrote: post #334
nine_k wrote: post #319

Earlier quoted context omitted.

In Python, such steps like map() and filter() would execute concurrently, without large intermediate arrays. It lacks the chaining syntax for them, too. Java streams are the closest equivalent, both by the concurrent execution model, and syntactically. And yes, the Java debugger can show you the state of the intermediate streams.

> would execute concurrently Iterators are not (necessarily) concurrent. I believe you mean lazily.

Concurrent, not parallel.

That is, iterators' execution flow is interspersed, with the `yield` statement explicitly giving control to another coroutine, and then continuing the current coroutine at another yield point, like the call to next(). This is very similar to JS coroutines implemented via promises, with `await` yielding control.

Even though there is only one thread of execution, the parts of the pipeline execute together in lockstep, not sequentially, so there's no need for a previous part to completely compute a large list before the following part can start iterating over it.

Re: Pipelining might be my favorite programming language feature

#336

Post by runeks »

invalidator wrote: post #205

The author keeps calling it "pipelining", but I think the right term is "method chaining". Compare with a simple pipeline in bash: grep needle Each of those components executes in parallel, with the intermediate results streaming between them. You get a similar effect with coroutines. Compare Ruby: data = File.readlines("haystack.txt") .map(&:strip) .grep(/needle/) .map { |i| i.gsub('foo', 'bar') } .map { |i| File.re…

> The author keeps calling it "pipelining", but I think the right term is "method chaining".

Allow me, too, to disagree. I think the right term is "function composition".

Instead of writing

  h(g(f(x)))
as a way to say "first apply f to x, after which g is applied to the result of this, after which h is applied to the result of this", we can use function composition to compose f, g and h, and then "stuff" the value x into this "pipeline of composed functions".

We can use whatever syntax we want for that, but I like Elm syntax which would look like:

  x |> f >> g >> h

Re: Pipelining might be my favorite programming language feature

#337

Post by alganet »

_heimdall wrote: post #333
alganet wrote: post #325

Earlier quoted context omitted.

Yes, you totally misunderstood.

Yep not sure how I totally misread it here. Looking back they're describing currying. I've used languages and libraries that call it piping, ramda has a .pipe() method for example. Don't think I've ever seen it called pipelining but I see how you could get there.

The kid goes to the zoo and sees a tiger. It says: "look, a big cat!"

Then the zookeeper angrily beats down the kid while screaming: "You stupid moron, that's a Panthera tigris"

His father instead, buys him a book that say "tiger" and has some cool illustrations.

Re: Pipelining might be my favorite programming language feature

#339

Post by manmal »

smallerize wrote: post #216
manmal wrote: post #170

Earlier quoted context omitted.

I wish there were a variation that can destructure more ergonomically. Instead of: ``` fetch_data() |> (fn {:ok, val, _meta} -> val :error -> "default value" end).() |> String.upcase() ``` Something like this: ``` fetch_data() |>? {:ok, val, _meta} -> val |>? :error -> "default value" |> String.upcase() ```

fetch_data() |> case do {:ok, val, _meta} -> val :error -> "default value" end You have the extra "case do...end" block but it's pretty close? This is for sequential conditions. If you have nested conditions, check out a where block instead. https://dev.to/martinthenth/using-elixirs-with-statement-5e3...

Thanks, that looks good!

Re: Pipelining might be my favorite programming language feature

#340

Post by tasuki »

RHSeeger wrote: post #119
tasuki wrote: post #105

Earlier quoted context omitted.

Oh wow, are we living in the same universe? To me the one-line example and your example with line breaks... they just... look about the same? See how adding line breaks still keeps the `|w| w.alive` very far from the `filter` call? And the `|w| w.id` very far from the `map` call? If you don't have the pipeline operator, please at least format it something like this: fn get_ids(data: Vec ) -> Vec { collect( map( filte…

> It's not about line breaks, it's about the order of applying the operations For me, it's both. Honestly, I find it much less readable the way you're split it up. The way I had it makes it very easy for me to read it in reverse; map, filter, map, collect > Also see how this still reads fine despite being one line It doesn't read fine, to me. I have to spend mental effort figuring out what the various "steps" are. Ef…

> The way I had it makes it very easy for me to read it in reverse; map, filter, map, collect

Yes, sure, who cares. But the way you wrote it, it's impossible to match those "map, filter, map, collect" to their parameters: `, |w| w.toWingding()), |w| w.alive), |w| w.id))`. Impossible!

You just include all the parameters to the various function calls on one very long line! To top it off, it's the most indented line that you decide to make the longest! If I could I'd put you in jail for this!

Post reply on HN