Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why Rust closures are somewhat hard

stevedonovan.github.io

Post reply on HN
Print view View original

71–80 of 92 posts

Re: Why Rust closures are somewhat hard

#71

Post by Sean1708 »

Diggsey wrote: post #25
tomjakubowski wrote: post #23

Earlier quoted context omitted.

Lambdas have unique types, and you can't use a generic parameter in a return type (which is impl Trait's raison d'etre), so I think it would have to be this: fn compose T, G: Fn(T) -> T>(f: F, g: G) -> impl Fn(T) - > T or: fn compose (f: F, g: G) -> impl Fn(T) -> T where F: Fn(T) -> T, G: Fn(T) -> T

Maybe you just miscommunicated, but you absolutely can use generic parameters in a return type: https://doc.rust-lang.org/nightly/std/iter/trait.Iterator.ht... ... and `impl Trait` exists to be able to have return types which are difficult, verbose, or impossible to name.

It's impossible to write a compose function with the declaration

  fn compose T>(f1: F, f2: F) -> F
As far as I'm aware the only way to write a function like that is

  fn compose T>(f1: F, f2: F) -> F {
    f1
  }
and even then you can only call it like this

  let clos = |x| 2 * x;
  compose(clos, clos);

Re: Why Rust closures are somewhat hard

#72

Post by lifthrasiir »

zelly wrote: post #48

Everything that's hard in Rust could be solved by GC. With Rust, programmers have to spend most of their mental energy worrying about management of memory, which has largely been automated already. I always go back to this quote from Andrei Alexandrescu (creator of D): A disharmonic personality. Reading any amount of Rust code evokes the joke "friends don't let friends skip leg day" and the comic imagery of men with…

First of all, Andrei Alexandrescu is not the creator of D.

> With Rust, programmers have to spend most of their mental energy worrying about management of memory, which has largely been automated already.

This is a common misunderstanding. I think the word "automated" is misleading, and suggest that we should look at two phases of memory management: the plan and the execution.

In traditional low-level languages like C, you plan how to manage memory and you write code to execute that plan yourself. Your code may not match the plan, thus all sort of problems ensues.

In languages with GC, you are allowed to not think how to manage memory and GC will execute their own inferred plan... until it is not. At some point you are forced to think how to manage memory and also tune GC and/or code to fit to your belated plan. This point may not be reached (simple scripts) or can be delayed much further (good modern GC), of course, so it still remains very useful.

In (a typical use case of) Rust, you plan how to manage memory but give that plan to the compiler to check if the plan is executed correctly. The compiler also generate code for common executions that you don't have to. So the execution is mostly automated (or rather, delegated), while planning is not.

A common theme here is that you can't avoid at least the planning once you've got past a threshold. If GC would supplant even that planning, you won't need value types at all; isn't that what the generation hypothesis is all about? But many languages with GC but not value types now get them. While it can be argued that Rust's affine typing is not sufficient to capture some common memory management patterns [1], that fact doesn't diminish the value of explicit memory management planning, and thus Rust's.

[1] And I think it is actually true! Indeed, I believe affine typing plus GC can be actually better than affine typing or GC alone (and value types mimic a good subset of this). There are some Rust libraries for tracing GC (e.g. [2]) as a starting point.

[2] https://github.com/Manishearth/rust-gc

Re: Why Rust closures are somewhat hard

#73

Post by zozbot234 »

imtringued wrote: post #64
zelly wrote: post #48

Everything that's hard in Rust could be solved by GC. With Rust, programmers have to spend most of their mental energy worrying about management of memory, which has largely been automated already. I always go back to this quote from Andrei Alexandrescu (creator of D): A disharmonic personality. Reading any amount of Rust code evokes the joke "friends don't let friends skip leg day" and the comic imagery of men with…

Gargabe collection is a much bigger crutch. Once you use it your entire program is contaminated by the performance characteristics of a single improperly written function. You cannot mix real time code with non real time code. If you do your real time code will be delayed by the non real time code. The unpredictability of the GC has become a property of your entire application.

Yes, but you can have other kinds of cycle-aware memory management, which integrate better with their surrounding code than plain tracing GC. E.g. something like https://github.com/artichoke/cactusref which can collect cycles deterministically, and does not have to trace the whole heap to do so (which implies a lot of hidden overhead in plain GC, because all heap-allocated structures must then be legible to the tracing routine).

Re: Why Rust closures are somewhat hard

#74

Post by mathw »

lmm wrote: post #60
YatoRust wrote: post #26

Earlier quoted context omitted.

Yes, I think he meant that it would be a bad idea to lock all of the type parameters to the same type, because all closures have different types. So you wouldn't be able to do something like this compose(|x| x, |x| x) because the closures have the same type.

Because the closures don't have the same type, if I'm following you correctly.

Because the parameters demand the same type (and for the return value), but the supplied parameters are not the same type because the only way to get that is to pass in exactly the same instance of the closure.

It's a fun little thing, and a good example of why impl Trait in argument position is a really nice addition to Rust, even though based on what we were originally excited about (impl Trait in return position for returning Iterators and other such things), the argument position form didn't seem so important.

Re: Why Rust closures are somewhat hard

#75

Post by Sean1708 »

saghm wrote: post #30
jimbob45 wrote: post #8

I don't understand the intuition of closures and they turn me off to languages immediately. They feel like a hack from someone who didn't want to store a copy of a parent-scope variable within a function. The idea that I can touch variables that have gone out of scope (and that have ostensibly been GC'd) makes me feel that it is impossible to reason about variable lifetimes when dealing with closures. Is there some p…

> The idea that I can touch variables that have gone out of scope (and that have ostensibly been GC'd) I think this might be due to your mental model of GC not quite matching the way it's usually implemented (at least, in traditional GC, as opposed to reference counting); most GC gives no guarantees that memory will immediately be cleaned up once the last reference goes out of scope, just that it will happen some tim…

> most GC gives no guarantees that memory will immediately be cleaned up once the last reference goes out of scope

But in this example the last reference hasn't gone out of scope, the last reference is in the closure. If the GC ignored closures and acted as if last reference had gone out of scope then there's a chance that that variable would be cleaned up before the closure gets called, which would make closures completely useless.

Re: Why Rust closures are somewhat hard

#76

Post by zozbot234 »

jimbob45 wrote: post #8

I don't understand the intuition of closures and they turn me off to languages immediately. They feel like a hack from someone who didn't want to store a copy of a parent-scope variable within a function. The idea that I can touch variables that have gone out of scope (and that have ostensibly been GC'd) makes me feel that it is impossible to reason about variable lifetimes when dealing with closures. Is there some p…

> Is it literally just the language invisibly adding the parent-scope variables and their values to the top of my function?

Yup. A "first class" closure is just reifying some arguments of a future function call and bundling them up nicely in a struct/record. This is especially clear in languages that support currying, since that means you'll literally be building that record step-by-step - only after a full set of arguments is provided does a true function call really occur.

Re: Why Rust closures are somewhat hard

#77

Post by zelly »

Sean1708 wrote: post #69
millstone wrote: post #63

Earlier quoted context omitted.

It’s a real issue. Consider (as you say) the Rust compiler. It has an IR phase called MIR, which is a tree, and elements in the tree need to know how to find themselves in the parent. For example, a function has a set of basic blocks, and each BB needs to know about its function. This is a very typical IR; LLVM is the same. Backreferences are hard in Rust, so a BB instead maintains its index into a Vec , owned by the…

> It is effectively a slow ... raw pointer. In my experience (unless I've misunderstood what you're trying to say) this is the fastest way to write graphs because of how much more cache-friendly it is than having to chase a load of pointers.

True, but you can do the same thing more efficiently with pointers.

Suppose the parent maintains an array of children

  Node*
Then each child has a

  Node**
member which points to itself in that array. That's one dereference to get up to the parent context.

In the Rust example, to do the same would take an extra few steps and probably miss the cache. I think speculative execution would favor the pointer method.

Re: Why Rust closures are somewhat hard

#78

Post by zelly »

imtringued wrote: post #64
zelly wrote: post #48

Everything that's hard in Rust could be solved by GC. With Rust, programmers have to spend most of their mental energy worrying about management of memory, which has largely been automated already. I always go back to this quote from Andrei Alexandrescu (creator of D): A disharmonic personality. Reading any amount of Rust code evokes the joke "friends don't let friends skip leg day" and the comic imagery of men with…

Gargabe collection is a much bigger crutch. Once you use it your entire program is contaminated by the performance characteristics of a single improperly written function. You cannot mix real time code with non real time code. If you do your real time code will be delayed by the non real time code. The unpredictability of the GC has become a property of your entire application.

Yes, but let's be real, 95% of programmers do not build real-time systems.

Especially not the type of programmers Rust is appealing to. They seem to be more excited by WebAssembly than the stuff that actual low latency people worry about, like tuning obscure FPGAs and vectorizing math code. These are industries like audio processing, SpaceX, industrial equipment manufacturers, self-driving cars, high frequency trading. Definitely not code targeting x86_64 cloud VMs or iPhones.

In hard real-time environments they do not even do memory management. They just statically allocate the whole heap and never malloc or free while the program is running. So even if Rust is better at memory management, it won't make them switch from C because it wasn't an issue.

Re: Why Rust closures are somewhat hard

#79

Post by hoseja »

stefs wrote: post #34
rafaelvasco wrote: post #11

Earlier quoted context omitted.

Yeah. Makes sense. Rust just explicitly exposes certain features so that the programmer has more control.

something i've read times and times again: programmers of other languages who learned rust tend to discover potentially problematic code in their earlier work.

That's just normal for learning to be a better programmer.

Re: Why Rust closures are somewhat hard

#80

Post by aratakareigen »

Diggsey wrote: post #25
tomjakubowski wrote: post #23

Earlier quoted context omitted.

Lambdas have unique types, and you can't use a generic parameter in a return type (which is impl Trait's raison d'etre), so I think it would have to be this: fn compose T, G: Fn(T) -> T>(f: F, g: G) -> impl Fn(T) - > T or: fn compose (f: F, g: G) -> impl Fn(T) -> T where F: Fn(T) -> T, G: Fn(T) -> T

Maybe you just miscommunicated, but you absolutely can use generic parameters in a return type: https://doc.rust-lang.org/nightly/std/iter/trait.Iterator.ht... ... and `impl Trait` exists to be able to have return types which are difficult, verbose, or impossible to name.

Return position impl Trait also exists so that you can change the actual return type without making a breaking change.
Post reply on HN