Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The humble for loop in Rust

blog.startifact.com

Post reply on HN
Print view View original

21–30 of 60 posts

Re: The humble for loop in Rust

#21

Post by zamalek »

nemo1618 wrote: post #4

> Why is map so much faster? I am not sure. I suspect with the map() option the Rust compiler figures out it can avoid allocations altogether by simply writing over the original vector, while with the loop it can't. Or maybe it's using SIMD? I tried to look in the compiler explorer but I'm not competent enough yet to figure it out. Maybe someone else can explain! Yep, it's due to SIMD -- in the assembly for `using_ma…

It's not only because of SIMD. Contrasted to many other languages (though not all) the compiler is working with code here, not an arbitrary function pointer. In essence, JS and the like are operating with this:

    let result: Vec = list.into_iter().map::>(Box::new(transform)).collect()
Rust is able to inline the transform code right into the loop, which then becomes available for SIMD etc.

Rust further brings really nice ergonomics and comprehensive type inference to the equation, which makes it feel like writing C#/JS/whatever. JITted languages could detect and elide creating and immediately using a function pointer, but I don't think that any do.

Edit: these languages may not always allocate (specifically if nothing is captured in a closure), but the core concept remains: they erase the type, which means that they also erase the function body.

Re: The humble for loop in Rust

#23

Post by mastax »

ironhaven wrote: post #16
hedgehog wrote: post #10

Earlier quoted context omitted.

It seems `map` should have less restrictive semantics (specifically ordering) than `for`, does that allow more optimization? I don't know much about Rust internals.

Reading the godbolt it looks like for the push loop llvm is unable to remove the `grow_one` capacity check after every push. Becaue of this the Vec could possibly reallocate after every push meaning it can't auto vectorize.

It's a little bit surprising to me that LLVM can't eliminate the grow_one check. It looks like there's a test ensure it's not needed in the easier case of vec.push(vec.pop()) [0]. With the iterator the optimization is handled in the standard library using specialization and TrustedLen[1].

[0]: https://github.com/rust-lang/rust/blob/master/tests/codegen/...

[1]: https://github.com/rust-lang/rust/blob/d4025ee454169fbd22f57...

Re: The humble for loop in Rust

#25

Post by aidos »

c0balt wrote: post #9
aidos wrote: post #6

Earlier quoted context omitted.

Ah thanks! I was going to ask in here to see if anyone knew because the explanation about overwriting the same vector seemed pretty off base. Never worked in Rust though so wondered if the iterator api had some weird optional notion of size that could be utilised throughout the chain.

> Never worked in Rust though so wondered if the iterator api had some weird optional notion of size that could be utilised throughout the chain. Fwiw, this does exist: [Iterator::size_hint] ( https://doc.rust-lang.org/std/iter/trait.Iterator.html#metho... )

Interesting! Thanks.

Re: The humble for loop in Rust

#26

Post by scotty79 »

I'm currently doing advent of code in Scala 3 to learn a bit of this language.

For loop does weird things there. It can be used as a flat map as it can iterate over multiple iterators at once and yield a value for each combination.

What's bitten me so far multiple times is that when you iterate over a Set or a Map the result of for expression is also a Set or a Map.

But since you have access to keys during iteration then some iterations, if they return same map key or same set value, might get silently overwritten by others.

I don't remember having this problem in Rust because there I had to be very intentional about iterators.

The other thing that bit me was that arithmetic on Int overflows silently, but that's apparently a Java thing, which made me wonder how is Java an enterprise language.

Otherwise Scala 3 is superb expeirience. Syntax is ultra-flexible and local extensibility of everything and access to things from the context of where your code is defined and even from the context where it's running is magical.

Re: The humble for loop in Rust

#27

Post by CrendKing »

Maybe I'm dumb, but I can't see how the code in the "Errors and map" section can compile. "transform_list" returns a Result, yet "result" is just a Vec. I thought you always need to wrap it with Ok()? Is that a new nightly feature?

Re: The humble for loop in Rust

#28

Post by winwang »

scotty79 wrote: post #26

I'm currently doing advent of code in Scala 3 to learn a bit of this language. For loop does weird things there. It can be used as a flat map as it can iterate over multiple iterators at once and yield a value for each combination. What's bitten me so far multiple times is that when you iterate over a Set or a Map the result of for expression is also a Set or a Map. But since you have access to keys during iteration…

glad to see another Scala fan :)

keep in mind that "for loops" are really "for comprehensions" and desugae into flatMap/map

Re: The humble for loop in Rust

#29

Post by [deleted] »

CrendKing wrote: post #27

Maybe I'm dumb, but I can't see how the code in the "Errors and map" section can compile. "transform_list" returns a Result , yet "result" is just a Vec. I thought you always need to wrap it with Ok()? Is that a new nightly feature?

[deleted]

Re: The humble for loop in Rust

#30

Post by zozbot234 »

IshKebab wrote: post #17

Yeah I kind of wish there was a way to still use `?` inside map/filter/etc. lambdas. Error handling with that functional style is generally way more awkward than for loops, but also it's often more elegant in other ways (e.g. Rayon). I think Ruby has some kind of feature that works like that but IIRC it looked less foot-gun more foot-bazooka. Does anyone know of any languages that solve that problem elegantly?

"Ad-Hoc Effects in Rust" https://capi.hannobraun.com/daily/2024-12-12 discusses the issue in detail.

An early attempt at a solution (2022) was provided by https://blog.rust-lang.org/inside-rust/2022/07/27/keyword-ge...

Post reply on HN