Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

24 days of Rust – Rayon

siciarz.net

Post reply on HN
Print view View original

21–30 of 66 posts

Re: 24 days of Rust – Rayon

#21

Post by GolDDranks »

lukaslalinsky wrote: post #17
kybernetikos wrote: post #14

Earlier quoted context omitted.

I know scala a bit and rust only barely, but they seem like very different languages to me. Scala tries to enable you to create whatever api you want, so it has many different flavours of magic to allow flexibility of expression. Predictably this is used and abused horribly by a community who can't come to a consensus on what good taste is. Rust on the other hand makes you explicitly say everything that happens. Noth…

> Nothing will happen without you being aware of it This is exactly the opposite what I understood from the article that Rayon does. For example, it will schedule your code differently depending on the current conditions of the CPU. That kind of non-determinism is hard to deal with if you are doing something more complex than adding numbers. I imagine this is fine for building mathematical libraries, where you care a…

I think that data parallelism is a very much wrong tool if you are interacting with external APIs – that's in the realm of concurrency and asynchrony. (Check out the work on Futures-rs and Tokio, they are inspiring projects!)

Rayon requires the callbacks it calls to be `Sync`. That means that they are declared to be safe to run across threads. That means that the API it provides explicitly requires the calculations to be parallelizable (and this is compiler checked), and if it doesn't run them in parallel, that can't hurt, right?

Re: 24 days of Rust – Rayon

#22

Post by dbaupp »

lukaslalinsky wrote: post #17
kybernetikos wrote: post #14

Earlier quoted context omitted.

I know scala a bit and rust only barely, but they seem like very different languages to me. Scala tries to enable you to create whatever api you want, so it has many different flavours of magic to allow flexibility of expression. Predictably this is used and abused horribly by a community who can't come to a consensus on what good taste is. Rust on the other hand makes you explicitly say everything that happens. Noth…

> Nothing will happen without you being aware of it This is exactly the opposite what I understood from the article that Rayon does. For example, it will schedule your code differently depending on the current conditions of the CPU. That kind of non-determinism is hard to deal with if you are doing something more complex than adding numbers. I imagine this is fine for building mathematical libraries, where you care a…

Rust can't stop random libraries from providing abstractions like this, but it does allow such libraries to be written to absolutely minimise overhead and maximise performance for when people opt in to using it. If this sort of parallelism isn't appropriate for the task at hand, the language isn't opinionated about what other schemes can work just as well as rayon.

Re: 24 days of Rust – Rayon

#23

Post by tatterdemalion »

lukaslalinsky wrote: post #17
kybernetikos wrote: post #14

Earlier quoted context omitted.

I know scala a bit and rust only barely, but they seem like very different languages to me. Scala tries to enable you to create whatever api you want, so it has many different flavours of magic to allow flexibility of expression. Predictably this is used and abused horribly by a community who can't come to a consensus on what good taste is. Rust on the other hand makes you explicitly say everything that happens. Noth…

> Nothing will happen without you being aware of it This is exactly the opposite what I understood from the article that Rayon does. For example, it will schedule your code differently depending on the current conditions of the CPU. That kind of non-determinism is hard to deal with if you are doing something more complex than adding numbers. I imagine this is fine for building mathematical libraries, where you care a…

This makes no sense to me. You are absolutely aware this will happen because you called Rayon's `.par_iter()` method!

The pitfalls of parallel and concurrent computing are well known, and you are correct should not use a data parallelism library for effectful actions whose order matters. But of course the same concerns can arise when using goroutines.

In contrast, Rust actually does guarantee an absence of data races, in rayon or in any other concurrency or parallelism library, whereas Go provides no such guarantee.

Re: 24 days of Rust – Rayon

#24

Post by kibwen »

lukaslalinsky wrote: post #4

I have not done any real programming in Rust, but whenever I see Rust code I'm amazed how different is it from Go, despite both having some shared use cases. Go's main selling point beyond concurrency is simplicity. And it's the simplicity that I like about it. On the other hand, it looks to me like Rust is turning into Scala.

If one were to have experience in both Rust and Scala, they would see that the comparison between the two is flawed. Scala revels in implicitness, whereas Rust is very explicit. Scala also has expressiveness as a goal, whereas in Rust expressiveness is an anti-goal. There are no user-defined operators in Rust, and less magic than in any language that I've used other than C.

Re: 24 days of Rust – Rayon

#25

Post by steveklabnik »

kibwen wrote: post #24
lukaslalinsky wrote: post #4

I have not done any real programming in Rust, but whenever I see Rust code I'm amazed how different is it from Go, despite both having some shared use cases. Go's main selling point beyond concurrency is simplicity. And it's the simplicity that I like about it. On the other hand, it looks to me like Rust is turning into Scala.

If one were to have experience in both Rust and Scala, they would see that the comparison between the two is flawed. Scala revels in implicitness, whereas Rust is very explicit. Scala also has expressiveness as a goal, whereas in Rust expressiveness is an anti-goal. There are no user-defined operators in Rust, and less magic than in any language that I've used other than C.

> whereas in Rust expressiveness is an anti-goal.

I think this is seriously over-stating it. We don't want Rust to be ugly or hard to use.

Re: 24 days of Rust – Rayon

#26

Post by kibwen »

kbenson wrote: post #12
anp wrote: post #6

Earlier quoted context omitted.

My personal experience with Rust (vs. Go and other languages) is that there is something really magical that all the sigils and syntactic complexity give you: once you've internalized Rust's approach it's ridiculously easy to build an accurate mental model of what's happening in almost any piece of code. From the high level constructs down to generated assembly, Rust produces the most predictable code I've ever writt…

The problem with people referring to more or less simple mental models, syntax, learning curves, expressiveness, etc is that they usually only focus on one of those at a time, when it requires multiple of them to get a good picture of how a language will work in practice for you, and in general. As an extremely simple example of this, compare BASIC and APL in terms of the learning curve. If we examine it in isolation…

I would compare Rust to Python rather than Perl. Like Python, Rust tries to have only one obvious way to do things, and avoiding TIMTOWTDI is a conscious goal.

Re: 24 days of Rust – Rayon

#27

Post by kibwen »

virtualwhys wrote: post #15
derkha wrote: post #8

Earlier quoted context omitted.

What do you mean by "turning into"? I don't see much special syntax in the post apart from the general expression-oriented functional style, including higher-order functions. Which has always been part of Rust. If that means turning into Scala, then almost every high-level programming language is currently turning into Scala.

> If that means turning into Scala, then almost every high-level programming language is currently turning into Scala. Martin Odersky, the creator of Scala, has said as much. Just look at Kotlin, Swift, and to some degree OCaml (upcoming modular implicits) and C# 7. Even Java 9 adopts Scala syntax with underscore now reserved. Rust is its own thing, however. OP is probably referring to complexity of syntax that more…

I believe the underscore predates Scala, with roots in ML.

Re: 24 days of Rust – Rayon

#28

Post by lukaslalinsky »

kibwen wrote: post #24
lukaslalinsky wrote: post #4

I have not done any real programming in Rust, but whenever I see Rust code I'm amazed how different is it from Go, despite both having some shared use cases. Go's main selling point beyond concurrency is simplicity. And it's the simplicity that I like about it. On the other hand, it looks to me like Rust is turning into Scala.

If one were to have experience in both Rust and Scala, they would see that the comparison between the two is flawed. Scala revels in implicitness, whereas Rust is very explicit. Scala also has expressiveness as a goal, whereas in Rust expressiveness is an anti-goal. There are no user-defined operators in Rust, and less magic than in any language that I've used other than C.

I have left more comments on this thread than I should considering my lack of experience with Rust, but I think the comparison with Scala is valid.

What got me thinking about that was the implementation of into_par_iter, which is a trait implemented on a bunch of standard types, not unlike implicit methods in Scala. I know this is rare code, but reading this left with very similar feeling that I had when reading Scala standard library code:

https://github.com/nikomatsakis/rayon/blob/master/src/par_it...

Re: 24 days of Rust – Rayon

#29

Post by pcwalton »

lukaslalinsky wrote: post #19
pcwalton wrote: post #16

Earlier quoted context omitted.

How does not being able to write the high-level par_iter() abstraction make Golang more high-level?

I didn't say Go is more high-level, I said Go doesn't allow you to do low-level things efficiently. Writing your own parallelism library requires good access to the language's execution model internals. Go doesn't give you that, because Go wants you to write application code, not library code. It's often frustrating, because sometimes you want to write a generic low-level library that will add no overhead to the code…

None of the primitives Golang gives you allow you to achieve the same results that Rayon does. The Go work stealing system only works with goroutines, which are too expensive to spawn on every iteration of a tight loop.

Re: 24 days of Rust – Rayon

#30

Post by pjmlp »

dikaiosune wrote: post #7
ceronman wrote: post #5

Rayon seems very similar to Java 8 Parallel Streams or C# Parallel LinQ. What are the advantages/disadvantages of the Rust approach?

Closures in Rust are stack-allocated and LLVM can inline them and optimize them as if they're any regular imperative code, for one thing. This means that there's less overhead from managing the iterator chains, and that they're statically dispatched which saves on runtime indirection. The borrow checker also makes sure that you don't accidentally mutate non-thread-safe data from your parallel iterators.

Regarding closures, that is also possible in Java and .NET, just you don't control when it might happen.
Post reply on HN