Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Concurrency in Rust

doc.rust-lang.org

Post reply on HN
Print view View original

21–30 of 160 posts

Re: Concurrency in Rust

#21

Post by reycharles »

barosl wrote: post #15
djhworld wrote: post #4

I'm having a tough time trying to understand this snippet for i in 0..3 { thread::spawn(move || { data[i] += 1; }); } What is the 'move' thing here before the ||

So, for those who may know JavaScript, you may have seen code like this: var closures = []; for (var i=0;i The code above will result in incorrect results: 5, 5, 5, 5, 5. Because you're capturing `i` as a reference. To avoid this, JS devs typically do this: closures.push((function(i) { return function() { console.log(i); }; })(i)); Or, if you can afford the ES6 support: for (let i=0;i Rust supports this pattern by a…

The ES6 "fix" is an abomination in my opinion. You're closing over a mutable variable, so you should see the mutations! IMO the real fix would be to write

    for (var i = 0; i
In your ES6 solution if you e.g. increment "i" inside the loop after the closure the closure will see the mutation!

The real cause of confusion is mutation and javascript's scoping rules.

Re: Concurrency in Rust

#22

Post by kzrdude »

losvedir wrote: post #14
askyourmother wrote: post #8

What about the assumption (fatally flawed decision?) that malloc never fails when rust asks? Sounds like something that could affect concurrency

That's not a property of rust the language, but the standard library. I believe you could use other allocators that behave differently. In any case, the standard library panics on OOM, and panics are described at the bottom of the linked page.

calls abort(), not panic!(). This is important since unwinding does not happen with the former.

Re: Concurrency in Rust

#23

Post by alkonaut »

Manishearth wrote: post #11
z1mm32m4n wrote: post #6

Does Rust have a way to work with SIMD concurrency as opposed to just fork/join concurrency? Something along the lines of how openmp or cilk let you do a parallel for all?

Rust supports SIMD, with some utility structs that let you do it easily. I don't know of any libraries that auto-simd things though IIRC LLVM can do this on its own sometimes.

I think the parent was referring to "lightweight task"-based concurrency like C#'s PLinq:

    // C# 
    int sum;
    Parallel.ForEach(myCollection, item => sum += item);
Which operates using a reasonably sized thread pool rather than a thread per item. Something similar in Rust (Excuse my rusty rust) would look like.

    let someList = ...
    parallel::for(someList.iter(), |item| {
       // Do thing with each item
    };
I think there is ongoing work on this topic, but it will likely only be library-level and not language-level (Just like Linq is a language level feature in C# but PLinq is a library). There are some third party crates that do this like simple_parallel.

Edit: Low-level simd exist as intrinsics and of course through llvm vectorizations.

Re: Concurrency in Rust

#24

Post by kbenson »

Chilinot wrote: post #10
z1mm32m4n wrote: post #6

Does Rust have a way to work with SIMD concurrency as opposed to just fork/join concurrency? Something along the lines of how openmp or cilk let you do a parallel for all?

A simple google search would have told you that there is work being done the introduce simd in Rust. There appear to be some basics there right now but not much if i understood my quick search.

An active discussion about concurrency in rust with first developers commenting is a very appropriate place to ask that question, and ultimately much more likely to yield valid and current information than Google.

Re: Concurrency in Rust

#25

Post by grayrest »

alkonaut wrote: post #23
Manishearth wrote: post #11

Earlier quoted context omitted.

Rust supports SIMD, with some utility structs that let you do it easily. I don't know of any libraries that auto-simd things though IIRC LLVM can do this on its own sometimes.

I think the parent was referring to "lightweight task"-based concurrency like C#'s PLinq: // C# int sum; Parallel.ForEach(myCollection, item => sum += item); Which operates using a reasonably sized thread pool rather than a thread per item. Something similar in Rust (Excuse my rusty rust) would look like. let someList = ... parallel::for(someList.iter(), |item| { // Do thing with each item }; I think there is ongoing…

The simplest library for this kind of data parallelism is Rayon[1]. The README provides a nice overview and links to a series of blog posts about the details.

[1] https://github.com/nikomatsakis/rayon/

Re: Concurrency in Rust

#26

Post by Manishearth »

alkonaut wrote: post #23
Manishearth wrote: post #11

Earlier quoted context omitted.

Rust supports SIMD, with some utility structs that let you do it easily. I don't know of any libraries that auto-simd things though IIRC LLVM can do this on its own sometimes.

I think the parent was referring to "lightweight task"-based concurrency like C#'s PLinq: // C# int sum; Parallel.ForEach(myCollection, item => sum += item); Which operates using a reasonably sized thread pool rather than a thread per item. Something similar in Rust (Excuse my rusty rust) would look like. let someList = ... parallel::for(someList.iter(), |item| { // Do thing with each item }; I think there is ongoing…

simple_parallel exists and is pretty neat, yes. Rayon is another cool library here (https://github.com/nikomatsakis/rayon), which lets you do foreach/mapreduce operations via work stealing on a pool with an extensible API that looks exactly like the regular iterator api (https://github.com/nikomatsakis/rayon#parallel-iterators, for example).

Note that a simd library exists which abstracts over the intrinsics (https://github.com/huonw/simd)

Re: Concurrency in Rust

#27

Post by masklinn »

anon4 wrote: post #12
askyourmother wrote: post #8

What about the assumption (fatally flawed decision?) that malloc never fails when rust asks? Sounds like something that could affect concurrency

Malloc never fails, but you might die if you touch the memory. In general, modern OSes don't have a good story about exhausting available memory beyond "let's kill a bunch of processes to free up memory".

> Malloc never fails

malloc can fail, even on default linux (overcommit enabled), if you go above the process's vmem limit for instance (because 32b or rlimited). And of course not all OS overcommit, Windows famously does not.

Re: Concurrency in Rust

#28

Post by pdpi »

singlow wrote: post #7
djhworld wrote: post #4

I'm having a tough time trying to understand this snippet for i in 0..3 { thread::spawn(move || { data[i] += 1; }); } What is the 'move' thing here before the ||

A move closure takes copies of the environment, instead of mutating the parent environment. https://doc.rust-lang.org/book/closures.html#move-closures

It's more accurate to say that it takes ownership of the captured environment. Whether that means copying the values or moving ownership depends on whether the values are of a type that implements Copy

Re: Concurrency in Rust

#29

Post by pmarreck »

kentosi wrote: post #18
barosl wrote: post #15

Earlier quoted context omitted.

So, for those who may know JavaScript, you may have seen code like this: var closures = []; for (var i=0;i The code above will result in incorrect results: 5, 5, 5, 5, 5. Because you're capturing `i` as a reference. To avoid this, JS devs typically do this: closures.push((function(i) { return function() { console.log(i); }; })(i)); Or, if you can afford the ES6 support: for (let i=0;i Rust supports this pattern by a…

Thanks for the explanation, but how would the move keyword know that you're referring to the i variable? Ie - what is there were multiple variables (such as a for loop with j in there)? Wouldn't it have been a better approach to add some sort of demarkation, such as i* or i^ (or whatever) to indicate this? Just curious.

I agree that this seems ugly

Re: Concurrency in Rust

#30

Post by gregwebs »

Send + Sync are great. The downside of concurrency in Rust is:

1) There isn't transparent integration with IO in the runtime as in Go or Haskell. Rust probably won't ever do this because although such a model scales well in general, it does create overhead and a runtime.

2) OS threads are difficult to work with compared to a nice M:N threading abstraction (which again are the default in Go or Haskell). OS threads leads to lowest common denominator APIs (there is no way to kill a thread in Rust) and some difficulty in reasoning about performance implications. I am attempting to solve this aspect by using the mioco library, although due to point #1 IO is going to be a little awkward.

Post reply on HN