Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Fearless Concurrency in Firefox Quantum

blog.rust-lang.org

Post reply on HN
Print view View original

41–50 of 177 posts

Re: Fearless Concurrency in Firefox Quantum

#41

Post by throwaway613834 »

maffydub wrote: post #39
throwaway613834 wrote: post #30

Could someone give a very simple, to-the-point example of a kind of concurrency bug that Rust prevents, for those of us who don't know Rust? (The author explicitly fails to think of any, so I'm hoping someone else can. It'd be more convincing to see one.) EDIT: I meant a code example, not a paragraph. And I would obviously expect to see how the intended goal is achieved without the bug... otherwise it'd be trivial to…

Borrowing from https://blog.rust-lang.org/2015/04/10/Fearless-Concurrency.h... (linked to from the original post), the following code tries to access a lock-protected vector. fn use_lock(mutex: &Mutex >) { let vec = { // acquire the lock let mut guard = lock(mutex); // attempt to return a borrow of the data access(&mut guard) // guard is destroyed here, releasing the lock }; // attempt to access the data outside of t…

Thanks (upvoted)! However how in the world would the compiler know if the mutex belongs to the same vector you are accessing? (Unless you're saying the mutex wraps the vector implying that each vector can have one corresponding mutex at most? Which seems quite limiting?)

Re: Fearless Concurrency in Firefox Quantum

#42

Post by kuschku »

twic wrote: post #35
rkangel wrote: post #15

Earlier quoted context omitted.

Note that the panic you get by calling unwrap() where you shouldn't isn't a crash. It's a controlled program exit due to an unexpected condition. While yes, the panic will cause your program to stop, it will do it in a clean deterministic way (with a backtrace). Actual crashes (due to segfaults) can happen a long way from the bug that actually caused the issue, can happen intermittently and generally be a nightmare t…

When this controlled program exit happens, does your monitoring system wake your operations staff up? If so, it's a crash.

Yes, but you also immediately know where it happened, what happened, and how to fix it.

That’s a massive improvement over alternatives

Re: Fearless Concurrency in Firefox Quantum

#43

Post by arielb1 »

seren wrote: post #34
throwaway613834 wrote: post #30

Could someone give a very simple, to-the-point example of a kind of concurrency bug that Rust prevents, for those of us who don't know Rust? (The author explicitly fails to think of any, so I'm hoping someone else can. It'd be more convincing to see one.) EDIT: I meant a code example, not a paragraph. And I would obviously expect to see how the intended goal is achieved without the bug... otherwise it'd be trivial to…

The most basic example would be two threads trying to write the same variable concurrently without synchronisation. As far as I know, as long as you don't use any unsafe code block, it is not possible, i.e. the compiler will protest loudly and the program won't compile. You are forced by the compiler to implement an explicit exclusion mechanism. The point is that kind of issue are silent bugs most of the time(up to p…

For example, accidentally sharing a lock-less cache or a non-atomic reference counted pointer between threads.

For example this code, which tries to send a reference-counted pointer between threads, which can cause the reference counter to become unsynchronized and random use-after-free:

    use std::thread;
    use std::rc::Rc;

    fn main() {
        let rcs = Rc::new("Hello, World!".to_string());
    
        let thread_rcs = rcs.clone();
        thread::spawn(move || {
            println!("{}", thread_rcs);
        });
    }
Is detected by the compiler and causes this error

    error[E0277]: the trait bound `std::rc::Rc: std::marker::Send` is not satisfied in `[closure@src/main.rs:8:19: 10:6 thread_rcs:std::rc::Rc]`
     --> src/main.rs:8:5
      |
    8 |     thread::spawn(move || {
      |     ^^^^^^^^^^^^^ `std::rc::Rc` cannot be sent between threads safely
      |
      = help: within `[closure@src/main.rs:8:19: 10:6 thread_rcs:std::rc::Rc]`, the trait `std::marker::Send` is not implemented for `std::rc::Rc`
      = note: required because it appears within the type `[closure@src/main.rs:8:19: 10:6 thread_rcs:std::rc::Rc]`
      = note: required by `std::thread::spawn`

Re: Fearless Concurrency in Firefox Quantum

#44

Post by icen »

throwaway613834 wrote: post #40
TheCycoONE wrote: post #37

Earlier quoted context omitted.

Sure, the obvious example is your classic thead1 i++, thead2 i-- bug. In C you run each thread a large number of times in parallel and you end up with something other than 0. The solution is to use atomic operations or locks. Rust doesn't allow having two mutable references to the same memory location at the same time, so you would never encounter that.

> Rust doesn't allow having two mutable references to the same memory location at the same time, so you would never encounter that. Wouldn't that make a whole class of efficient algorithms impossible though? Like let's say you have an std::list and you want a sorted "view" of the elements. In C++ you'd create an array of pointers and then sort it, and after that you can just modify whatever each slot points to. In Ru…

There's always the `unsafe` escape hatch - if you need to potentially alias mutable pointers, you can (via using the 'raw pointer' types `const T` and `mut T`).

You can only do this in an `unsafe` block. If you get a bug caused by aliased mutation, then you know exactly where it comes from.

Re: Fearless Concurrency in Firefox Quantum

#45

Post by ekidd »

kibwen wrote: post #8

I like this explanatory comment by Manishearth, a Servo dev, in the thread over on /r/rust: "This blog post brought to you by the 'how many times can you say 'fearless concurrency' and keep a straight face' cabal. "Seriously though, I now appreciate that term a lot more. One thing that cropped up in the review of this post was that I didn't have examples of bugs Rust prevented. Because I couldn't think of any concret…

I've spent the last few days aggressively parallelizing some Rust code with crossbeam, and it's really just... painless (once you're used to Rust). Rust actually understands data races, and it grumbles at me until my code is provably safe, and then everything Just Works.

The Rayon library is also lovely for data parallelism.

Sometimes, I think, "Rust is basically a nicer C++ with the obvious foot guns removed and great tooling", but then there are those moments where I'm just blown away by what a good job it does.

(I think it helps that I have some functional programming experience under my belt, and that I tend to use mutability sparingly, and mostly in very simple ways.)

Re: Fearless Concurrency in Firefox Quantum

#46

Post by maffydub »

throwaway613834 wrote: post #41
maffydub wrote: post #39

Earlier quoted context omitted.

Borrowing from https://blog.rust-lang.org/2015/04/10/Fearless-Concurrency.h... (linked to from the original post), the following code tries to access a lock-protected vector. fn use_lock(mutex: &Mutex >) { let vec = { // acquire the lock let mut guard = lock(mutex); // attempt to return a borrow of the data access(&mut guard) // guard is destroyed here, releasing the lock }; // attempt to access the data outside of t…

Thanks (upvoted)! However how in the world would the compiler know if the mutex belongs to the same vector you are accessing? (Unless you're saying the mutex wraps the vector implying that each vector can have one corresponding mutex at most? Which seems quite limiting?)

Thanks!

Yes, when you construct a mutex, you give it ownership of the vector you want it to protect. Due to the way Rust works, once you've given ownership to something else, you can no longer access it yourself.

The only way you can get access to the data again is to "borrow" a reference to it, but this borrow has a "lifetime", which is tied to the period for which you're holding the mutex. This is how the compiler can spot that you've tried to access the vector after the mutex has been released.

As you say, this means that each vector can only have one mutex wrapping it - at least for this implementation of mutex, known as std::sync::Mutex ( https://doc.rust-lang.org/std/sync/struct.Mutex.html).

However, if you're looking for something like a read/write lock, Rust supports that too via std::sync::RwLock - see https://doc.rust-lang.org/std/sync/struct.RwLock.html for more details.

Obviously, the example uses a vector, but Rust has a pretty strong system of generics, so your mutex (or read/write lock) can wrap pretty much any type, e.g. a struct you've defined.

Hope that makes sense - this was one of the areas of Rust that impressed me most!

Re: Fearless Concurrency in Firefox Quantum

#47

Post by wtetzner »

vanderZwan wrote: post #16
kibwen wrote: post #8

I like this explanatory comment by Manishearth, a Servo dev, in the thread over on /r/rust: "This blog post brought to you by the 'how many times can you say 'fearless concurrency' and keep a straight face' cabal. "Seriously though, I now appreciate that term a lot more. One thing that cropped up in the review of this post was that I didn't have examples of bugs Rust prevented. Because I couldn't think of any concret…

Quoting what I just replied in that thread: > Given that this Servo code replaces an existing code base, couldn't we get a "guestimate" by looking at how many unsolved bug reports are now closed because their associated previous (presumably C++) code has been replaced? How many open bugs existed in Stylo's precursor that are removed now?

But was the existing code base that was replaced running concurrently?

Re: Fearless Concurrency in Firefox Quantum

#48

Post by simias »

adrianN wrote: post #18
simias wrote: post #14

Earlier quoted context omitted.

I'm a former C++ dev who went all in on Rust. I think the main problem is that the learning curve works in Rust's disadvantage here. If you start learning C++ it's relatively smooth sailing at first, especially if you're already familiar with C. Basic OOP, basic RAII, inheritance, virtual functions, basic templates. Easy peasy. It's once you start getting to the advanced topics that the footguns become apparent. The…

Already basic C contains more than enough footguns. If you think basic C++ is relatively free of footguns you're kidding yourself. Rust has a steep learning curve because the compiler nags you a lot about things that would have been a potential footgun in C. Unfortunately it is not smart enough to see in all cases that your code wouldn't have triggered that particular footgun and has to be overly conservative.

I meant that basic C++ seems rather straightforward and at first doesn't appear to introduce too many new footguns over the C subset. Actually thanks to vectors and RAII it can remove whole classes of classic C mistakes.

It's only when you start to have all these elements work (or not work) together that you realize that it's not as simple as it first seemed and the various side effects, overloadings and implied constraints sprinkled throughout the code turned it into a virtual minefield.

Case in point, this video I've watched the past week: https://channel9.msdn.com/posts/C-and-Beyond-2012-Herb-Sutte...

It's a very interesting talk (and probably worth a watch if you're a C++ developer) but the amusing thing to me is that he begins by showing two short pieces of rather simple C++ code and asks the audience if they are UB or not. Seems like nobody can (or want to) answer that question.

Spoiler: the conclusion of the talk is that one of these pieces of code is only legal if the copy constructor of the custom type adheres to certain implied constraints that are not enforced by the compiler and, it seems, few people are aware of.

Re: Fearless Concurrency in Firefox Quantum

#49

Post by rbehrends »

throwaway613834 wrote: post #40
TheCycoONE wrote: post #37

Earlier quoted context omitted.

Sure, the obvious example is your classic thead1 i++, thead2 i-- bug. In C you run each thread a large number of times in parallel and you end up with something other than 0. The solution is to use atomic operations or locks. Rust doesn't allow having two mutable references to the same memory location at the same time, so you would never encounter that.

> Rust doesn't allow having two mutable references to the same memory location at the same time, so you would never encounter that. Wouldn't that make a whole class of efficient algorithms impossible though? Like let's say you have an std::list and you want a sorted "view" of the elements. In C++ you'd create an array of pointers and then sort it, and after that you can just modify whatever each slot points to. In Ru…

> Wouldn't that make a whole class of efficient algorithms impossible though?

Correct. What it does is prevent data races; this is per se nothing new, but was done as early as the 70s [1]. There've been various and sundry approaches to the same problem over the years (the 90s and early aughts produced a lot of research in this area).

In a way, it is frustrating that so few programming languages offer prevention of data races, so I'm grateful that Rust pushes that; but in another way, it is understandable, because all such mechanisms restrict what you can do, often in undesirable ways. Google "benign data races", for example: data races can be purposely exploited for more efficient implementations.

The other problem is that data races are really the easy part to solve about concurrency. The hard parts are general race conditions, non-determinism/causality, liveness properties (such as freedom from deadlocks and starvation), and performance.

Performance is one of the trickier aspects: in a shared memory system, performance is primarily a matter of reducing contention. But virtually any system that relies on some form of mutual exclusion to prevent data races introduces contention, and you have a constant tug of war between the two concerns.

Particularly frustrating is that all of these issues are what software engineers call "cross-cutting concerns" [2], i.e. things that cannot easily be modularized. For example, performance may suffer from a serialization bottleneck in a module's implementation where you cannot avoid that bottleneck without exposing the module's internals.

[1] https://en.wikipedia.org/wiki/Concurrent_Pascal

[2] https://en.wikipedia.org/wiki/Cross-cutting_concern

Re: Fearless Concurrency in Firefox Quantum

#50

Post by StavrosK »

ekidd wrote: post #45
kibwen wrote: post #8

I like this explanatory comment by Manishearth, a Servo dev, in the thread over on /r/rust: "This blog post brought to you by the 'how many times can you say 'fearless concurrency' and keep a straight face' cabal. "Seriously though, I now appreciate that term a lot more. One thing that cropped up in the review of this post was that I didn't have examples of bugs Rust prevented. Because I couldn't think of any concret…

I've spent the last few days aggressively parallelizing some Rust code with crossbeam, and it's really just... painless (once you're used to Rust). Rust actually understands data races, and it grumbles at me until my code is provably safe, and then everything Just Works. The Rayon library is also lovely for data parallelism. Sometimes, I think, "Rust is basically a nicer C++ with the obvious foot guns removed and gre…

> painless (once you're used to Rust)

This needs to be Rust's motto or something.

Post reply on HN