Fearless concurrency with Rust • Hacker News Board

Re: Fearless concurrency with Rust

#31

Post by steveklabnik » Fri, Apr 10, 2015, 6:37 PM UTC

rntz wrote: post #28

steveklabnik wrote: post #25

Earlier quoted context omitted.

Ah, thanks for that, I'll look into it. I guess what I was trying to get at is that 'race conditions' can fall under the same kinds of errors as logic errors. This is the way that people usually attack the phrase "if it compiles, it works."

I'm not sure what you mean when you say that race conditions can be the same kind of error as logic errors. I would define a race condition as "an error due to nondeterministic concurrency". If you ban nondeterminism, for example, race conditions are impossible. In practice, nobody wants to ban nondeterminism, so races creep in. There might be smarter ways to eliminate races (ways of only allowing benign nondetermini…

I think I'm just being imprecise with my terms, since this is just a comment. I've seen people describe race conditions in practice as "most of the time, message A comes in, then B, but I didn't write it correctly, so in certain cases, B comes in, and then A." Which I would casually describe as racy, but as you point out, would be more accurately described as a pure logic error.

Re: Fearless concurrency with Rust

#32

Post by rcxdude » Fri, Apr 10, 2015, 6:39 PM UTC

Jweb_Guru wrote: post #26

rntz wrote: post #23

Earlier quoted context omitted.

Preventing deadlocks is not an unsolvable problem. Linear-session-typed process calculi are guaranteed deadlock-free and race-free. (The connection with Rust's ownership type-system makes me wonder if there might be some way to backport this guarantee to Rust, but I suspect the connection doesn't go far enough.) This comes at a price, however: certain process topologies are impossible to construct, and there's no way…

The safe subset of Rust (which is more-or-less linearly typed with elided drops) is indeed deadlock-free. Features like mutexes are written using the unsafe sublanguage because shared memory concurrency is useful. Because potential deadlocks are opt-in rather than opt-out, it's not unreasonable to imagine that future extensions to Rust (or some other language in the same mold) could bridge this gap in a practical way…

This is a bit of a misunderstanding of what Rust's safety guarantees are and what 'unsafe' actually means in Rust.

Rust safety guarantees are based around a few rules involving references, ownership, lifetimes and borrowing. None of them forbid the creation of mutexes or anything else which you will find in Rust.

'unsafe' Rust code shouldn't break these rules any more than the 'safe' code does, the only difference is that the compiler lets you do something which it doesn't know is safe. It's up to you to design the code such that it is.

If code marked as 'unsafe' does something which actually violates one of the assumptions on which the Rust compiler works, then the safety of the whole program is jeopardised.

(There was actually a bit of bike-shedding surrounding possibly renaming the 'unsafe' keyword to better reflect these facts, which included suggestions like 'trustme' and 'yolo', but a satisfactory term was not found)

Re: Fearless concurrency with Rust

#33

Post by Jweb_Guru » Fri, Apr 10, 2015, 6:42 PM UTC

rcxdude wrote: post #32

Jweb_Guru wrote: post #26

Earlier quoted context omitted.

The safe subset of Rust (which is more-or-less linearly typed with elided drops) is indeed deadlock-free. Features like mutexes are written using the unsafe sublanguage because shared memory concurrency is useful. Because potential deadlocks are opt-in rather than opt-out, it's not unreasonable to imagine that future extensions to Rust (or some other language in the same mold) could bridge this gap in a practical way…

This is a bit of a misunderstanding of what Rust's safety guarantees are and what 'unsafe' actually means in Rust. Rust safety guarantees are based around a few rules involving references, ownership, lifetimes and borrowing. None of them forbid the creation of mutexes or anything else which you will find in Rust. 'unsafe' Rust code shouldn't break these rules any more than the 'safe' code does, the only difference is…

You are correct. When I refer to the safe sublanguage, I mean the language without any unsafe blocks. Since so much of Rust is defined in libraries, precisely what is allowed in those unsafe blocks is a fairly important detail; they can violate semantic guarantees that "pure" safe Rust provides. In this case, the Rust core team made the deliberate choice (which they have discussed on occasion) not to consider deadlocks unsafe behavior, because they were not able to get it to work to their satisfaction. As another example, there are fairly subtle implications for concurrency around the fact that Rust provides (using unsafe code) a swap function for mutable references. Swap cannot be written in the pure safe sublanguage. Yet another example: in pure, safe Rust, memory leaks are not possible (except via premature abort). This is not true of Rc, hence Rust does not guarantee freedom from memory leaks.

Re: Fearless concurrency with Rust

#34

Post by pcwalton » Fri, Apr 10, 2015, 6:43 PM UTC

chimeracoder wrote: post #29

Animats wrote: post #12

Rust's ownership model, and the borrow checker that enforces it, are a major breakthrough in language design. It's so simple, yet it solves so many problems. This is what Go should have done. Then Go's "share by communicating, not by sharing" would be real, not PR. Go code often passes references over channels, which results in shared memory between two threads with no locking. In Rust, when you do that, you pass own…

First, please don't compare Go and Rust. They are completely different languages with completely different target use cases. Go has very different design goals than Rust, so very little that Rust does would actually be possible in Go, and vice versa. > Go code often passes references over channels, which results in shared memory between two threads with no locking Having a complex type system would cut into compile t…

> (an explicit primary design goal, to the point where the Go compiler must be written to read each file exactly once[0], no more).

Same in Rust. Thanks to the module system, the Rust compiler never rereads a file more than once.

> [1] I'd need to think about this, but I don't think it'd be difficult to detect this statically[2] at compile-time and enforce that stack-allocated rvalues into channels are never used again in the same scope. It's definitely possible to extend `go vet` to handle this, and it may even be possible to write this in as a compiler error in a future version of Go.

That trivially fails to solve the problem, due to aliasing.

> [2] Incidentally, one of the reasons that it's so easy to do reliable static analysis on Go code (compared to other languages) is that the grammar is incredibly simple - it's almost entirely context-free, which is very rare among non-Lisps. Having a more complex type system usually requires at least some additional syntax to along with this, which means you'd have to start sacrificing this design goal as well in order to create a more elaborate type system.

Rust's grammar is also context-free, as far as I know.

Your proposed static analysis is not reliable. The "more complex" type system in Rust exists precisely so that we can do more reliable static analysis.

Re: Fearless concurrency with Rust

#35

Post by mafribe » Fri, Apr 10, 2015, 6:45 PM UTC

rntz wrote: post #23

steveklabnik wrote: post #20

Earlier quoted context omitted.

> Sorry if I'm missing something obvious, but how do these locks stop a deadlock from happening? This is the difference between a data race and a race condition. We can't generally prevent deadlocks, I would imagine that's (like all race conditions) is an unsolvable problem at the language level. Maybe some PhD will prove me right or wrong, though...

Preventing deadlocks is not an unsolvable problem. Linear-session-typed process calculi are guaranteed deadlock-free and race-free. (The connection with Rust's ownership type-system makes me wonder if there might be some way to backport this guarantee to Rust, but I suspect the connection doesn't go far enough.) This comes at a price, however: certain process topologies are impossible to construct, and there's no way…

> Linear-session-typed process calculi are guaranteed deadlock-free and race-free

While this may be true for certain restricted systems, this claim is misleading. Session types, as invented by K. Honda and refined by numerous others, guarantee deadlock and race freedom only within any given session. Session initiation (basically messages to replicated inputs) can deadlock and race in many (most?) session typing systems. The decision not to guarantee the absence of deadlock at session initiation points is what makes session types so simple.

It is possible to restrict session types so that they are completely deadlock-free and race-free, but -- as far as we know -- that guarantee comes at the heavy price of making the typing system completely inexpressive, so you cannot type many natural forms of concurrent programming. Alternatively you can give up on simplicity and make the typing system very complicated (e.g. the system pioneered in "Strong Normalisation in the Pi-Calculus" by Honda et al, and its successors). The former is a real problem, as witnessed by the continuing inability of the session typing crowd to get a handle on message passing in Erlang.

Finding a system that is expressive, simple and guarantees deadlock-freedom is a major open research problem (and I suspect infeasible).

Re: Fearless concurrency with Rust

#36

Post by Argorak » Fri, Apr 10, 2015, 6:47 PM UTC

steveklabnik wrote: post #7

rntz wrote: post #3

This post observes that > the same tools that make Rust safe also help you tackle concurrency head-on. The tools that make Rust safe, in this instance, being its ownership type system. Why on earth should that be the case? Why would ownership types make concurrency easier? The post gives plenty of in-depth answers to this question, but to my mind it misses the bigger picture. The big-picture reason why ownership type…

We may end up adding linear types, not just affine, to Rust as well: https://github.com/rust-lang/rfcs/issues/814

Interesting. Could that also help with types that should actually be explicitly dropped, like this case?

http://internals.rust-lang.org/t/std-fs-tempdir-ignoring-err...

Edit: yep, it was mentioned in the discussion here. http://internals.rust-lang.org/t/issues-in-new-i-o/1658/4?u=...

Re: Fearless concurrency with Rust

#37

Post by steveklabnik » Fri, Apr 10, 2015, 6:52 PM UTC

pcwalton wrote: post #34

chimeracoder wrote: post #29

Earlier quoted context omitted.

First, please don't compare Go and Rust. They are completely different languages with completely different target use cases. Go has very different design goals than Rust, so very little that Rust does would actually be possible in Go, and vice versa. > Go code often passes references over channels, which results in shared memory between two threads with no locking Having a complex type system would cut into compile t…

> (an explicit primary design goal, to the point where the Go compiler must be written to read each file exactly once[0], no more). Same in Rust. Thanks to the module system, the Rust compiler never rereads a file more than once. > [1] I'd need to think about this, but I don't think it'd be difficult to detect this statically[2] at compile-time and enforce that stack-allocated rvalues into channels are never used aga…

> Rust's grammar is also context-free, as far as I know.

I believe bytestrings are context-sensitive :( I don't remember the exact details here, so I could be wrong. But other than that...

Re: Fearless concurrency with Rust

#38

Post by Jweb_Guru » Fri, Apr 10, 2015, 6:53 PM UTC

Anderkent wrote: post #30

aturon wrote: post #14

Earlier quoted context omitted.

The key is that the MutexGuard is responsible for unlocking (and it does so automatically on destruction). That way, you're tying the scope for the &mut reference to the scope in which the lock is actually held.

I take it there's no way of tying the lifetime of the MutexGuard to the lifetime of the .access return, so that you don't have to have this intermediate step?

The version presented in the blog post is simplified. In actual Rust, you don't need to write .access, as it is elided using the Deref trait. So in practice there isn't an intermediate step.

Re: Fearless concurrency with Rust

#39

Post by steveklabnik » Fri, Apr 10, 2015, 6:54 PM UTC

fny wrote: post #27

steveklabnik wrote: post #16

Earlier quoted context omitted.

> Also, are there any plans to integrate tutorial-style guides similar to those from Rust for Rubyists into the Rust book? I'd been waiting for beta to drop to do a re-organization of the TOC of the book. You can see it on nightly here: http://doc.rust-lang.org/nightly/book/ I've carved out a whole section, "Effective Rust", specifically for this kind of thing. I'm looking for a better name than "Effective Rust", sin…

Never new Rust by Example existed until today actually! You really should link to it somewhere more prominent. I'm fairly far along, but would have loved the example book had I know about it sooner.

It was originally a community project, but was then abandoned for a while, and then donated to Rust proper. As such, it's always been sort of secondary. I plan on linking it more prominently once I've actually gone through it and cleaned it up to my own personal standards, for now I just keep CI green. As such, the text has gotten out of sync with the code in a number of places, and I don't want to mislead anyone.

We do link it from http://doc.rust-lang.org/ , but I guess that still says community...

Re: Fearless concurrency with Rust

#40

Post by chimeracoder » Fri, Apr 10, 2015, 6:54 PM UTC

pcwalton wrote: post #34

chimeracoder wrote: post #29

Earlier quoted context omitted.

First, please don't compare Go and Rust. They are completely different languages with completely different target use cases. Go has very different design goals than Rust, so very little that Rust does would actually be possible in Go, and vice versa. > Go code often passes references over channels, which results in shared memory between two threads with no locking Having a complex type system would cut into compile t…

> (an explicit primary design goal, to the point where the Go compiler must be written to read each file exactly once[0], no more). Same in Rust. Thanks to the module system, the Rust compiler never rereads a file more than once. > [1] I'd need to think about this, but I don't think it'd be difficult to detect this statically[2] at compile-time and enforce that stack-allocated rvalues into channels are never used aga…

> > (an explicit primary design goal, to the point where the Go compiler must be written to read each file exactly once[0], no more).

> Same in Rust. Thanks to the module system, the Rust compiler never rereads a file more than once.

As I said right at the beginning of my post, I'm not trying to compare Rust and Go directly, because I don't think that's meaningful. I'm explaining why these particular features would be difficult to incorporate into Go. Note that I never said that Rust reads a file more than once, or that Rust's grammar is not context-free. In fact, the word "Rust" doesn't appear anywhere in my comment at all except in that very first paragraph.

I love talking about PLT and would otherwise be interested in having a discussion about static analysis and hearing why you think it would not solve the problem, but I have to say, it's both frustrating and discouraging to post an in-depth response and then get downvoted twice, with the only reply being one which very clearly ignores the very first line of my entire response.