Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Futurelock: A subtle risk in async Rust

rfd.shared.oxide.computer

Post reply on HN
Print view View original

151–160 of 251 posts

Re: Futurelock: A subtle risk in async Rust

#151

Post by baq »

shepmaster wrote: post #20
sunshowers wrote: post #10

Earlier quoted context omitted.

Not a Rust designer, but a big motivation for Rust's async design was wanting it to work on embedded, meaning no malloc and no threads. This unfortunately precludes the vast majority of the design space here, from active futures as seen in JS/C#/Go to the actor model. You can write code using the actor model with Tokio. But it's not natural to do so.

> But it's not natural to do so. I tend to write most of my async Rust following the actor model and I find it natural. Alice Rhyl, a prominent Tokio contributor, has written about the specific patterns: https://ryhl.io/blog/actors-with-tokio/

The ‘Beware of cycles’ section at the end has some striking similarities with futurelock avoidance recommendations from the original article… not sure what to make of this except to say that this stuff is hard?

Re: Futurelock: A subtle risk in async Rust

#152

Post by rtpg »

logicchains wrote: post #149
smallstepforman wrote: post #129

Earlier quoted context omitted.

I love Actors and have used them professionally for over 6 years (C++). However to solve real world problems I have had to introduce “locks” to the Actor framework to support various scenarios. With my home-grown actor library, this was trivial to add, however for some 3rd party actor libraries, ideology is dominant and the devs refuse to add such a purity-breaking feature to their actor framework, and hence I cannot…

What scenario requires locks that can't be solved by just having a single actor that owns the resource and controls access?

I would imagine that in... "soft realtime" might be much but in performance sensitive scenarios the actual cost to having some coordination code in that space might start mattering.

Maybe actor abstractions end up compiling away fairly nicely in Rust though!

Re: Futurelock: A subtle risk in async Rust

#153

Post by lowbloodsugar »

I know this is going to sound trite, but “don’t do that”. It’s no different than deciding to poll the win32 event queue inside an a method you executed in response to polling the event queue. Nested shit is always going to cause a bug. I guess each new generation just has to learn.

Re: Futurelock: A subtle risk in async Rust

#155

Post by gf000 »

rtpg wrote: post #97
hitekker wrote: post #62

Skimming through, this document feels thorough and transparent. Clearly, a hard lesson learned. The footnotes, in particular, caught my eye https://rfd.shared.oxide.computer/rfd/397#_external_referenc... > Why does this situation suck? It’s clear that many of us haven’t been aware of cancellation safety and it seems likely there are many cancellation issues all over Omicron. It’s awfully stressful to find out while w…

I guess one big question here is whether there's a higher layer abstraction that is available to wrap around patterns to avoid this. It does feel like there's still generally possibilities of deadlocks in Rust concurrency right? I understand the feeling here that it feels like ... uhh... RAII-style _something_ should be preventing this, because it feels like statically we should be able to identify this issue in this…

> It does feel like there's still generally possibilities of deadlocks in Rust concurrency right?

I mean, is there any generic computation model where you can't have deadlocks? Even with stuff like actors you can trivially have cycles and now your blocking primitive is just different (not CPU-level), and we call it a livelock, but it's fundamentally the same.

Re: Futurelock: A subtle risk in async Rust

#156

Post by gf000 »

embedding-shape wrote: post #113
rtpg wrote: post #97

Earlier quoted context omitted.

I guess one big question here is whether there's a higher layer abstraction that is available to wrap around patterns to avoid this. It does feel like there's still generally possibilities of deadlocks in Rust concurrency right? I understand the feeling here that it feels like ... uhh... RAII-style _something_ should be preventing this, because it feels like statically we should be able to identify this issue in this…

> I guess one big question here is whether there's a higher layer abstraction that is available to wrap around patterns to avoid this. Something like Actors, on top of Tokio, would be one way: https://ryhl.io/blog/actors-with-tokio/

Then you just replace deadlocks with livelocks, the fundamental problem AFAIK can't be avoided.

Re: Futurelock: A subtle risk in async Rust

#157

Post by gf000 »

mycoliza wrote: post #47
oconnor663 wrote: post #35

Earlier quoted context omitted.

> I wonder if there is a similar idea possible with tokio? E.g. if you are awaiting a Mutex held by a future that "can't run", then poll that future instead. Something like this could make sense for Tokio tasks . (I don't know how complicated their task scheduler is; maybe it already does stuff like this?) But it's not possible for futures within a task, as in this post. This goes all the way back to the "futures are…

Yeah, a coworker coming from Go asked a similar question about why Rust doesn't have something like the Go runtime's deadlock detector. Your comment is quite similar to the explanation I gave him. Go, unlike Rust, does not really have a notion of intra-task concurrency; goroutines are the fundamental unit of concurrency and parallelism. So, the Go runtime can reason about dependencies between goroutines quite easily,…

I think an important takeaway here that many often ignore is that in language design, not having low-level control over something is sometimes just as important design tradeoff as having it.

From that it also follows that it may not be too fruitful to try to tackle every domain there is with a single language only.

(With that said, I absolutely love sync Rust, and Go is definitely not a good example of an elegantly designed language, I am talking in a more general way here)

Re: Futurelock: A subtle risk in async Rust

#158

Post by formerly_proven »

csande17 wrote: post #121
hitekker wrote: post #62

Skimming through, this document feels thorough and transparent. Clearly, a hard lesson learned. The footnotes, in particular, caught my eye https://rfd.shared.oxide.computer/rfd/397#_external_referenc... > Why does this situation suck? It’s clear that many of us haven’t been aware of cancellation safety and it seems likely there are many cancellation issues all over Omicron. It’s awfully stressful to find out while w…

In case anyone else was confused: the link/quote in this comment are from the previous "async cancellation issue" write-up, which describes a situation where you "drop" a future: the code in the async function stops running, and all the destructors on its local variables are executed. The new write-up from OP is that you can "forget" a future (or just hold onto it longer than you meant to), in which case the code in…

async rust continues to strike me as half-baked and too complex, if you’re developing an application (as opposed to some high performance utility like e.g. a data plane component) just use threads, they’re plenty cheap and not even half as messy.

Re: Futurelock: A subtle risk in async Rust

#159

Post by dvratil »

mleonhard wrote: post #124

> // Start a background task that takes the lock and holds it for a few seconds. Holding a lock while waiting for IO can destroy a system's performance. With async Rust, we can prevent this by making the MutexGuard !Send, so it cannot be held across an await. Specifically, because it is !Send, it cannot be stored in the Future [2], so it must be dropped immediately, freeing the lock. This also prevents Futurelock dea…

I would guess this is just to make the explanation of the bug easier.

In real world, the futurelock could occur even with very short locks, it just wouldn't be so deterministic. Having a minimal reproducer that you have to run a thousand times and it will maybe futurelock doesn't really make for a good example :)

Re: Futurelock: A subtle risk in async Rust

#160

Post by sgt »

lll-o-lll wrote: post #17
sunshowers wrote: post #10

Earlier quoted context omitted.

Not a Rust designer, but a big motivation for Rust's async design was wanting it to work on embedded, meaning no malloc and no threads. This unfortunately precludes the vast majority of the design space here, from active futures as seen in JS/C#/Go to the actor model. You can write code using the actor model with Tokio. But it's not natural to do so.

As a curious bystander, it will be interesting to see how the Zig async implementation pans out. They have the advantage of getting to see the pitfalls of those that have come before. Getting back to Rust, even if not natural, I agree with the parent that the actor model is simply the better paradigm. Zero runtime allocation should still be possible, you just have to accept some constraints. I think async looks simpl…

I was wondering when someone would bring up Zig. I think it's fascinating how far it has come in the last couple of years and now the new IO interface/async implementation.

Question is - when will Zig become mature enough to become a legit choice next to say, Go or Rust?

I mean for a regular dev team, not necessarily someone who works deeply along with Andrew Kelley etc like Tigerbeetle.

Post reply on HN