Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Futurelock: A subtle risk in async Rust

rfd.shared.oxide.computer

Post reply on HN
Print view View original

11–20 of 251 posts

Re: Futurelock: A subtle risk in async Rust

#11

Post by raggi »

cogman10 wrote: post #5
Sytten wrote: post #3

I am wondering if there is a larger RFC for Rust to force users to not hold a variable across await points. In my mind futurelock is similar to keeping a sync lock across an await point. We have nothing right now to force a drop and I think the solution to that problem would help here.

The ideas that have been batted around is called "async drop" [1] And it looks like it's still just an unaddressed well known problem [2]. Honestly, once the Mozilla sackening of rust devs happened it seems like the language has been practically rudderless. The RFC system seems almost dead as a lot of the main contributors are no longer working on rust. This initiative hasn't had motion since 2021. [3] [1] https://ru…

Those pages are out of date, and AsyncDrop is in progress: https://github.com/rust-lang/rust/issues/126482

I think "practically rudderless" here is fairly misinformed and a little harmful/rude to all the folks doing tons of great work still.

It's a shame there are some stale pages around and so on, but they're not good measures of the state of the project or ecosystem.

The problem of holding objects across async points is also partially implemented in this unstable lint marker which is used by some projects: https://dev-doc.rust-lang.org/unstable-book/language-feature...

You also get a similar effect in multi-threaded runtimes by not arbitrarily making everything in your object model Send and instead designing your architecture so that most things between wake-ups don't become arbitrarily movable references.

These aren't perfect mitigations, but some tools.

Re: Futurelock: A subtle risk in async Rust

#12

Post by ameliaquining »

Sytten wrote: post #3

I am wondering if there is a larger RFC for Rust to force users to not hold a variable across await points. In my mind futurelock is similar to keeping a sync lock across an await point. We have nothing right now to force a drop and I think the solution to that problem would help here.

There's an existing lint that lets you prohibit instances of specific types from being held across await points: https://rust-lang.github.io/rust-clippy/stable/index.html#aw...

Re: Futurelock: A subtle risk in async Rust

#13

Post by raggi »

moralestapia wrote: post #9

Hmm, curious to see if this could happen on JS. I'll reproduce the code.

yes, you can produce similar issues with promise guarded states and so on as well, it's a fairly common issue in async programming, but can be surprising when it's hidden by layers of abstraction / far up/down a call-chain.

Re: Futurelock: A subtle risk in async Rust

#14

Post by dvt »

> &mut future1 is dropped, but this is just a reference and so has no effect. Importantly, the future itself (future1) is not dropped.

There's a lot of talk about Rust's await implementation, but I don't really think that's the issue here. After all, Rust doesn't guarantee convergence. Tokio, on the other hand (being a library that handles multi-threading), should (at least when using its own constructs, e.g. the `select!` macro).

So, since the crux of the problem is the `tokio::select!` macro, it seems like a pretty clear tokio bug. Side note, I never looked at it before, but the macro[1] is absolutely hideous.

[1] https://docs.rs/tokio/1.34.0/src/tokio/macros/select.rs.html

Re: Futurelock: A subtle risk in async Rust

#15

Post by raggi »

jacquesm wrote: post #8

If any rust designers are lurking about here: what made you decide to go for the async design pattern instead of the actor pattern, which - to me at least - seems so much cleaner and so much harder to get wrong? Ever since I started using Erlang it felt like I finally found 'the right way' when before then I did a lot of work with sockets and asynchronous worker threads. But even though it usually worked as advertise…

_an answer_ is performance - the necessity of creating copyable/copied messages for inter-actor communication everywhere in the program _can be_ expensive.

that said there are a lot of parts of a lot of programs where a fully inlined and shake optimized async state machine isn't so critical.

it's reasonable to want a mix, to use async which can be heavily compiler optimized for performance sensitive paths, and use higher level abstractions like actors, channels, single threaded tasks, etc for less sensitive areas.

Re: Futurelock: A subtle risk in async Rust

#16

Post by raggi »

dvt wrote: post #14

> &mut future1 is dropped, but this is just a reference and so has no effect. Importantly, the future itself (future1) is not dropped. There's a lot of talk about Rust's await implementation, but I don't really think that's the issue here. After all, Rust doesn't guarantee convergence. Tokio, on the other hand (being a library that handles multi-threading), should (at least when using its own constructs, e.g. the `se…

i forget if this part unwinds to the exact same place, but some of this kind of design constraint in tokio stems from the much earlier language capabilities and is prohibitive to adjust without breaking the user ecosystem.

one of the key advertised selling points in some of the other runtimes was specifically around behavior of tasks on drop of their join handles for example, for reasons closely related to this post.

Re: Futurelock: A subtle risk in async Rust

#17

Post by lll-o-lll »

sunshowers wrote: post #10
jacquesm wrote: post #8

If any rust designers are lurking about here: what made you decide to go for the async design pattern instead of the actor pattern, which - to me at least - seems so much cleaner and so much harder to get wrong? Ever since I started using Erlang it felt like I finally found 'the right way' when before then I did a lot of work with sockets and asynchronous worker threads. But even though it usually worked as advertise…

Not a Rust designer, but a big motivation for Rust's async design was wanting it to work on embedded, meaning no malloc and no threads. This unfortunately precludes the vast majority of the design space here, from active futures as seen in JS/C#/Go to the actor model. You can write code using the actor model with Tokio. But it's not natural to do so.

As a curious bystander, it will be interesting to see how the Zig async implementation pans out. They have the advantage of getting to see the pitfalls of those that have come before.

Getting back to Rust, even if not natural, I agree with the parent that the actor model is simply the better paradigm. Zero runtime allocation should still be possible, you just have to accept some constraints.

I think async looks simple because it looks like writing imperative code; unfortunately it is just obfuscating the complex reality underlying. The actor model makes things easier to reason about, even if it looks more complicated initially.

Re: Futurelock: A subtle risk in async Rust

#18

Post by the__alchemist »

jacquesm wrote: post #8

If any rust designers are lurking about here: what made you decide to go for the async design pattern instead of the actor pattern, which - to me at least - seems so much cleaner and so much harder to get wrong? Ever since I started using Erlang it felt like I finally found 'the right way' when before then I did a lot of work with sockets and asynchronous worker threads. But even though it usually worked as advertise…

I'm surprised learning this too. I know the hobby embedded, and HTTP-server OSS ecosystem have committed to Async, but I didn't expect Oxide would.

Re: Futurelock: A subtle risk in async Rust

#19

Post by lll-o-lll »

raggi wrote: post #15
jacquesm wrote: post #8

If any rust designers are lurking about here: what made you decide to go for the async design pattern instead of the actor pattern, which - to me at least - seems so much cleaner and so much harder to get wrong? Ever since I started using Erlang it felt like I finally found 'the right way' when before then I did a lot of work with sockets and asynchronous worker threads. But even though it usually worked as advertise…

_an answer_ is performance - the necessity of creating copyable/copied messages for inter-actor communication everywhere in the program _can be_ expensive. that said there are a lot of parts of a lot of programs where a fully inlined and shake optimized async state machine isn't so critical. it's reasonable to want a mix, to use async which can be heavily compiler optimized for performance sensitive paths, and use hi…

I’m not sure this is actually true? Do messages have to be copied?

Re: Futurelock: A subtle risk in async Rust

#20

Post by shepmaster »

sunshowers wrote: post #10
jacquesm wrote: post #8

If any rust designers are lurking about here: what made you decide to go for the async design pattern instead of the actor pattern, which - to me at least - seems so much cleaner and so much harder to get wrong? Ever since I started using Erlang it felt like I finally found 'the right way' when before then I did a lot of work with sockets and asynchronous worker threads. But even though it usually worked as advertise…

Not a Rust designer, but a big motivation for Rust's async design was wanting it to work on embedded, meaning no malloc and no threads. This unfortunately precludes the vast majority of the design space here, from active futures as seen in JS/C#/Go to the actor model. You can write code using the actor model with Tokio. But it's not natural to do so.

> But it's not natural to do so.

I tend to write most of my async Rust following the actor model and I find it natural. Alice Rhyl, a prominent Tokio contributor, has written about the specific patterns:

https://ryhl.io/blog/actors-with-tokio/

Post reply on HN