Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Futurelock: A subtle risk in async Rust

rfd.shared.oxide.computer

Post reply on HN
Print view View original

221–230 of 251 posts

Re: Futurelock: A subtle risk in async Rust

#221

Post by mpeklar »

When considering this issue alongside with RFD 397, it seems to me that the problem is actually using future drops as an implicit (!) cancellation signal. This makes drop handlers responsible for handling every cancellation-related task, which they are not very good at. If a future is not immediately dropped after selecting on it, you get futurelock, and if it is, you get an async cancellation correctness problem, where the only way to try and interact with the cancellation execution flow is to use drop handlers (maybe in the form of scope guards).

Sadly, the only solution I know of is to use an explicit cancellation signal, and to modify ~everything to work with it. In that world, almost all async functions would need to accept a cancellation parameter of some sort, like a Go Context or like the tokio-utils CancellationToken, and explicitly check it every time they await a function. The new select!-equivalent would need to signal cancellations and then keep polling all unfinished cancellation-aware futures in a loop until they finished, and maybe immediately drop all non-aware futures to prevent futurelock. The entire Tokio API would need to be wrapped to take into account cancellation tokens, as well as any other async library you would want to use.

A lot of work, and you would need to do something if cancel-aware futures get dropped anyway. What a mess.

Re: Futurelock: A subtle risk in async Rust

#222

Post by smallstepforman »

logicchains wrote: post #149
smallstepforman wrote: post #129

Earlier quoted context omitted.

I love Actors and have used them professionally for over 6 years (C++). However to solve real world problems I have had to introduce “locks” to the Actor framework to support various scenarios. With my home-grown actor library, this was trivial to add, however for some 3rd party actor libraries, ideology is dominant and the devs refuse to add such a purity-breaking feature to their actor framework, and hence I cannot…

What scenario requires locks that can't be solved by just having a single actor that owns the resource and controls access?

Any scenario where you have to atomically update 2 actors. To use a simple analogy for illustrative purposes, transferring money between 2 accounts, you need to lock both actors before incrementing/decrementing. Because in the real world, the accounts can change from other pending parallel transactions and edits. Handshakes are very error prone. Lock the actor, do the critical transaction, unlock.

In a rationale world, this works. In a prejudiced world, devs fight against locks in actor models.

Hence why I had to roll my own …

Re: Futurelock: A subtle risk in async Rust

#223

Post by kibwen »

clarkmcc wrote: post #123
mdasen wrote: post #104

I rewrote this in Go and it also deadlocks. It doesn't seem to be something that's Rust specific. I'm going to write down the order of events. 1. Background task takes the lock and holds it for 5 seconds. 2. Async Thing 1 tries to take the lock, but must wait for background task to release it. It is next in line to get the lock. 3. We fire off a goroutine that's just sleeping for a second. 4. Select wants to find a c…

I think the thing that rubs me the wrong way is that Rust was supposed to be "fearless" concurrency. Go doesn't claim that title so I'm not offended when it doesn't live up to it.

Despite "fearless concurrency", Rust has been careful to never claim to prevent deadlocks/race conditions in general, in either async code or non-async code. It's certainly easier to get deadlocks in async Rust than in non-async Rust, but this isn't some sort of novel failure mode.

Re: Futurelock: A subtle risk in async Rust

#224

Post by kibwen »

tux3 wrote: post #185
vlovich123 wrote: post #135

Earlier quoted context omitted.

If it’s not inert, how do you use async in the kernel or microcontrollers? A non-inert implementation presumes a single runtime implementation within std+compiler and not usable in environments where you need to implement your own meaning of dispatch.

I think the kernel and microcontroller use-case has been overstated. A few bare metal projects use stackless coroutines (technically resumable functions) for concurrency, but it has turned out to be a much smaller use-case than anticipated. In practice C and C++ coroutines are really not worth the pain that they are to use, and Rust async has mostly taken off with heavy-duty executors like Tokio that very much don't…

> it has turned out to be a much smaller use-case than anticipated

Well, no, at the time of the design of Rust's async MVP, everyone was pretty well aware that the vast majority of the users would be writing webservers, and that the embedded use case would be a decided minority, if it ever existed at all. That Embassy exists and its ecosystem as vibrant as it is is, if anything, an unexpected triumph.

But regardless of how many people were actually expected to use it in practice, the underlying philosophy remained thus: there exist no features of Rust-the-language that are incompatible with no_std environments (e.g. Rust goes well out of its way, and introduces a lot of complexity, to make things like closures work given such constraints), and it would be exceptional and unprecedented for Rust to violate this principle when it comes to async.

Re: Futurelock: A subtle risk in async Rust

#225

Post by NobodyNada »

quietbritishjim wrote: post #208
NobodyNada wrote: post #96

Earlier quoted context omitted.

> if tokio::select!() only accepted tasks (or implicitly turned any futures it receives into tasks, like Python's asyncio.gather() does) then it wouldn't have this problem? Yes, this is correct. However, many of the use cases for select rely on the fact that it doesn't run all the tasks to completion. I've written many a select! statement to implements timeouts or other forms of intentionally preempting a task. Somet…

> However, many of the use cases for select rely on the fact that it doesn't run all the tasks to completion. That seems like a different requirement than "all arguments are tasks". If I understand it right (and quite possibly I don't), making them all tasks means that they are all polled and therefore continue progressing until they are dropped. It doesn't mean that select! would have to run them all the way to comp…

I was sloppy with my wording, I should have said "it doesn't run all the futures to completion".

> making them all tasks means that they are all polled and therefore continue progressing until they are dropped. It doesn't mean that select! would have to run them all the way to completion.

This is exactly correct, but oftentimes the reason you're using select is because you don't want to run the futures all the way to completion. In my experience, the most common use cases for select are:

- An event handler loop that receives input from multiple channels. You could replace this with multiple tasks, one reading from each channel; but this could potentially mess with your design for queueing / backpressure -- often it's important for the loop to pause reading from the channels while processing the event.

- An operation that's run with a timeout, or a shutdown event from a controlling task. In this case I want the future to be dropped when the task is cancelled.

The example in the original post was the second case: an operation with a timeout. They wanted the operation to be cancelled when the timeout expired, but because the select statement borrowed the future, it only suspended the future instead of cancelling it. This is a very common code pattern when calling select! in a loop, when you want a future to be resumed instead of restarted on the next loop iteration -- it's very intentional that select! allows you to use either way, because you often want either behavior.

Re: Futurelock: A subtle risk in async Rust

#227

Post by Veserv »

oconnor663 wrote: post #144
Veserv wrote: post #54

Earlier quoted context omitted.

I thought Rust async is a colored stackless coroutine model and thus it would be unsafe to continue execution of previously executing async functions. To explain, generally speaking, stackless coroutine async only need coloring because they are actually “independent stack”less coroutines. What they actually do is that they share the stack for their local state. This forces async function execution to proceed in LIFO…

Rust futures are "just" structs with a poll() method. The poll() method is a function like any other, so it can have local variables on the stack as usual, but anything it wants to save between calls needs to be a field of the struct instead of a stack local. The magic of async/await is that the compiler figures out which of your async function's variables need to be fields on that struct, and it generated the struct…

I see. The Rust implementation effectively splats out the transitive closure of all your callee stack frames upfront which would enable continuing previously executing async functions.

Re: Futurelock: A subtle risk in async Rust

#228

Post by samdoesnothing »

mustache_kimono wrote: post #148
samdoesnothing wrote: post #143

LOL. All the Rust evangelists talk about safety when stuff like this exists? JFC. Can we stop calling Rust safe now? Finally? I mean we all know deep in our hearts that trivial memory safety doesn't mean programs are correct or "safe" by any means but its nice to have proof that Rust is fundamentally unsafe for asynchronous tasks at least. Or at least "unsound". Structured concurrency will always win IMO.

> All the Rust evangelists talk about safety when stuff like this exists? JFC. Deadlocks can happen anywhere? You can replicate this pattern in golang.

Golang doesn't have legions of evangelicals claiming it's a safe language and everything should be written in it.

Re: Futurelock: A subtle risk in async Rust

#229

Post by dap »

nialv7 wrote: post #165
csande17 wrote: post #121

Earlier quoted context omitted.

In case anyone else was confused: the link/quote in this comment are from the previous "async cancellation issue" write-up, which describes a situation where you "drop" a future: the code in the async function stops running, and all the destructors on its local variables are executed. The new write-up from OP is that you can "forget" a future (or just hold onto it longer than you meant to), in which case the code in…

Yeah, Rust mostly just eliminates memory safety and data race problems, which is an enormous improvement compared to what we had previously. Unfortunately right now if you really want to write software that's guaranteed to be correct, there's not alternative to formal verification.

I would say it can go further than that: Rust enables you to construct many APIs in a way that can’t be misused. It’s not at all unique in this way, but compared with C or Go or the like, you can encode so many more constraints in types.

Re: Futurelock: A subtle risk in async Rust

#230

Post by tux3 »

kibwen wrote: post #224
tux3 wrote: post #185

Earlier quoted context omitted.

I think the kernel and microcontroller use-case has been overstated. A few bare metal projects use stackless coroutines (technically resumable functions) for concurrency, but it has turned out to be a much smaller use-case than anticipated. In practice C and C++ coroutines are really not worth the pain that they are to use, and Rust async has mostly taken off with heavy-duty executors like Tokio that very much don't…

> it has turned out to be a much smaller use-case than anticipated Well, no, at the time of the design of Rust's async MVP, everyone was pretty well aware that the vast majority of the users would be writing webservers, and that the embedded use case would be a decided minority, if it ever existed at all. That Embassy exists and its ecosystem as vibrant as it is is, if anything, an unexpected triumph. But regardless…

Point taken, I might have formed the wrong impression at the time.

With my C++ background, I'm very much at home with that philosophy, but I think there is room for nuance in how strictly orthodox we are.

C++ does have optional language features that introduce some often unwelcone runtime overhead, like RTTI and unwinding.

Rust does not come configured for freestanding environments out of the box either. Like C++, you are opting out of language features like unwinding as well as the standard library when going freestanding.

I want to affirm that I'm convinced Rust is great for embedded. It's more that I mostly love async when I get to use it for background I/O with a full fledged work stealing thread-per-core marvel of engineering like tokio!

In freestanding Rust the I/O code is platform specific, suddenly I'd have to write the low-level async code myself, and it's not clear this makes the typical embedded project that much higher performance, or all that easy to maintain.

So, I don't want to say anything too radical. But I think the philosophy doesn't have to be as clear cut as no language feature ever incompatible with no-std. Offering a std only language feature is not necessarily closing a door to embedded. We sort of already make opt-out concessions to have a friendlier experience for most people.

(Apologies for the wall of text)

Post reply on HN