Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Futurelock: A subtle risk in async Rust

rfd.shared.oxide.computer

Post reply on HN
Print view View original

131–140 of 251 posts

Re: Futurelock: A subtle risk in async Rust

#131

Post by jcalvinowens »

I have very little Rust experience... but I'm hung up on this:

> The lock is given to future1

> future1 cannot run (and therefore cannot drop the Mutex) until the task starts running it.

This seems like a contradiction to me. How can future1 acquire the Mutex in the first place, if it cannot run? The word "given" is really odd to me.

Why would do_async_thing() not immediately run the prints, return, and drop the lock after acquiring it? Why does future1 need to be "polled" for that to happen? I get that due to the select! behavior, the result of future1 is not consumed, but I don't understand how that prevents it from releasing the mutex.

It's more typical in my experience that the act of granting the lock to a thread is what makes it runnable, and it runs right then. Having to take some explicit second action to make that happen seems fundamentally broken to me...

EDIT: Rephrased for clarity.

Re: Futurelock: A subtle risk in async Rust

#132

Post by hu3 »

mdasen wrote: post #104

I rewrote this in Go and it also deadlocks. It doesn't seem to be something that's Rust specific. I'm going to write down the order of events. 1. Background task takes the lock and holds it for 5 seconds. 2. Async Thing 1 tries to take the lock, but must wait for background task to release it. It is next in line to get the lock. 3. We fire off a goroutine that's just sleeping for a second. 4. Select wants to find a c…

Yeah but Go makes it abvious why it is deadlocking because the async primitives are more explicit. Even a dumb LLM could have told us where the problem is (I tested).

Menawhile in Rust it looks like it took thousands of dollars in engineering time to find the issue.

Re: Futurelock: A subtle risk in async Rust

#133

Post by toast0 »

the__alchemist wrote: post #32
jacquesm wrote: post #8

If any rust designers are lurking about here: what made you decide to go for the async design pattern instead of the actor pattern, which - to me at least - seems so much cleaner and so much harder to get wrong? Ever since I started using Erlang it felt like I finally found 'the right way' when before then I did a lot of work with sockets and asynchronous worker threads. But even though it usually worked as advertise…

Your application needs concurrency. So, the answer is... switch your entire application, code style, and libraries it uses into a separate domain that is borderline incompatible with normal one? And has its own dialects that have their own compatibility barriers? Doesn't make sense to me.

It's easier to write all applications, concurrent or not, in a style that works well for concurrency. Lots of applications can benefit from concurrency.

You can do straight line, single threaded, non concurrent code in an actor model. Mostly, that's what most of the actor code will look like. Get a message, update local state in a straight forward way, send a response, repeat.

Re: Futurelock: A subtle risk in async Rust

#134

Post by oconnor663 »

jcalvinowens wrote: post #131

I have very little Rust experience... but I'm hung up on this: > The lock is given to future1 > future1 cannot run (and therefore cannot drop the Mutex) until the task starts running it. This seems like a contradiction to me. How can future1 acquire the Mutex in the first place, if it cannot run? The word "given" is really odd to me. Why would do_async_thing() not immediately run the prints, return, and drop the lock…

> This seems like a contradiction to me. How can future1 acquire the Mutex in the first place, if it cannot run? The word "given" is really odd to me.

`future1` did run for a bit, and it got far enough to acquire the mutex. (As the article mentioned, technically it took a position in a queue that means it will get the mutex, but that's morally the same thing here.) Then it was "paused". I put "paused" in scare quotes because it kind of makes futures sound like processes or threads, which have a "life of their own" until/unless something "interrupts" them, but an important part of this story is that Rust futures aren't really like that. When you get down to the details, they're more like a struct or a class that just sits there being data unless you call certain methods on it (repeatedly). That's what the `.await` keyword does for you, but when you use more interesting constructs like `select!`, you start to get more of the details in your face.

It's hard to be more concrete than that without getting into an overwhelming amount of detail. I wrote a set of blog posts that try to cover it without hand-waving the details away, but they're not short, and they do require some Rust background: https://jacko.io/async_intro.html

Re: Futurelock: A subtle risk in async Rust

#135

Post by vlovich123 »

Rusky wrote: post #85
filleduchaos wrote: post #76

Earlier quoted context omitted.

I feel there's a difference between a preference and a flaw. Rust has targets that make anything except inert futures simply unworkable, and in my opinion it's entirely valid for a programming language to prioritise those targets.

The requirement is that the futures are not separate heap allocations, not that they are inert. It's not at all obvious that Rust's is the only possible design that would work here. I strongly suspect it is not. In fact, early Rust did some experimentation with exactly the sort of stack layout tricks you would need to approach this differently. For example, see Graydon's post here about the original implementation of…

If it’s not inert, how do you use async in the kernel or microcontrollers? A non-inert implementation presumes a single runtime implementation within std+compiler and not usable in environments where you need to implement your own meaning of dispatch.

Re: Futurelock: A subtle risk in async Rust

#136

Post by jitl »

zenmac wrote: post #120
mjevans wrote: post #110

Earlier quoted context omitted.

Thank you. Every time I've tried to approach the concept of Rust's parallelism this is what rubs me the wrong way. I haven't yet read a way to prove it's correct, or even to reasonably prove a given program's use is not going to block. With more traditional threads my mental model is that _everything_ always has to be interrupt-able, have some form of engineer chosen timeout for a parallel operation, and address fail…

Guess Rust is more built for memory safety not concurrency? Erlang maybe? Why can't we just have a language that is memory safe and built for concurrency? Like Ocaml and Erlang combine?

Are you looking for Gleam? Simple but powerful typed functional language for BEAM and JavaScript. It’s a bit high level compared to Ocaml in terms of needing a thick runtime and being somewhat far from machine code.

Really beautiful language design imo. Does a great job avoiding the typelevel brainfuck problem I have with Haskell.

https://gleam.run/

Re: Futurelock: A subtle risk in async Rust

#137

Post by fpoling »

tony69 wrote: post #64
fpoling wrote: post #44

Earlier quoted context omitted.

Rust async still uses a native stack which just a form of memory allocator that uses LIFO order. And controlling stack usage in the embedding world is just as important as not relying on the system allocator. So its a pity that Rust async design tried so hard to avoid any explicit allocations rather than using an explicit allocator that embedding can use to preallocate and reuse objects.

Stack allocation/deallocation does not fragment memory, that’s a yuge difference for embedded systems and the main reason to avoid the heap

Even with the stack the memory can fragment. Just consider one created 10 features on the stack and the last completed last. Then memory for the first 9 will not be released until the last completes.

This problem does not happen with a custom allocator where things to allocate are of roughly the same size and allocator uses same-sized cells to allocate.

Re: Futurelock: A subtle risk in async Rust

#138

Post by jitl »

mjevans wrote: post #128
danielheath wrote: post #126

Earlier quoted context omitted.

Does that imply a lot of syscalls to get the monotonic clock value? Or is there another way to do that?

If the scheduler is doing _any_ sort of accounting at all to figure out any remote sort of fairness balancing at all, then whatever resolution that is probably works. At least for Linux, offhand, popular task scheduler frequencies used to be 100 and 1000hz. Looks like the Kernel's tracking that for tasks: https://www.kernel.org/doc/html/latest/scheduler/sched-desig... "In CFS the virtual runtime is expressed and trac…

The issue is that no scheduler manages futures. The scheduler sees tasks, futures are just a struct. See discussion of embedded above: there is no “kernel esque” parallel thread

Re: Futurelock: A subtle risk in async Rust

#139

Post by 0x1ceb00da »

levodelellis wrote: post #84

In October alone I seen 5+ articles and comments about multi-threading and I don't know why I always said if your code locks or use atomics, it's wrong. Everyone says I'm wrong but you get things like what's described in the article. I'd like to recommend a solution but there's pretty much no reasonable way to implement multi-threading when you're not an expert. I heard Erlang and Elixir are good but I haven't tried…

> I always said if your code locks or use atomics, it's wrong.

Why atomics?

Post reply on HN