Why you might want async in your project

Wrote Python professionally for years and didn’t know all of this. Thanks!

The state machine transformation is not specific to any async libraries. The compiler is the one that desugars async fns / blocks to state machines. AFAIK there is nothing other than dumping the HIR / MIR from rustc to inspect it. But even without that the transformation is pretty straightforward to do mentally.

The first transformation is that every async block / fn compiles to a generator where `future.await` is essentially replaced by `loop { match future.poll() { Ready(value) => break value, Pending => yield } }`. ie either polling the inner future will resolve immediately, or it will return Pending and yield the generator, and the next time the generator is resumed it will go back to the start of the loop to poll the future again.

The second transformation is that every generator compiles to essentially an enum. Every variant of the enum represents one region of code between two `yield`s, and the data of that variant is all the local variables that in the scope of that region.

Putting both together:

    async fn foo(i: i32, j: i32) {
        sleep(5).await;
        i + j
    }

... essentially compiles to:

    fn foo(i: i32, j: i32) -> FooFuture {
        FooFuture::Step0 { i, j }
    }

    enum FooFuture {
        Step0 { i: i32, j: i32 }
        Step1 { i: i32, j: i32, sleep: SleepFuture }
        Step2,
    }

    impl Future for FooFuture {
        fn poll(self) -> Poll {
            loop {
                match self {
                    Self::Step0 { i, j } => {
                        let sleep = sleep(5);
                        self = Self::Step1 { i, j, sleep };
                    }

                    Self::Step1 { i, j, sleep } => {
                        let () = match sleep.poll() {
                            Poll::Ready(()) => (),
                            Poll::Pending => return Poll::Pending,
                        };
                        self = Self::Step2;
                        return Poll::Ready(i + j);
                    }

                    Self::Step2 => panic!("already run to completion"),
                }
            }
        }
    }

Erlang is probably the closest. The word you want to search for is "port". If it doesn't seem like it at first, keep reading. It's a very idiosyncratic take on the topic of you view it from this perspective because it isn't exactly their focus. But it does have a mechanism for collecting files, sockets, open pipes to other programs, and a number of other things. Not fully generic, though.

>Except, this isn’t a problem with Rust’s async, it’s a problem with tokio. tokio uses a 'static, threaded runtime that has its benefits but requires its futures to be Send and 'static.

It's not a problem with tokio either. The author's point is specifically about the multi-threaded tokio runtime that allows tasks to be moved between worker threads, which is why it requires the tasks to be Send + 'static. Alternatively you can either a) create a single-threaded tokio runtime instead which will remove the need for tasks to be Send, or b) use a LocalSet within the current worker that will scope all tasks to that LocalSet's lifetime so they will not need to be Send or 'static.

If you go the single-threaded tokio runtime route, that doesn't mean you're limited to one worker total. You can create your own pseudo-multi-threaded tokio runtime by creating multiple OS threads and running one single-threaded tokio runtime on each. This will be similar to the real multi-threaded tokio runtime except it doesn't support moving tasks between workers, which means it won't require the tasks to be Send. This is also what the author's smol example does. But note that allowing tasks to migrate between workers prevents hotspots, so there are pros and cons to both approaches.

Biggest issue I have with async is the lack of native async traits and the lack of support for async closures. You can work around the traits issue but the closure issue you can’t. I’ve spent hours trying to work around closures that wrap async code.