A four year plan for async Rust

One thing about Rust is that it's a fairly young language. We've had to implement a number of things ourselves that we wouldn't have had to implement in other languages.

If there's not enough community interest for there to already exist a solution for your problem, you really only have three options: write it yourself (including opening a PR to add it to an existing implementation), pay someone to write it, or switch languages. If none of these are options, you're just out of luck, and complaining about it is unlikely to do any good. I'm not saying you can't complain of course, but I am saying that you're more likely to get what you want via another path.

But regardless, I don't think it's an odd argument. Indeed, no one is stopping anyone from writing a new, low-level, memory-safe programming language! That's why Rust exists! Turns out the market was huge! Writing a new one now would be easier because of all the great ideas that Rust helped to prove work at scale. We're seeing this already with the success of languages like Zig, which makes different tradeoffs than Rust did and seems to be finding success in slightly different niches.

And sort of disproving your point, we use postgres, and I can think of three different implementations of postgres drivers offhand (sqlx, tokio-postgres, and diesel). I'll also note that the author of tokio-postgres also publishes https://docs.rs/postgres/latest/postgres/, which is not async! It's impressive how many options we have for such a complex thing in such a young ecosystem.

> Most people aren't re-writing NGINX, after all (hopefully...).

those people probably may use another language (java, go, c#) if absolute performance is not critical for them.

> Probably worth revisiting those discussions to see where this has been thought through by the Rust team

The issue generated a lot of hot air at the time, far more than I have time to read, but I think I have the gist of it.

The performance issues were due to an implementation choice. All green thread implementations I'm aware of hide the code colouring event driven I/O introduces, and Rust did the same. The hiding has to be done at run time. That translates to every I/O call has at some point chose the blocking or non-blocking implementation. This is typically done using vtables (dyn in Rust), but whatever mechanism is used, it introduces runtime overhead. Worse, it slows down everything - including code that doesn't use green threads. It gets radically worse if you try to hide C calls blocking.

Async has the same issue, some solves it by making coloured code the programmers problem. If they had of made the same design decision for green threads the performance issues go away. We don't have to speculate about that. There is a green thread crate out there called "may", then has independent benchmarks covering it, async Rust implementations, and other languages. "may" beat everything at one point, but it's a very noisy benchmark so the only conclusion I would draw from it is that "may" looks to run at the same speed as async.

As for the rest of the issues: they revolve around needing a separate stack. I tried to cover the trade-offs above. Summarising: green threads lead to simpler code that's easier to write and theoretically could run faster than async, but have larger setup overhead and use more memory.

> But, generators tend to be short lived. You tend to use them to iterate over an array, rather than serving a web request (the typical use a task is put to). That means the overhead of creating the stack for a green thread isn't a small proportion of the time a long running task takes, but could well dominate the time it takes to iterate over a small array. Thus async is a much better fit for iterators.

The current async approach requires the compiler to statically calculate a fixed size for each invocation context. Except when it can't, in which case you have to dynamically allocate space for each invocation. That same logic could be used to transparently optimize stack creation, opportunistically avoiding both a pessimistically large stack and the costly guard pages. The compiler could even choose to instantiate the generator stack on the caller's stack, just as today.

async Rust optimizes the common case but completely neglects the hard case. In theory Rust users should be able to have their cake and eat it too, especially given that the difficult static analysis work already exists to support the current async model.

(I've made this point before and feel like I may have forgotten some counterpoints. I apologize in advance if that's so.)

Async isn’t new in programming language theory. It’s a syntax sugar for state machines and continuations. I think it could be argued that PLT was already way ahead of async/await – monads are more general than futures, and Rust’s async wasn’t generalized to be an effect system.

Also it’s simply not true that async wasn’t fully understood or carefully evaluated. It took years to design, and then bikeshed every detail, to the point people involved were burned out. It had multiple prototypes, and an early callback-based implementation used by hundreds of libraries, in production, for over a year. It’s probably the most thoroughly designed and tested feature in Rust’s history.