Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How Rust optimizes async/await

tmandry.gitlab.io

Post reply on HN
Print view View original

51–60 of 130 posts

Re: How Rust optimizes async/await

#51

Post by jnwatson »

zackmorris wrote: post #50

This is one of the most concise tutorials on how generators, coroutines and futures/promises are related (from first principles) that I've seen. I'm hopeful that eventually promises and async/await fade into history as a fad that turned out to be too unwieldy. I think that lightweight processes with no shared memory, connected by streams (the Erlang/Elixer, Go and Actor model) are the way to go. The advantage to usin…

Regarding determinism, async is way more deterministic than multiple threads, because you don’t have arbitrary point where execution contexts can change.

Re: How Rust optimizes async/await

#52

Post by continuational »

zwkrt wrote: post #44
continuational wrote: post #9

I don't quite follow. What exactly is the overhead that other languages have for futures that is eliminated here?

So for example, in Kotlin each piece of synchronous code within an async function is compiled into what is essentially a Java Runnable object, which must be allocated on the heap.

Ah. What kind of asynchronous task executes so fast that a heap allocation is measurable?

Re: How Rust optimizes async/await

#53

Post by tmandry »

jcranmer wrote: post #48
rapsey wrote: post #24

Earlier quoted context omitted.

Pre-await Tokio code looks way worse to me.

A lot of that, I suspect, is that Rust's borrow checker makes specifying ownership correctly for callback chains extremely painful and unergonomic. In my experience, closure-based async programming is only sufficiently painless in managed languages to be worth using. When you're dealing with manual memory management (or, rather, smart pointer memory management), you spend a lot more time trying to make the captures h…

I'd agree with this, and emphasize the point that this stuff is really tricky to get right without GC. Fighting the borrow checker is somewhat expected when you're dealing with this level of inherent complexity in your memory management.

One of the key reasons for shipping async/await is that it erases almost all of this difficulty and lets you write straight-line code again.

Re: How Rust optimizes async/await

#54

Post by tmandry »

zackmorris wrote: post #50

This is one of the most concise tutorials on how generators, coroutines and futures/promises are related (from first principles) that I've seen. I'm hopeful that eventually promises and async/await fade into history as a fad that turned out to be too unwieldy. I think that lightweight processes with no shared memory, connected by streams (the Erlang/Elixer, Go and Actor model) are the way to go. The advantage to usin…

It's the same underlying mechanism for generators as for futures: they are stackless coroutines. All the space they need for local variables is allocated ahead of time.

In my experience, the fact that they are stackless is not at all obvious when you're coding with them. Rust makes working with them really simple and intuitive.

Re: How Rust optimizes async/await

#55

Post by leshow »

jnordwick wrote: post #33
oconnor663 wrote: post #22

Earlier quoted context omitted.

See these examples of how async/await simplifies looping and error handling: https://docs.rs/dtolnay/0.0.3/dtolnay/macro._01__await_a_min... . Await syntax makes it much easier to compose asynchronous code from different libraries, in the same way that ordinary function calls compose synchronous code. Talking to epoll/mio directly is certainly possible, but it's hard to make two different libraries work well together…

everybody always points to the 5 line basics, never to how complex async/await gets in a real system with timers, tuneouts, multiple errors types that need to be handled in different ways, heartbeats, side channel info, etc... It isn't better at that point. Cool, you can write a 5 line echo server easier, but good luck with a high performance server handling important transactions. (If there is documentation on how t…

Probably because you're just looking at examples; that's selection bias. The combinators/manual implementing Future approach to async IO gets even worse at scale, async/await is infinitely easier. If you don't like it, nobody is forcing you to use it.

Re: How Rust optimizes async/await

#56

Post by saurik »

Does anyone know how Rust's implementation compares to C++2a's? The C++ people seem to have spent a lot of time creating an extremely generic framework for async/await wherein it is easy to change out how the scheduler works (I currently have a trivial work stack, but am going to be moving to something more akin to a deadline scheduler in the near future for a large codebase I am working with, which needs to be able to associate extra prioritization data into the task object, something that is reasonably simple to do with await_transform). I am also under the impression that existing implementation in LLVM already does some of these optimizations that Rust says they will get around to doing (as the C++ people also care a lot about zero-cost).

Re: How Rust optimizes async/await

#57

Post by kbenson »

jnordwick wrote: post #46
pcwalton wrote: post #43

Earlier quoted context omitted.

If you think that Rust doesn't care about addressing the needs of C++ users, I'm not sure what to tell you. It's not like C++ only focuses on features with zero overhead. The zero-overhead principle means that you don't pay for what you don't use. It doesn't mean that compiler developers are forbidden from working on any language or library feature that consumes more than zero cycles at runtime.

So an opt-in GC would be considered zero-cost abstraction?

I think the issue is that the zero-cost abstraction terminology is used in two ways often. First, to mean a feature you can use and costs nothing because the compiler will change it into a more common form for you (such as functional programming approaches where it will take multiple filters and map passes and turn it into a single loop), and second, to mean that features added that do add overhead, only do so when you use them (as in, there's no runtime to manage M:N threading unless you include a crate that does so, because developers not using that feature shouldn't have to pay for it).

I think the terminology here is looser than it should be, and these are not really the same thing, even though I've seen zero-cost abstractions used to describe them both. It causes confusion sometimes, as I believe it is doing here.

Re: How Rust optimizes async/await

#58

Post by fnord77 »

I'm a big user of Rust, but I'm kinda dismayed that async IO wasn't part of the original design.

It's nice they're making Futures a zero-cost abstraction, but it feels like it is at the expense of ergonomics.

Re: How Rust optimizes async/await

#59

Post by tmandry »

saurik wrote: post #56

Does anyone know how Rust's implementation compares to C++2a's? The C++ people seem to have spent a lot of time creating an extremely generic framework for async/await wherein it is easy to change out how the scheduler works (I currently have a trivial work stack, but am going to be moving to something more akin to a deadline scheduler in the near future for a large codebase I am working with, which needs to be able…

Disclaimer: I'm not an expert on the proposal, but have looked at it some, and can offer my impressions here. (Sorry, this got a bit long!)

The C++ proposal definitely attacks the problem from a different angle than Rust. One somewhat surface-level difference is that it implements co_yield in terms of co_await, which is the opposite of Rust implementing await in terms of yield.

Another difference is that in Rust, all heap allocations of your generators/futures are explicit. In C++, technically every initialization of a sub-coroutine starts defaults to being a new heap allocation. I don't want to spread FUD: my understanding is that the vast majority of these are optimized out by the compiler. But one downside of this approach is that you could change your code and accidentally disable one of these optimizations.

In Rust, all the "state inlining" is explicitly done as part of the language. This means that in cases where you can't inline state, you must introduce an explicit indirection. (Imagine, say, a recursive generator - it's impossible to inline inside of itself! When you recurse, you must allocate the new generator on the heap, inside a Box.)

To be clear, the optimizations I'm talking about in the blog post are all implemented today. I'll be covering what they do and don't do, as well as future work needed, in future blog posts.

One benefit of C++ that you allude to is that there are a lot of extension points. I admit to not fully understanding what each one of them is for, but my feeling is that some of it comes from approaching the problem differently. Some of it absolutely represents missing features in Rust's initial implementation. But as I say in the post, we can and will add more features on a rolling basis.

The way I would approach the specific problem you mention is with a custom executor. When you write the executor, you control how new tasks are scheduled, and can add an API that allows specifying a task priority. You can also allow modifying this priority within the task: when you poll a task, set a thread-local variable to point to that task. Then inside the task, you can gain a reference to yourself and modify your priority.

Re: How Rust optimizes async/await

#60

Post by tmandry »

fnord77 wrote: post #58

I'm a big user of Rust, but I'm kinda dismayed that async IO wasn't part of the original design. It's nice they're making Futures a zero-cost abstraction, but it feels like it is at the expense of ergonomics.

Futures were developed outside Rust core, in a third-party library, before being brought into the language. Working with them in combinator form definitely was less ergonomic, but async/await fixes that.
Post reply on HN