Live data from Hacker News

How Rust optimizes async/await

tmandry.gitlab.io

91–100 of 130 posts

Re: How Rust optimizes async/await

#91

As a reminder, you don't need to use async/await to implement socket servers in Rust. You can use threads, and they scale quite well. M:N scheduling was found to be slower than 1:1 scheduling on Linux, which is why Rust doesn't use that solution. Async/await is a great feature for those who require performance beyond what threads (backed by either 1:1 or M:N implementations) can provide. One of the major reasons behi…

To implement, no. To make a performant server- yes. Most systems have limited number of threads (low limit constant compiled with the kernel), and each thread is triggered by the scheduler, not by network events, which is very uneconomical.

You with application level concurrency you get 100x performance boost for network servers.

Re: How Rust optimizes async/await

#92
post #77

As a reminder, you don't need to use async/await to implement socket servers in Rust. You can use threads, and they scale quite well. M:N scheduling was found to be slower than 1:1 scheduling on Linux, which is why Rust doesn't use that solution. Async/await is a great feature for those who require performance beyond what threads (backed by either 1:1 or M:N implementations) can provide. One of the major reasons behi…

Curious what is the fundamental difference that makes Go do M:N thread efficiently? Considering that the compiler has far less information than Rust about the program.

Go is more efficient at M:N then Rust can be mostly for two reasons:

1. Go can start stacks small and relocate them, because the runtime needed to implement garbage collection allows relocation of pointers into the stack by rewriting them. Rust has no such runtime infrastructure: it cannot tell what stack values correspond to pointers and which correspond to other values. Additionally, Rust allows for pointers from the heap into the stack in certain circumstances, which Go does not (I don't think, anyway). So what Rust must do is to reserve a relatively large amount of address space for each thread's stack, because those stacks cannot move in memory. (Note that in the case of Rust the kernel does not have to, and typically does not, actually allocate that much physical memory for each thread until actually needed; the reservation is virtual address space only.) In contrast, Go can start thread stacks small, which makes them faster to allocate, and copy them around as they grow bigger. Note that async/await in Rust has the potential to be more efficient than even Go's stack growth, as the runtime can allocate all the needed space up front in some cases and avoid the copies; this is the consequence of async/await compiling to a static state machine instead of the dynamic control flow that threads have.

2. Rust cares more about fast interoperability with C code that may not be compiled with knowledge of async I/O and stack growth. Go chooses fast M:N threading over this, sacrificing fast FFI in the process as every cgo call needs to switch to a big stack. This is just a tradeoff. Given that 1:1 threading is quite fast and scalable on Linux, it's the right tradeoff for Rust's domain, as losing M:N threading isn't losing that much anyway.

Re: How Rust optimizes async/await

#93
post #85
post #80

Earlier quoted context omitted.

I find async/await easier to reason about than threads for anything more involved than the 1 request per thread web server use case. This is because you avoid bringing in the abstraction of threads (or green threads) and their communication with one another. You trade syntactical complexity (what color is your function, etc), for semantic complexity (threads, channels, thread safety, lock races).

They have the same semantic complexity. You have tasks in async/await and you still need to deal with inter-task communication, locking, etc.

The main difference is that in async/await you control where context switches occur and the syntax (.await) explicitly points them out. This means you can often avoid locks and do things in a more straightforward manner.

Re: How Rust optimizes async/await

#94

As a reminder, you don't need to use async/await to implement socket servers in Rust. You can use threads, and they scale quite well. M:N scheduling was found to be slower than 1:1 scheduling on Linux, which is why Rust doesn't use that solution. Async/await is a great feature for those who require performance beyond what threads (backed by either 1:1 or M:N implementations) can provide. One of the major reasons behi…

To implement, no. To make a performant server- yes. Most systems have limited number of threads (low limit constant compiled with the kernel), and each thread is triggered by the scheduler, not by network events, which is very uneconomical. You with application level concurrency you get 100x performance boost for network servers.

On extreme workloads, perhaps. But we have people happily running Rust code with thousands of threads per second in production.

M:N was experimentally found to be slower than 1:1 in Rust.

Re: How Rust optimizes async/await

#95

Earlier quoted context omitted.

Sure. It's called Rc and Arc.

https://en.wikipedia.org/wiki/Cycle_detection

Reference counting is a form of garbage collection, despite the fact that it does not detect cycles without extra work. This has been the correct terminology in the memory management literature for decades.

Re: How Rust optimizes async/await

#96

Earlier quoted context omitted.

So an opt-in GC would be considered zero-cost abstraction?

Is shared_ptr a zero cost abstraction?

Mostly, with the caveat that in single threaded usage, you're paying for the unnecessary overhead of atomic ops on the reference count.

Rust has `Rc`, which is not atomic but `!Send` so the type system ensures that it stays within a thread, as well as the atomic `Arc`.

Re: How Rust optimizes async/await

#97
Related questions: does anybody know:

1) which was the first language that introduced the async/await keywords? I want to say C#, but I’m not sure.

2) are there any papers that describe the code transformation to state machine that is commonly performed to implement these keywords?

Re: How Rust optimizes async/await

#98

Related questions: does anybody know: 1) which was the first language that introduced the async/await keywords? I want to say C#, but I’m not sure. 2) are there any papers that describe the code transformation to state machine that is commonly performed to implement these keywords?

For #1, there's a well-sourced Stack Overflow post https://softwareengineering.stackexchange.com/a/377514.

Re: How Rust optimizes async/await

#99
post #98

Related questions: does anybody know: 1) which was the first language that introduced the async/await keywords? I want to say C#, but I’m not sure. 2) are there any papers that describe the code transformation to state machine that is commonly performed to implement these keywords?

For #1, there's a well-sourced Stack Overflow post https://softwareengineering.stackexchange.com/a/377514 .

Cool, I'll take a look!

I was quickly searching around and found this paper [1]:

"Pause 'n' play: formalizing asynchronous C#".

It looks promising, although it is behind a paywall ;-(. Also, the keyword "formalizing" tells me that maybe this goes a bit deeper than the kind of description I'm looking for...

1: https://dl.acm.org/citation.cfm?id=2367181

Re: How Rust optimizes async/await

#100
post #63
post #59

Earlier quoted context omitted.

Disclaimer: I'm not an expert on the proposal, but have looked at it some, and can offer my impressions here. (Sorry, this got a bit long!) The C++ proposal definitely attacks the problem from a different angle than Rust. One somewhat surface-level difference is that it implements co_yield in terms of co_await, which is the opposite of Rust implementing await in terms of yield. Another difference is that in Rust, all…

Thanks for the information!! On your last paragraph, the thing I'm concerned by is where this extra priority information is stored and propogated, as the term "task" is interesting: isn't every single separate thing being awaited its own task? There isn't (in my mental model) a concept that maps into something like a "pseudo-thread" (but maybe Rust does something like this, requiring a very structured form of concurr…

I don’t fully understand your use case, but in case this helps:

- Future is a trait, not a type, so you can write your own future types.

- Future’s poll method takes a context argument, and async/await-based futures pass that argument unchanged to any sub-futures that they await. The context argument is a pointer to a standard library type (std::task::Context), which itself contains a pointer to a “waker” object that’s meant to be supplied by the executor, with some predefined methods. There’s some room for customization here, but it’s not as flexible as it probably should be – e.g. for now, as far as I can tell, you can’t get the pointer back out of the waker, only call the standard methods on it. Thread-local storage is an option, of course.

Post reply on HN