Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How Rust optimizes async/await

tmandry.gitlab.io

Post reply on HN
Print view View original

121–130 of 130 posts

Re: How Rust optimizes async/await

#121

Post by quietbritishjim »

jnordwick wrote: post #46
pcwalton wrote: post #43

Earlier quoted context omitted.

If you think that Rust doesn't care about addressing the needs of C++ users, I'm not sure what to tell you. It's not like C++ only focuses on features with zero overhead. The zero-overhead principle means that you don't pay for what you don't use. It doesn't mean that compiler developers are forbidden from working on any language or library feature that consumes more than zero cycles at runtime.

So an opt-in GC would be considered zero-cost abstraction?

Sure, and there are a few hooks for one in the C++ standard library e.g. std::declare_reachable [1], although it doesn't include an actual implementation. Only mark and sweep is feasible, not mark and compact, for obvious reasons [2]. Here's a mention of garbage collection Stroustrop's old C++11 FAQ [3]

[1] https://en.cppreference.com/w/cpp/memory/gc/declare_reachabl...

[2] https://herbsutter.com/2011/10/25/garbage-collection-synopsi...

[3] https://web.archive.org/web/20120228143039/http://www2.resea...

Re: How Rust optimizes async/await

#122

Post by littlestymaar »

omeid2 wrote: post #89
newacctjhro wrote: post #84

Earlier quoted context omitted.

Go needs to allocate a growing stack on the heap, needs to move it around, etc. It's not as efficient as Rust's async.

But is it as efficient or more than the linux threads?

I don't have the full answer (and I would love if someone more knowledgeable could jump in this thread) but I'd say it depends since there are a few antagonistic effects :

- goroutines are (unless it changed since last time I used it) cooperatively scheduled. It's cheaper than preemptive scheduling, but it can lead to big inefficiencies on some workload of you're not careful enough (tight loops can hold a (Linux) thread for a long time and prevent any other goroutines from running on this thread).

- goroutines start with a really small (a few kB) stack which needs to be copied to be grown. If you end up with a stack as big as a native stack, you'd have done a lot of copies in the process, that wouldn't have been necessary if the stack was allocated upfront.

Re: How Rust optimizes async/await

#123

Post by zackmorris »

adwn wrote: post #120
zackmorris wrote: post #50

This is one of the most concise tutorials on how generators, coroutines and futures/promises are related (from first principles) that I've seen. I'm hopeful that eventually promises and async/await fade into history as a fad that turned out to be too unwieldy. I think that lightweight processes with no shared memory, connected by streams (the Erlang/Elixer, Go and Actor model) are the way to go. The advantage to usin…

> I think that lightweight processes with no shared memory, connected by streams [...] are the way to go. No, at least not in general. There are a lot of problems in the real world for which "no shared memory" is incompatible with "efficient parallelism" .

That's actually not true - the efficiencies due to copying can be overcome with the runtime or abstractions.

For example, the copy on write (COW) mechanism of unix where memory pages are mapped to the same location until a forked process writes to one, in which case the virtual memory manager makes a mutable copy.

There's also Redux and Closure's immutable state tree that makes copies under a similar mechanism to COW but through code, since they run at a level of abstraction above C++ or Rust.

My feeling is that these techniques run within a few percent of the speed of hand-optimization. But in the real world, I've seen very little human code remain optimized over the long term. Someone invariably comes along who doesn't understand the principles behind the code and inadvertently does a manual copy somewhere or breaks the O() speed of the algorithm by using the wrong abstractions. Meanwhile immutable code using mechanisms like COW avoids these pitfalls because the code is small and obvious.

I feel that the things that Rust is trying to do were solved long ago under FP, so I don't think it's the language for me. That's also why I moved away from C#, Java, C++, etc. Better languages might be Elixer or Clojure/ClojureScript, although they still have ugly syntax from a mainstream perspective compared to say Javascript or Python. I love that Rust exists as a formal spec of a mature imperative programming language. I think it's still useful in kernels and the embedded space. But I'm concerned that it's borrowing ideas like async/await that trade determinism for performance.

Re: How Rust optimizes async/await

#124

Post by zackmorris »

jnwatson wrote: post #51
zackmorris wrote: post #50

This is one of the most concise tutorials on how generators, coroutines and futures/promises are related (from first principles) that I've seen. I'm hopeful that eventually promises and async/await fade into history as a fad that turned out to be too unwieldy. I think that lightweight processes with no shared memory, connected by streams (the Erlang/Elixer, Go and Actor model) are the way to go. The advantage to usin…

Regarding determinism, async is way more deterministic than multiple threads, because you don’t have arbitrary point where execution contexts can change.

That's true in a way, but only for multithreaded code. Multi-process code with full isolation uses different metaphors like joining threads within higher order functions to achieve parallelism in code that looks single-threaded.

For example, lisp-based languages like Clojure can be statically analyzed and then parallelized so that all noninteracting code runs in its own process. This can also be done for code that operates on vectors like MATLAB and TensorFlow.

For me, isolated processes under the Actor model in languages like Elixer/Erlang and Go is much simpler conceptually than async/await, which is only one step above promises/futures, which is only one step above callback hell. I know that the web world uses async/await for now, but someday I think that will be replaced with something that works more like Go.

Re: How Rust optimizes async/await

#125

Post by ryani »

zackmorris wrote: post #123
adwn wrote: post #120

Earlier quoted context omitted.

> I think that lightweight processes with no shared memory, connected by streams [...] are the way to go. No, at least not in general. There are a lot of problems in the real world for which "no shared memory" is incompatible with "efficient parallelism" .

That's actually not true - the efficiencies due to copying can be overcome with the runtime or abstractions. For example, the copy on write (COW) mechanism of unix where memory pages are mapped to the same location until a forked process writes to one, in which case the virtual memory manager makes a mutable copy. There's also Redux and Closure's immutable state tree that makes copies under a similar mechanism to COW…

> I feel that the things that Rust is trying to do were solved long ago under FP

I don't think this is true at all. Statically analyzed Sync/Send are an amazing tool that I don't see in any other language. In fact, your point of view w.r.t. the actor model is extremely well-supported in Rust, due to Sync/Send traits. Immutable objects can be shared between threads using simple pointers, and mutable objects can have ownership moved between threads with just a pointer copy.

Those threads may have 'shared memory' in the strict sense of the term. But compared to other languages, Rust makes working with shared memory 'feel' like working with independent processes, except with extremely efficient message passing. The language statically guards against threads interacting directly as long as you don't use primitives like Mutex.

Re: How Rust optimizes async/await

#126

Post by zackmorris »

truncate wrote: post #78
zackmorris wrote: post #50

This is one of the most concise tutorials on how generators, coroutines and futures/promises are related (from first principles) that I've seen. I'm hopeful that eventually promises and async/await fade into history as a fad that turned out to be too unwieldy. I think that lightweight processes with no shared memory, connected by streams (the Erlang/Elixer, Go and Actor model) are the way to go. The advantage to usin…

>> dynamic stack allocation of coroutines, which can probably be optimized away anyway This seems interesting. Do you have any pointers to places/papers I can look more into this? I'm also curious, since the stacks have to be rather small when you are running several thousands of coroutines (like Go), how often people get into issues of running out of stack because of some big stack allocation somewhere and stuff lik…

I haven't studied it deeply, but a breadcrumb would be that cooperative threads (green threads) are equivalent to coroutines.

Ok it looks like current techniques are stackless runtimes and compiling coroutines to stackless continuations:

https://en.wikipedia.org/wiki/Stackless_Python

https://stackless.readthedocs.io/en/v3.6.4-slp/library/stack...

http://jessenoller.com/blog/2009/02/23/stackless-you-got-you...

https://engagedscholarship.csuohio.edu/cgi/viewcontent.cgi?r...

https://pdfs.semanticscholar.org/b9aa/49e4b7a00e6c9f0d8c18ba...

https://www.osnews.com/story/9822/protothreads-extremely-lig...

This looks like a rare gem, although I just started reading it:

https://cs.indiana.edu/~dfried/dfried/mex.pdf

I grew up with the cooperative multithreading of classic Mac OS and was really shocked when I first saw Javascript back in the 90s and it had no notion of it (because it didn't have generators). That sent us down the callback hell evolutionary dead end, through promises/futures and finally to async/await where we are now. That could have been largely avoided if we had listened to programming language experts!

Re: How Rust optimizes async/await

#127

Post by adwn »

zackmorris wrote: post #123
adwn wrote: post #120

Earlier quoted context omitted.

> I think that lightweight processes with no shared memory, connected by streams [...] are the way to go. No, at least not in general. There are a lot of problems in the real world for which "no shared memory" is incompatible with "efficient parallelism" .

That's actually not true - the efficiencies due to copying can be overcome with the runtime or abstractions. For example, the copy on write (COW) mechanism of unix where memory pages are mapped to the same location until a forked process writes to one, in which case the virtual memory manager makes a mutable copy. There's also Redux and Closure's immutable state tree that makes copies under a similar mechanism to COW…

> For example, the copy on write (COW) mechanism of unix where memory pages are mapped to the same location until a forked process writes to one

That is shared memory – memory which is shared between two or more threads or processes.

> There's also Redux and Closure's immutable state tree that makes copies under a similar mechanism to COW but through code

That's also shared memory.

"Shared memory" does not necessarily mean "shared mutable memory". Rust, for example, statically prevents memory from being simultaneously shared and mutable (except through synchronization primitives like mutexes etc.). A pure actor-based language, in contrast, would prevent even immutable memory from being shared.

Re: How Rust optimizes async/await

#128

Post by truncate »

zackmorris wrote: post #126
truncate wrote: post #78

Earlier quoted context omitted.

>> dynamic stack allocation of coroutines, which can probably be optimized away anyway This seems interesting. Do you have any pointers to places/papers I can look more into this? I'm also curious, since the stacks have to be rather small when you are running several thousands of coroutines (like Go), how often people get into issues of running out of stack because of some big stack allocation somewhere and stuff lik…

I haven't studied it deeply, but a breadcrumb would be that cooperative threads (green threads) are equivalent to coroutines. Ok it looks like current techniques are stackless runtimes and compiling coroutines to stackless continuations: https://en.wikipedia.org/wiki/Stackless_Python https://stackless.readthedocs.io/en/v3.6.4-slp/library/stack... http://jessenoller.com/blog/2009/02/23/stackless-you-got-you... https:/…

Oh I think I misundetstood you (or you me), and I didn't really articulate my question well. I'm well aware of how stackless coroutines are implemented.

My main concern was whenever you do that, you lose the ability to look at backtrace when something goes wrong. So some implementations keep a separate stack for each coroutine (like Go), and switch SP whenever context is switched. That way you don't have bunch of random function calls in your stack trace. The problem though is that these individual stack for each couroutine has to be pretty small in general (since you would spawn hundreds of thousands of these). Go solved this temporarily using fragmented stack, and later ditched that in favor of copying the whole stack over. And I thought, you were talking about optimizations around this whole "stackfull" coroutines thing so that I can have my backtrace.

Re: How Rust optimizes async/await

#129

Post by zwirbl »

shepmaster wrote: post #73
continuational wrote: post #52

Earlier quoted context omitted.

Ah. What kind of asynchronous task executes so fast that a heap allocation is measurable?

I like to think of it from the other direction. I'm super excited to use futures and async/await in embedded hardware, where "the heap" might not even exist. Hardware interrupts can be treated as events that wake the executor. That lets me write some code like this: async fn bracketed_echo() -> ! { loop { let mut buf = 0; Serial.rx(&mut buf).await; Serial.tx(b'>').await; Serial.tx(buf).await; Serial.tx(b' This reads…

That's exactly what I am looking forward to, the state machine generation can make a lot of code targeting embedded platforms, especially the IO part, much more comfortable. Might I ask which controller you are using? With Rust I'm still on a Cortex, but looking to apply it on other architectures in the future
Post reply on HN