Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How to think about async/await in Rust

cliffle.com

Post reply on HN
Print view View original

131–140 of 268 posts

Re: How to think about async/await in Rust

#131

Post by gpderetta »

samsquire wrote: post #9

Thanks for this article. I feel the goal of a tool should be to make common patterns easy to represent: so "sprinkling async everywhere" only doesn't work because of some common desirable pattern not being easily representable in modern languages and gotchas of the languages. I have a lightweight thread scheduler written in C and I communicate between threads with a lockless ringbuffer. IO threads do IO. I like the i…

Take a look at Cilk (a lightweight C first and C++ later dialect) that pretty much uses the same syntax you are using. Scheduling is done via workstealing.

The Cilk papers in particular are very very good. They discuss the programming model, the compilation strategy, the scheduling algorithms and more.

edit: this one for example http://supertech.csail.mit.edu/papers/cilk5.pdf

Re: How to think about async/await in Rust

#132

Post by pornel »

leoedin wrote: post #4

The article shows a great example of how to implement a state machine with internal delays (do something, wait for a defined time, do something else), which is very useful in a driver or embedded context where you often just have to wait for an external device to be ready. However, it doesn't really address how you'd construct a state machine with an external tick. It's pretty common to have a state machine called at…

Code using a fixed timestep usually explicitly takes advantage of this design, and is intentionally a hand-rolled state machine, so async doesn't improve it IMHO. Async is meant to hide the event loop (the ticks).

It depends how deep you want the ticks integrated with your async code. At minimum you can do:

   async { loop { next_tick_time().await; do_tick(); } }
You could also write your own async executor that just polls all spawned futures on every tick, instead of the event (waker) mechanism used by async.

But both approaches are IMHO pointless. Async is meant to be a sugar on top of events and run code only when the events happen, not run it all the time at a fixed timestep.

Re: How to think about async/await in Rust

#133

Post by gpderetta »

pkolaczk wrote: post #27
eis wrote: post #15

I think Go got it right by inverting the logic around async/await. In Go you have to explicitly state that a function is to run in the background via "go fn(...)". This makes it much clearer that this code will execute concurrently. In the async/await world you can't tell by looking at a function call if it will block until it's done. Forgot an await? No compile error but your program might behave in weird ways. This…

You're talking about particular JS implementation problems, not general async/await problems. > In Go you have to explicitly state that a function is to run in the background via "go fn(...)". In Rust you have to explicitly `spawn` a task to detach it from the current coroutine and make it run in background. Typically this is much more costly than not spawning and executing async function concurrently as part of the…

> Awaiting implicitly would hide a potentially long and important operation.

but as you point out else thread, you can still hide blocking and potentially expensive operations in any function, so not seeing await give no guarantee that the operation won't block (it only guarantees that the operation won't return to the event loop, assuming that the rust event loop is not reentrant).

Hence await doesn't really protect any useful invariant.

Re: How to think about async/await in Rust

#134

Post by pornel »

samsquire wrote: post #9

Thanks for this article. I feel the goal of a tool should be to make common patterns easy to represent: so "sprinkling async everywhere" only doesn't work because of some common desirable pattern not being easily representable in modern languages and gotchas of the languages. I have a lightweight thread scheduler written in C and I communicate between threads with a lockless ringbuffer. IO threads do IO. I like the i…

Rust explicitly has "Send" and "non-Send" futures that can execute either on any thread, or always on the same thread. Details depend on the executor, and you can roll your own if you want.

This works in Rust:

    let thread_s = spawn(join!(state1(yes), send(message), receive(message2));
    let thread_r = spawn(join!(state1(yes), receive(message), send(message2));

Re: How to think about async/await in Rust

#135

Post by louthy »

gpderetta wrote: post #129
louthy wrote: post #78

Earlier quoted context omitted.

async in F# is not a language feature, it’s a library that leverages F# computation expressions (monads). It’s also possible to do async-like behaviour - without the async/await language feature - in C# using LINQ; so you could argue C# has had the capability (like F#) since LINQ was released. But, I believe C# was the first mainstream language to implement the async/await method-splitting coroutines state-machine (a…

Async is a special case of continuations. If you have fist class continuations (and monads do notation in practice gives you that), you hardly need async as a language feature.

That's exactly my point. It's not a language feature, it's a library. Haskell, F#, and any other language that supports monads (or as you say, first class continuations), have the ability to do async/await - in a way that appears first-class - but actually is just regular code.

C#, and other languages that have taken the C# approach [to async/await], don't have first class continuations (well, C# does with LINQ, but that compromises most ways the average OO dev works). They implement async/await with first-class keywords that indicate where to slice a method in two.

In my language-ext [1] project I have added the LINQ operators to `Task` which allows C# tasks to be used in the same way that Async is done in F#.

[1] https://github.com/louthy/language-ext/blob/main/LanguageExt...

Re: How to think about async/await in Rust

#136

Post by pornel »

eis wrote: post #34
pkolaczk wrote: post #27

Earlier quoted context omitted.

You're talking about particular JS implementation problems, not general async/await problems. > In Go you have to explicitly state that a function is to run in the background via "go fn(...)". In Rust you have to explicitly `spawn` a task to detach it from the current coroutine and make it run in background. Typically this is much more costly than not spawning and executing async function concurrently as part of the…

> foo(); Only if you know that foo is an async function. You can't tell by the function call itelf. > warning: unused implementer of `futures::Future` that must be used Interesting, I haven't seen this warning in the Rust codebase I worked a little with. I'll have to check the compiler settings. Anyways wouldn't it make sense to actually throw an error instead of just a warning? > Additionally there are certain thing…

> You can't tell by the function call itelf.

You can't know that in general. Any regular Go function could spawn a goroutine return immediately too. In JS a "blocking" function could call setImmediate(…) and return too. Even in C, a function could spawn a thread and return immediately too.

You never know at the call site whether a function will block or not, in any language.

So I think polled futures actually are closest to knowing this, since the block-or-not decision can be bubbled up to the caller. In Rust the "doesn't block" example would more likely be `runtime.spawn(foo())`, since the executor is not built into the language, so spawning asynchronously is easier when left up to the caller.

Re: How to think about async/await in Rust

#137

Post by gpderetta »

pkolaczk wrote: post #108
eis wrote: post #103

Earlier quoted context omitted.

> Creating a future in Rust does not have any side effects like running the future in background. This is not JS. Creating a future is just creating an object representing future (postponed) computation. There is nothing spawned on the executor. There are no special side effects (unless you code them explicitly). It works exactly as any other function returning a value, hence why should it be syntactically different?…

> Any normal function call can do these things. A normal function cannot switch threads. foo(); // executed on thread 1 doSomeIO().await; bar(); // possibly continued on thread 2 Now if foo() does some native calls that write some data to thread-local storage and bar() relies on that storage - that can make a huge impact on correctness. Rust is a systems programming language, so details like that matter.

surely lifetimes and the borrow checker are a better way to statically check for these sort of issues than relying on await side effects? What if an await is inadvertently introduced later inside your (implicit) critical section?

Re: How to think about async/await in Rust

#138

Post by gpderetta »

louthy wrote: post #135
gpderetta wrote: post #129

Earlier quoted context omitted.

Async is a special case of continuations. If you have fist class continuations (and monads do notation in practice gives you that), you hardly need async as a language feature.

That's exactly my point. It's not a language feature, it's a library. Haskell, F#, and any other language that supports monads (or as you say, first class continuations), have the ability to do async/await - in a way that appears first-class - but actually is just regular code. C#, and other languages that have taken the C# approach [to async/await], don't have first class continuations (well, C# does with LINQ, but…

Do-notation is a language feature though, and that's a superset of async/await.

Incidentally there are papers that are trying to improve on async in C++ by trying to sneak in generalized do-notation.

Re: How to think about async/await in Rust

#139

Post by dale_glass »

GolDDranks wrote: post #75
diarrhea wrote: post #40

Earlier quoted context omitted.

> We left behind that paradigm in Operating Systems decades ago, and with good reason. I'm curious, what reason? I grew up on Python and C#, and only know async/await, never done real threading (C# async is threading and coroutines under the hood, Python is just coroutines, single-threaded). I find that way of writing code very elegant, as one can encode points of blocking/switching explicitly. A bit like encoding lo…

The reason we left that paradigm in _Operating Systems_ is that OS's are supposed to be resilient. A single buggy app could easily freeze/crash the whole Windows 3.1 system, because the system has the naive assumption that all the apps are benevolent, bug-free, and happily co-operate with time-sharing. Try the same in Windows 2000; you can't, because the system is pre-emptive and forcibly ends the time slots of apps…

> However, that same reason doesn't apply within a single app, because a single app by a single author _can_ safely co-operate with itself. So co-operative time sharing can work and make sense within single app.

That's not the case in any non-trivial app. Any real program truly has dozens if not hundreds of authors whose code you're using sight unseen, and which may be difficult to modify.

And your program getting stuck can be just as bad as the OS getting stuck. Suddenly your program doesn't reply to API requests, or doesn't relinquish some expensive resource (like an expensive VM), or some such.

Re: How to think about async/await in Rust

#140

Post by flohofwoe »

justin_ wrote: post #14
usrbinbash wrote: post #2

Not specific to rust, but I think asynchronous programming in general is a hype. It didn't start because it is so awesome, it started because JS can't do parallel any other way. That's the long and short of it. People wanted to use JS in the backend for some reason. The backend requires concurrency. JS cannot do concurrency. Enter the event loop. Then enter some syntactic sugar for the event loop. And since JS is pop…

Asynchronous programming is a great fit for IO-driven programs, because modern IO is inherently asynchronous. This is clearly true for networking, but even for disk IO, generally commands are sent to the disks and results come back later. Another thing that’s asynchronous is user input, and that’s why JS has it. As for threading vs. explicit yielding (e.g. coroutines), I’d say it’s a matter of taste. I generally pref…

> Asynchronous programming is a great fit for IO-driven programs

Yeah, but this could already be solved without "async/await compiler magic" in native code just with OS primitives, for instance with Windows-style event objects, it might look like this in pseudo-code:

    const event1 = read_async(...);
    const event2 = read_async(...);
    const event3 = read_async(...);
    wait_all(event1, event2, event3);
This would run three IO operations "in parallel", and you're waiting for all three to finish until execution continues after the wait_all() call.

Looks just as convenient as async/await style, but doesn't need special language features or a tricky code-transformation pass in the compiler which turns sequential code into a switch-case state machine (and most importantly, it doesn't have the 'function-color problem').

(this really makes me wonder why Rust has gone down the Javascript-style async/await route with function coloring - the only reason why it remotely makes sense is that it also works in WASM).

Post reply on HN