Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How to think about async/await in Rust

cliffle.com

Post reply on HN
Print view View original

191–200 of 268 posts

Re: How to think about async/await in Rust

#191

Post by pitaj »

gavinhoward wrote: post #176
pitaj wrote: post #172

Earlier quoted context omitted.

async event loops in Rust are invoked explicitly by the programmer as well.

Are they always invoked explicitly? Or is it sometimes implicit?

The closest to implicit you can get is the `#[tokio::main]` attribute macro [1], which expands to something like this

    fn main() {
        tokio::runtime::Builder::new_multi_thread()
            .enable_all()
            .build()
            .unwrap()
            .block_on(async {
                println!("Hello world");
            })
    }
[1]: https://docs.rs/tokio/latest/tokio/attr.main.html#using-the-...

Re: How to think about async/await in Rust

#192

Post by starcraft2wol »

littlestymaar wrote: post #127
usrbinbash wrote: post #2

Not specific to rust, but I think asynchronous programming in general is a hype. It didn't start because it is so awesome, it started because JS can't do parallel any other way. That's the long and short of it. People wanted to use JS in the backend for some reason. The backend requires concurrency. JS cannot do concurrency. Enter the event loop. Then enter some syntactic sugar for the event loop. And since JS is pop…

> Code written using threads is, at least to me, much more readable and easier to reason about. It's “easier” because it lies to you and makes you assume that everything is sequential, but it's not, and sometime that “everything is sequential” abstraction is leaky, and you can't really see what's going on without diving to the bottom of every functions. I've been accustomed so much to the transparency of async/await,…

> “everything is sequential” abstraction is leaky,

Can you explain what details leak?

The sequential model was developed for programming because that's a natural way to reason about proccesses. `if then else`. `do this, then do that. The "async/await" designers seem to agree, as they attempt to tame async by emulating this behavior.

Note that to do anything other than sequential is extremely complicated to reason about, not because of computers, but because of logic/math. All sorts of concerns like: race conditions, synchronization, dead lock, etc are inherent.

Any approach that does not directly address these issues is the one that's creating a leaky abstraction.

> so you known you need to spawn a new thread if you don't want to wait until the completion.

All functions take "blocking time" to execute. It's a spectrum of how long you want to wait.

Re: How to think about async/await in Rust

#193

Post by cmrdporcupine »

"From my perspective, this is the fundamental promise of async fn: easier, composable, explicit state machines."

Honestly, working in embedded... I don't want my state machine hidden like this. I want it up front and explicit, documented, and observable. I want to be able to query components to find out what state they're in. I want to be able to keep metrics on their transitions. I want logging at the transition points. And I want the whole thing specified in code and comments, and I want to be able to see up front how it's using its resources and I want each component to describe its potential state movements. Even better if I can use declarative/formal analysis tools to check them, too (see e.g. stateright)

The risk with the async call-flow pattern is the creeping emergence of new undocumented state transitions, and potential new error states and edge conditions that come from them.

Re: How to think about async/await in Rust

#194

Post by sa46 »

orangepurple wrote: post #154
3np wrote: post #3

Earlier quoted context omitted.

Your comment seems to be conflating concurrency with parallelism. JS doesn't have any language-level abstractions for parallelism (async or not) but you do have Web Workers[0] and process forking (depending on runtime) to get actual parallel programming. JS async deals with concurrency, not parallelism. Threads are the opposite: They are interfaces for parallel programming and their use is orthogonal to how your appl…

Web Workers are not parallel - they are only concurrent to the main context: think OS threads. Async JS is akin to using very lightweight simulated threads. You will not necessarily utilize more CPU cores by spawning additional Web Workers because they are not inherenent parallel. The actual performance of Web Workers depend on how your browser and OS schedules threads. They are OS threads despite the mountain of mis…

I thought the primary purpose of web workers was that the browser can run the workers in parallel to the main thread. As the spec says:

> [Web workers] allow long tasks to be executed without yielding to keep the page responsive

Re: How to think about async/await in Rust

#195

Post by cgh »

bluepod4 wrote: post #160
usrbinbash wrote: post #2

Not specific to rust, but I think asynchronous programming in general is a hype. It didn't start because it is so awesome, it started because JS can't do parallel any other way. That's the long and short of it. People wanted to use JS in the backend for some reason. The backend requires concurrency. JS cannot do concurrency. Enter the event loop. Then enter some syntactic sugar for the event loop. And since JS is pop…

Maybe I’m wrong but the “asynchronous programming started with JavaScript” take doesn’t seem factual.

Correct, I believe it originated with Microsoft via .Net in the mid-2000s and was picked up by the JS ecosystem much later. The fact that Microsoft had a hand in async/await’s emergence may influence how people feel about it.

Re: How to think about async/await in Rust

#196

Post by starcraft2wol »

hot_gril wrote: post #190
starcraft2wol wrote: post #178

Earlier quoted context omitted.

> add more threads to the pool, start hitting overhead from that I would like to know more detail about this claim. The scenario you are describing is one were 64-128 OS threads are fully blocked waiting for IO. If that's the case, is it likely that you will have additional unused IO resources that could be being utilized? Also, what overhead do you see as the main limit on spawning a lot of threads? Is that the CPU…

> The scenario you are describing is one were 64-128 OS threads are fully blocked waiting for IO. If that's the case, is it likely that you will have additional unused IO resources that could be being utilized? One likely scenario is that you've issued 128 RPCs to some other services and are waiting to hear back. Even if each RPC is, say, on a separate TCP connection, your network stack can handle plenty more. > Also…

Thanks for the reply. I am still having a hard time seeing why "turning up the number of threads" doesn't solve this. Maybe for languages with JIT runtimes where each process occupies a larger piece of memory, that could be a problem. But then I see virtual memory coming in, because as you say, most of those processes are doing nothing.

I think I'm going to do some research and see what benchmarks/measurements I can find.

Re: How to think about async/await in Rust

#197

Post by atombender »

usrbinbash wrote: post #2

Not specific to rust, but I think asynchronous programming in general is a hype. It didn't start because it is so awesome, it started because JS can't do parallel any other way. That's the long and short of it. People wanted to use JS in the backend for some reason. The backend requires concurrency. JS cannot do concurrency. Enter the event loop. Then enter some syntactic sugar for the event loop. And since JS is pop…

The tradition of async programming goes back much further back than JS. Doing async I/O — usually referred to as event-driven programming — has been a popular technique in C and C++ for decades, with epoll(), kqueue, libevent/libev/libuv, Boost Asio, ASE, and so on. A lot of modern C software is built on async I/O, notably projects like Nginx, Memcached, Tor, Chrome, ntp, Redis, etc.

Re: How to think about async/await in Rust

#198

Post by gpderetta »

pkolaczk wrote: post #146
gpderetta wrote: post #137

Earlier quoted context omitted.

surely lifetimes and the borrow checker are a better way to statically check for these sort of issues than relying on await side effects? What if an await is inadvertently introduced later inside your (implicit) critical section?

The borrow checker does catch those issues. But it it does not do whole-program analysis. It analyzes code locally, by looking at signatures of functions being called. And also being forced to read distant code to understand if given snippet is correct would be a maintainability nightmare. I've had enough problems dealing with Java exceptions which are allowed to pop up from anywhere and are not visible in the code.

What I mean is that if preserving invariants across function calls is important enough that an async call can break it, you want the invariant to be enforced statically by the compiler and you do not want to rely on visual inspection of the source to confirm lack of reentrancy.

Once you do that, you do not need a call site annotation that a function can be preempted as the compiler will check it for you.

Rust is uniquely equipped to enforce these guarantees.

Re: How to think about async/await in Rust

#199

Post by cgh »

3np wrote: post #32
usrbinbash wrote: post #2

Not specific to rust, but I think asynchronous programming in general is a hype. It didn't start because it is so awesome, it started because JS can't do parallel any other way. That's the long and short of it. People wanted to use JS in the backend for some reason. The backend requires concurrency. JS cannot do concurrency. Enter the event loop. Then enter some syntactic sugar for the event loop. And since JS is pop…

Your rant reminded me of this classic post, which I believe shares your views but from different reasoning. From the discussion, you may be interested in looking at zig[0]. https://journal.stuffwithstuff.com/2015/02/01/what-color-is-... https://news.ycombinator.com/item?id=36597229 (fresh repost) Discussed previously: https://news.ycombinator.com/item?id=8984648 (8ya) https://news.ycombinator.com/item?id=16732948 (5y…

[Somewhat offtopic] Speaking of Zig, I understand concurrent programming is undergoing a rethink of some sort as async has been temporarily removed? I am only following cutting-edge Zig peripherally so I am probably wrong here. Does anyone know what Zig’s future concurrency story is?

Re: How to think about async/await in Rust

#200

Post by diarrhea »

snorkel wrote: post #162

Concurrency is like a single cook preparing many different recipes at the same time, and parallelism is multiple cooks in the same kitchen. Single-threaded processes can use async/await to context switch and it feels like parallelism but it's not unless you're executing each task to its own OS thread. ... and that didn't click for me until I understood that concurrency (async/await) and parallelism (multiple OS threa…

Is multi threading (always) parallelism? I never questioned this but it just occurred to me that on a single-core CPU with a single execution thread, multithreading in user code is a noop? The code will run but won’t be any faster.
Post reply on HN