Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Tokio/Rayon Trap and Why Async/Await Fails Concurrency

pmbanugo.me

Post reply on HN
Print view View original

51–60 of 63 posts

Re: The Tokio/Rayon Trap and Why Async/Await Fails Concurrency

#51

Post by silon42 »

jolux wrote: post #14

I am not really sure how much yet another post complaining about async/await that ends with “thread-per-core is the way to go” adds to this discussion. Granted I’m both an Erlang programmer and a big fan of Tokio and Rust’s async/await implementation in general and I think this post and many others like it betray a fundamental misunderstanding of these technologies so I am probably biased.

Yeah, he complains about blocking being a problem then suggests thread-per-core...

Personally I don't want thread-per-core in a general purpuse runtime, I believe C# and Go both do it correctly these days by starting new threads if all existing are busy for too long.

Re: The Tokio/Rayon Trap and Why Async/Await Fails Concurrency

#53

Post by elendilm »

FridgeSeal wrote: post #41
elendilm wrote: post #34

Earlier quoted context omitted.

I am now trembling at the thought of warriors who will skin me alive :). Jokes aside, there are use cases for rayon, use cases for tokio async, use cases for "may" coroutines, use cases for a custom scheduling policy, or use cases for a combination of these. We went with "may" coroutines (with its thread pinning) + custom numa work pinning (to a may thread) due to "may's" lightweight nature and not having to have our…

It sounds like you’ve got a super interesting stack going on there, evidently with a large performance/latency focus. May (haha) I ask what this is in service of? I’m somewhat a fan of the thread-per-core model, so I’m curious as to what you’re doing with it.

Our stack consists of two consumer facing mobile apps, Slyp and SlypBusiness, which necessitated developing the underlying infrastructure over time for scaling and efficiency.

That stack includes Dip (our database substrate), Monolog (our log broker), Singularity (our KV engine), Craft (a mobile app for app development using flow charts, snippets, and a WYSIWYG designer), and Portal (a mobile app for remote container management and a HITL AI workflow).

Craft and Portal are MVP-style functional, but aren't in production yet. Everything else is being rolled out gradually.

You're more than welcome. Happy to discuss any of it in more detail. Rather than digress further in this thread, feel free to reach out privately if you'd like to continue the conversation.

Re: The Tokio/Rayon Trap and Why Async/Await Fails Concurrency

#54

Post by tcfhgj »

oaiey wrote: post #21
jolux wrote: post #14

I am not really sure how much yet another post complaining about async/await that ends with “thread-per-core is the way to go” adds to this discussion. Granted I’m both an Erlang programmer and a big fan of Tokio and Rust’s async/await implementation in general and I think this post and many others like it betray a fundamental misunderstanding of these technologies so I am probably biased.

When I learned about async/await when it came out with .NET, they put tremendous amount into explaining that async/await is not concurrency. But that was in time when you did a training when a new version of your programming stack came out and you did not consume knowledge in 30s snippets.

Why is it not?

Re: The Tokio/Rayon Trap and Why Async/Await Fails Concurrency

#55

Post by diarrhea »

tcfhgj wrote: post #48
veber-alex wrote: post #45

Earlier quoted context omitted.

You can easily limit the number of blocking threads tokio can spawn: https://docs.rs/tokio/latest/tokio/runtime/struct.Builder.ht...

Might not be a good option though if you also spawn a bunch of blocking io tasks, which benefit from a large number of threads.

Exactly. I do not know the specifics, but for example if libraries you call into liberally spawn_blocking under the expectation that it is okay, you will be in trouble.

Says it right there actually:

> It’s recommended to not set this limit too low in order to avoid hanging on operations requiring spawn_blocking.

So a total like 6 - reasonable for a web backend - would be way too low.

Re: The Tokio/Rayon Trap and Why Async/Await Fails Concurrency

#56

Post by nextaccountic »

oersted wrote: post #13
Animats wrote: post #8

There are some fundamental assumptions in the Rust async system: - The program is mostly I/O bound. - All tasks have equal priority. If your program isn't like that, the Tokio model is a bad match to the problem. Real time control is not like that. MMO and metaverse game programs are not like that. Most web stuff is, but that's a special case. A big special case, but a special case.

To be fair, the Rust async model itself was intentionally designed not to be prescriptive in the way you describe. You can build, and there exists, different task executors that can handle things like priority and many other execution models. Async is just a way to describe a tree of concurrent tasks that may depend on (wait on) each other at certain points. It is mostly declarative. Tokio has taken over as the defau…

Unfortunately, rather than exposing a common interface for many executors, the Rust async ecosystem went with a Tokio monoculture

Because of that, many libraries will depend on Tokio rather than working out of box with those different executors. Things like libraries that define protocols, etc.

This creates a strong disincentive for anyone to mess with other executors

Worse yet: with no common interface, some libraries will have feature flags to support a bunch of executors, which increases the maintenance load. Some still keep around support for dead executors like async_std while skipping support for newer executors like glommio

Anyway nowadays there are some abstraction crates in crates.io but it is too little, too late. Tokio already has an insurmountable lock in. The only thing that can start to reverse that is the stdlib itself to offer cross-executor APIs

Re: The Tokio/Rayon Trap and Why Async/Await Fails Concurrency

#57

Post by Animats »

random17 wrote: post #35
VorpalWay wrote: post #17

Earlier quoted context omitted.

Outside of Embassy in embedded, tokio is the only realistic choice though, because it is likely that any third party async crate has a dependency on it already. Yes, smol, monio, glommio etc exist, but they are marginalised (and as far as I can tell they don't really help that much with mixed IO / compute workloads). In fact, async/await in Rust falls apart with a mixed IO / compute workload since scheduling is coope…

How do you properly mix IO/compute (in any language)? In Rust what I’ve done in the past is have two Tokio runtimes, one for IO and one for compute. I know you can also use Rayon but the abstractions are not always flexible/convenient enough. But in either case the boundary between the two types of async work is never easy to cross.

Threads.

I have a graphics program that has about eight different threads doing different things. The draw thread is high priority and does only the things that have to be done on the draw thread. The update thread, medium priority, is doing change events that affect the scene. The movement thread is doing repeated per-frame movement, computed in parallel with drawing. The asset-loading threads, low priority, are making blocking HTTPS requests to get assets from remote servers, and decompressing them, the biggest compute load. Plus there are a few other threads that do various intermittent tasks.

This works well in Rust because the language catches locking errors that cause race conditions. Doing this in C++ would be tough.

You don't really need async until you have thousands of things waiting.

Re: The Tokio/Rayon Trap and Why Async/Await Fails Concurrency

#59

Post by pmbanugo »

haberman wrote: post #16

I see two solid points here: 1. It's not reasonable to expect the application layer to carefully partition its work into "I/O heavy" and "CPU heavy" parts. 2. It's not reasonable to queue up an arbitrary amount of work without back-pressure. I haven't used Tokio much, but if it falls prey to these pitfalls, it would make me pause before adopting it. I think there are probably ways of using Rust async that don't fall…

It's not a problem specific to Rust. It's more around async/await as a concurrency primitive. Same can be said about Node.js or many other languages

Re: The Tokio/Rayon Trap and Why Async/Await Fails Concurrency

#60

Post by projct »

This is super slimy.

What the code actually does is reproduce the exact same problems existing solutions have and claim it solves them.

There is so much wrong with this I don't even know where to start.

Maybe here, with a restated version of the premise:

Async cooperative execution is dangerous because programmers may perform too much work between yields. Therefore, replace compiler-generated resumable tasks with manually written synchronous callbacks that have no yield points, permanently pin them to cores, prohibit dynamic load balancing, and recover from monopolization or memory faults using signals and longjmp.

ugh.

Post reply on HN