Local async executors and why they should be the default • Hacker News Board

Re: Local async executors and why they should be the default

#141

Post by zokier » Wed, Jul 19, 2023, 2:13 PM UTC

rich_sasha wrote: post #130

Moaning aside, what is (if any?) the direct equivalent to single-threaded asyncio, like Python or node.js, in Rust? The thing I enjoy about async in Python is that it's very easy to write "thread"-safe code - you know exactly where you might give up execution context, and 90% of the time you have no need for mutexes and locks. But as this article complains, in Rust, it seems to be sync or multi-threaded.

The article mentions smols LocalExecutor and tokios LocalSet, both which are single-threaded

Re: Local async executors and why they should be the default

#142

Post by jerf » Wed, Jul 19, 2023, 2:17 PM UTC

seanhunter wrote: post #115

jerf wrote: post #113

Earlier quoted context omitted.

I really think in the end, in another decade or two, the community consensus is going to be that async as it is conceived of today is simply a mistake, full stop. Think about it. Where did it come from in its current incarnation? Node. Why did Node choose it? Did it have a multiplicity of options and carefully choose the best one based on years of experience with each choice? No. Async was "chosen" because it was the…

Async in its current incarnation did not come from node. Eg Haskell had async/await in 1999. https://softwareengineering.stackexchange.com/questions/3774... Node wasn't released until 10 years later.

And I have code still in production written in the "Perl Object Environment", one of several async frameworks for Perl, written before Node even existed. Also before Node existed, I had written code in the Twisted framework for Python.

Node did not invent async by any means and I know it in the strongest possible sense, having used it before Node existed. But those libraries, in use for over a decade, did not make everyone using them exclaim in joy "Yes! This is what all concurrent programming should be!" A few, sure, but far from everyone. They're kind of a bear, even after years of polish.

Not only did those libraries already know about callback hell, they had already implemented some of the (partial) solutions to it. Watching the Node propaganda was almost surreal to me because I could and more-or-less did predict how it would go, years in advance, as it recreated the experience already had by other languages. Experience that did not leave the impression that this was the paradigm for the future. These libraries typically languished within their ecosystems, kept around only because they were the only practical way to, say, keep your Python but also get concurrency. The fact they were the only way to proceed kept them alive, but they never really flourished despite being right there, readily available.

Node is the reason people are running around today claiming async is the bee's knees. Node is the reason why people come on to /r/golang asking how Go can possibly be used for network programming when it lacks the Magic of Async. Contradicting the Node propaganda is why people are nervous about saying "uhmm, hey, I'm using async in a big program and it's not exactly sunshine and rainbows". Prior to Node that would have just been the common understanding; these libraries were often viewed as powerful within their target language because there was no other way to do what they did, but tricky and complex to work with. It was well understood they easily degraded into a mess and were often difficult to onboard a new developer because of the difficulty of following the control flow.

All I'm really saying here is, the community opinion was correct before Node and we should just go back to that opinion, founded on decades of experience.

Re: Local async executors and why they should be the default

#143

Post by rklaehn » Wed, Jul 19, 2023, 2:20 PM UTC

jacquesm wrote: post #137

pdimitar wrote: post #96

Just a personal take: after not coding with Rust for several months, I find it more and more difficult to return to an async code I was writing. The whole thing just reads... ugly and inconsistent. It needs too much already-accumulated knowledge. As the article correctly points out, you need a bunch of stuff that are seemingly unrelated (and syntactically speaking you would never guess they belong together). And as o…

Any kind of mechanism like that requires a mental model that carries a ton of state because it is no longer immediately visible what the scope of execution is. Continuations and the way Erlang handles this have far less mental overhead and help keeping the model and the mental representation of that model in sync. Differences between the two is where bugs will hide. The web isn't asynchronous it's synchronous in almo…

Kind of agree. The erlang runtime just gives you a process abstraction that is extremely cheap. So within a process you can do things logically sequentially without dragging the system as a whole down.

This approach would not have been the right choice for rust itself, since rust as a systems programming language can not have a runtime.

But there is a fun project called lunatic that implements a lightweight runtime in rust https://docs.rs/lunatic-runtime/latest/lunatic_runtime/ and then allows you to write lightweight processes for this runtime also in rust https://docs.rs/lunatic/latest/lunatic/

It seems heavily inspired by the actor model and erlang.

Re: Local async executors and why they should be the default

#144

Post by zokier » Wed, Jul 19, 2023, 2:21 PM UTC

cmrdporcupine wrote: post #84

sanxiyn wrote: post #42

Earlier quoted context omitted.

Your CRUD web application server almost certainly doesn't need async Rust. Using a blocking HTTP server is not "might be a good idea", it simply is a good idea. I recommend Rouille for this: https://github.com/tomaka/rouille . In case you are worried about performance, check the benchmark. Blocking Rouille is faster than builtin async server in Node.js.

Thanks for this reference. Will bookmark for if I ever need an HTTP service. Wish this also provided websockets. EDIT: it does https://github.com/tomaka/rouille/blob/master/src/websocket/... Looks a bit neglected though. Not much recent development.

There is also Astra https://github.com/ibraheemdev/astra

Re: Local async executors and why they should be the default

#145

Post by insanitybit » Wed, Jul 19, 2023, 2:21 PM UTC

jacquesm wrote: post #137

pdimitar wrote: post #96

Just a personal take: after not coding with Rust for several months, I find it more and more difficult to return to an async code I was writing. The whole thing just reads... ugly and inconsistent. It needs too much already-accumulated knowledge. As the article correctly points out, you need a bunch of stuff that are seemingly unrelated (and syntactically speaking you would never guess they belong together). And as o…

Any kind of mechanism like that requires a mental model that carries a ton of state because it is no longer immediately visible what the scope of execution is. Continuations and the way Erlang handles this have far less mental overhead and help keeping the model and the mental representation of that model in sync. Differences between the two is where bugs will hide. The web isn't asynchronous it's synchronous in almo…

> Continuations and the way Erlang handles this have far less mental overhead and help keeping the model and the mental representation of that model in sync. Differences between the two is where bugs will hide.

You still have a lot to think about in Erlang. For example, you need an entire supervisory system to handle the fact that an actor can die. You need to handle the fact that an actor A might send a message to actor B and not get a response, or it'll get a response way later. You need a native way to link actors, a native way to name actors, a tracing system, etc. Actors are not something you can simply slap onto a system and suddenly you've got Erlang superpowers.

TBH tokio tasks give you a very similar abstraction without the problems - you can join a task. That's a huge ergonomic improvement because you can isolate/join state at will without having to use a channel.

> it's an ugly wart on Rust and a distraction to boot.

It's really not that bad. Like, IMO the major reason to have async is for cancellation. I want to be able to have a network request time out. That is way, way easier to handle in async code than threaded code. The best I can do with a sync system is specify timeouts on an underlying resource like a socket, I can't do anything like "this specific request should have this specific timeout" without async.

Re: Local async executors and why they should be the default

#146

Post by [deleted] » Wed, Jul 19, 2023, 2:22 PM UTC

jacquesm wrote: post #137

pdimitar wrote: post #96

Just a personal take: after not coding with Rust for several months, I find it more and more difficult to return to an async code I was writing. The whole thing just reads... ugly and inconsistent. It needs too much already-accumulated knowledge. As the article correctly points out, you need a bunch of stuff that are seemingly unrelated (and syntactically speaking you would never guess they belong together). And as o…

Any kind of mechanism like that requires a mental model that carries a ton of state because it is no longer immediately visible what the scope of execution is. Continuations and the way Erlang handles this have far less mental overhead and help keeping the model and the mental representation of that model in sync. Differences between the two is where bugs will hide. The web isn't asynchronous it's synchronous in almo…

[deleted]

Re: Local async executors and why they should be the default

#147

Post by insanitybit » Wed, Jul 19, 2023, 2:23 PM UTC

jerf wrote: post #113

pdimitar wrote: post #96

Just a personal take: after not coding with Rust for several months, I find it more and more difficult to return to an async code I was writing. The whole thing just reads... ugly and inconsistent. It needs too much already-accumulated knowledge. As the article correctly points out, you need a bunch of stuff that are seemingly unrelated (and syntactically speaking you would never guess they belong together). And as o…

I really think in the end, in another decade or two, the community consensus is going to be that async as it is conceived of today is simply a mistake, full stop. Think about it. Where did it come from in its current incarnation? Node. Why did Node choose it? Did it have a multiplicity of options and carefully choose the best one based on years of experience with each choice? No. Async was "chosen" because it was the…

Saying that Rust chose async because node did is such an absurdly ignorant, incorrect statement.

Re: Local async executors and why they should be the default

#148

Post by insanitybit » Wed, Jul 19, 2023, 2:25 PM UTC

jacquesm wrote: post #140

insanitybit wrote: post #138

Earlier quoted context omitted.

I guess I just don't run into these issues. Maybe because I write my code to be effectively single threaded already. I almost never actually need a channel, mutex, etc, except in very specific and scoped areas. I wonder why I haven't hit these issues despite writing web services in Rust for years.

Sounds to me like you are dealing with this at the process boundary, more like actor and message based software. This is a mature approach and will serve you well. The typical context for stuff like async/await is code that should be running in a separate process so it can block but doesn't. Then it gets ugly fast.

Yes, I suspect you're right. I do not like building giant monoliths with internal task systems. I prefer using async/await on the inside and microservices with rpcs or, ideally, streams for communication.

I think perhaps people are trying to push too much into a single process?

Re: Local async executors and why they should be the default

#149

Post by insanitybit » Wed, Jul 19, 2023, 2:26 PM UTC

cmrdporcupine wrote: post #133

jerf wrote: post #113

Earlier quoted context omitted.

I really think in the end, in another decade or two, the community consensus is going to be that async as it is conceived of today is simply a mistake, full stop. Think about it. Where did it come from in its current incarnation? Node. Why did Node choose it? Did it have a multiplicity of options and carefully choose the best one based on years of experience with each choice? No. Async was "chosen" because it was the…

We wrote asynchronous driven network code for decades without syntactic sugar for it, and it was fine. That 5% performance gain can be had without it. Async syntax is a mistake. Not just because of the mess it makes across the program tree, but also because it brings with it a specific notion of how to do asynchronous I/O. io_uring for example gives an entirely different model, one with some lovely performance benefi…

I think this is just a common misconception of io_uring based on its earlier implementations. io_uring natively supports a polling model.

Re: Local async executors and why they should be the default

#150

Post by ahupp » Wed, Jul 19, 2023, 2:27 PM UTC

jerf wrote: post #113

pdimitar wrote: post #96

Just a personal take: after not coding with Rust for several months, I find it more and more difficult to return to an async code I was writing. The whole thing just reads... ugly and inconsistent. It needs too much already-accumulated knowledge. As the article correctly points out, you need a bunch of stuff that are seemingly unrelated (and syntactically speaking you would never guess they belong together). And as o…

I really think in the end, in another decade or two, the community consensus is going to be that async as it is conceived of today is simply a mistake, full stop. Think about it. Where did it come from in its current incarnation? Node. Why did Node choose it? Did it have a multiplicity of options and carefully choose the best one based on years of experience with each choice? No. Async was "chosen" because it was the…

I used/worked on a similar async model at FB long before I'd ever touched node. Threads are not really solving the same problem as async.

Imagine you are building the FB newsfeed. You have 10 stories and want to fetch them as quickly as possible. The stories are all different types and each refers to other data: profile info, privacy, comments (which depend its own profile fetch), likes, etc.

So you have this tree of dependent data fetches, how do you minimize the time waiting? You could walk the tree node-by-node, but that balloons your wait time. So you kick each bit of work off to a new thread and wait for it to be done. How do you wait for it? Lots of options of course, but they all end up looking like an ad-hoc implementation of async. Which is fine! But if you have a large codebase where a large fraction of code is dealing with this stuff you pretty soon want the runtime to handle this. And if you are mixing code written by others you really want everyone to use the same mechanism.

So the value of async isn't really in the specific implementation choices, it's that it lets you represent this tree in a consistent way. This is less about the number of requests per second, and more about how much code you have to deal with; plenty of places have low RPS but have accumulated lots and lots of code.

Second, threads have their own costs even ignoring performance. Arguably a big reason PHP/Hack worked so well for FB is that each request was single-threaded and started with a clean state; there's whole universes of bugs that are avoided in this model. And node doesn't have threads because JS doesn't have threads, microcontrollers may not have threads, etc.