Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why asynchronous Rust doesn't work

theta.eu.org

Post reply on HN
Print view View original

431–440 of 499 posts

Re: Why asynchronous Rust doesn't work

#431

Post by roca »

sai_c wrote: post #318
roca wrote: post #299

Earlier quoted context omitted.

I've been developing mostly in Rust for the last five years. C++ for decades before that. It is always comforting to dismiss something that you haven't made much effort to understand as unnecessarily overcomplex. I think we're all guilty of that at times.

Ah, i see. But the systems crowd (I'm going to lump them all together: microcontroller, kernel, high performance networking, etc.) is supposed to be the main target audience. For me it is hard to believe that so many of them just brush it off as too complex on first sight. On the other hand, maybe those folks decided, after giving it a serious try, it's not worth it. Mind you, that does not mean they are right. But i…

Sure, but we aren't making decisions in a vaccum with no consequences.

If I really like C and I ship a bunch of C code that gets exploited to build a botnet, that decision had consequences. I should have valued safety higher and the fact I didn't "like" safe alternatives was a distraction.

Re: Why asynchronous Rust doesn't work

#432

Post by pumpkinpal »

Would anyone be able to make some brief comparisons between Rust's asynchronous model and the model followed by Asio in C++ (on which C++23 executors/networking is based) and if there are any parallels with the sender/receiver concept (https://www.youtube.com/watch?v=h-ExnuD6jms)?

I've seen a few comments talking about the choice of Rust adopting a poll model as opposed to a completion model. Am I correct in assuming these are the same things as a reactor (kqeue/epoll etc.) vs a proactor (IOCP) and that a proactor can be implemented in terms of a reactor.

Re: Why asynchronous Rust doesn't work

#433

Post by api »

leshow wrote: post #417
api wrote: post #385

Earlier quoted context omitted.

I've jumped on the Rust bandwagon as part of ZeroTier 2.0 (not rewriting its core, but rewriting some service stuff in Rust and considering the core eventually). I've used a bit of async and while it's not as easy as Go (nothing is!) it's pretty damn ingenious for language-native async in a systems programming language. I personally would have just chickened out on language native async in Rust and told people to rol…

I skimmed some of this, but are you asking why you need to clone in the closure? Because "async closures" don't exist at the moment, the closest you can get is a closure that returns a future, this usually has the form: where F: Fn() -> Fut, Fut: Future i.e. you call some closure f that returns a future that you can then await on. when writing that out it will look like: || { // closure async move { // returned futur…

Wait so you're saying "|| async move {}" is equivalent to "|| move { async move {} }"? If so then mystery solved, but that is not obvious at all and should be documented somewhere more clearly.

In that case all I'm doing vs. their example is explicitly writing the function that returns the promise instead of letting it be "inferred?"

Re: Why asynchronous Rust doesn't work

#434

Post by marshray »

lmm wrote: post #23
ironmagma wrote: post #8

> The thing I really want to try and get across here is that *Rust is not a language where first-class functions are ergonomic.* So... don’t use first-class functions so much? It’s a systems language, not a functional language for describing algorithms in CS whitepapers. Or use `move` (the article does mention this). There are easy paths in most programming languages, and harder paths. Rust is no exception. The fact…

> So... don’t use first-class functions so much? It’s a systems language, not a functional language for describing algorithms in CS whitepapers. Then maybe it was a mistake to adopt an async paradigm from functional languages that relies heavily on the idea that first-class functions are cheap and easy? (FWIW I think Rust was right to pick Scala-style async; it's really the only nice way of working with async that I'…

> maybe it was a mistake to adopt an async paradigm from functional languages

I always thought of async as higher language sugar for coroutine patterns that I'd seen in use in assembler from the 1980's.

Re: Why asynchronous Rust doesn't work

#435

Post by marshray »

jnwatson wrote: post #308
filleduchaos wrote: post #221

Earlier quoted context omitted.

"Ergonomic" is such a nebulous word as to be nearly useless honestly. I don't see how it is [unduly] inefficient or uncomfortable for a language at Rust's level to ensure that the programmer actually thinks through the execution of the code they are writing. If one doesn't want to think about such things - and there's absolutely nothing wrong with that! - there are plenty of higher level languages that can do that le…

The whole point of async is entirely ergonomics. Anything you can do in async you can do in continuation-passing style. I believe the point the author is making is that they added a feature for ergonomics' sake that ended up not being ergonomic; async-style programming usually coincides with first-class functions and closures, and those are painful in Rust.

Nothing about Rust async depends on closures or functions being first-class. It's literally just a state machine transformation.

But now that state data that previously lived on a nice last-in-first-out stack has a different lifetime. Since Rust fastidiously encodes lifetimes in the type system, using async can result in having more complicated types to talk about.

To me it's just the price that must be paid to use a language that doesn't constantly spill garbage everywhere, necessitating collection.

Re: Why asynchronous Rust doesn't work

#436

Post by steveklabnik »

pumpkinpal wrote: post #432

Would anyone be able to make some brief comparisons between Rust's asynchronous model and the model followed by Asio in C++ (on which C++23 executors/networking is based) and if there are any parallels with the sender/receiver concept ( https://www.youtube.com/watch?v=h-ExnuD6jms )? I've seen a few comments talking about the choice of Rust adopting a poll model as opposed to a completion model. Am I correct in assumi…

I don't know enough about Asio to make a good comparison, but

> Am I correct in assuming these are the same things as a reactor (kqeue/epoll etc.) vs a proactor (IOCP)

Close enough, yes.

> a proactor can be implemented in terms of a reactor.

Yes.

Re: Why asynchronous Rust doesn't work

#437

Post by folex »

withoutboats2 wrote: post #269
newpavlov wrote: post #37

A bigger problem in my opinion is that Rust has chosen to follow the poll-based model (you can say that it was effectively designed around epoll), while the completion-based one (e.g. io-uring and IOCP) with high probability will be the way of doing async in future (especially in the light of Spectre and Meltdown). Instead of carefully weighing advantages and disadvantages of both models, the decision was effectively…

This post is completely and totally wrong. At least you got to ruin my day, I hope that's a consolation prize for you. There is NO meaningful connection between the completion vs polling futures model and the epoll vs io-uring IO models. comex's comments regarding this fact are mostly accurate. The polling model that Rust chose is the only approach that has been able to achieve single allocation state machines in Rus…

Thank you for your tremendous work!

Re: Why asynchronous Rust doesn't work

#438

Post by cormacrelf »

marcosdumay wrote: post #330
amelius wrote: post #242

Earlier quoted context omitted.

> the resource can wake the scheduler at any time and tell it to poll Isn't that called interrupting? The terminology seems a little off here, but perhaps that is only my perception.

No, it's not. Interrupting is when you call the scheduler at any time, even when it's doing some other work. When it's idle, it can not be interrupted. Interruptions are something one really tries to restrict to the hardware - kernel layers. Because when people write interruption handlers, they almost always write them wrong.

To elaborate on what it is rather than what it is not, when implementing poll based IO with rust async, typically you have code like “select(); waker.wake()” on a worker thread. Select blocks. Waking tells the executor to poll the related future again, from the top of its tree. The waker implementation may indeed cause an executor thread to stop waiting, it depends on the implementation. It could also be the case that the executor is already awake and the future is simply added to a synchronised queue. Etc. You can implement waking however you like, and technically this could involve an interruptible scheduler, if you really wanted. But you would kinda have to write that.

Re: Why asynchronous Rust doesn't work

#439

Post by moonchild »

roca wrote: post #430
moonchild wrote: post #426

Earlier quoted context omitted.

Cyclone was a c derivative (I believe it was even backwards compatible), and ats a blend of ml and c. Ml and c are both certainly proven. Cyclone was, and ats is, a research project; not necessarily intended to achieve widespread use. And again, obj-c was being used by apple in drivers, which is certainly a real-world application. > without GC I don't know what you mean by this. GC is a memory management policy in wh…

Cyclone wasn't backwards-compatible with C. ATS is not just "a blend of ML and C", it has a powerful proof system on top. You can't just say "well, these languages were derived from C in part, THEREFORE they must be easy to adopt at scale", that doesn't follow at all. Yes, Cyclone and ATS were research projects, that's why they were never able to accumulate the real-world experience needed to demonstrate that their i…

> Objective-C isn't memory safe.

No, but it is garbage collected.

> By "GC" here I meant memory reclamation schemes that require [...] object layout changes

Changes with respect to what?

One example of a popular GC is the boehm GC. It provides a drop-in replacement for malloc, usable in c for existing c structures without any ABI changes.

Perhaps you are thinking specifically of compacting GCs, which usually need objects to have a header with a forwarding pointer?

> require runtime support

‘malloc’ and ‘free’ are a memory reclamation scheme that is part of the c runtime. I don't think there's any argument to be made that they are garbage collection. What's the difference between them and some other runtime support?

  -----------------------------------------
Broadly, you are referring to mechanisms which can be used to implement garbage collection, but those are not what's interesting here. What's interesting is a memory management policy which supports garbage collection and is usable for a systems programming language.

  -----------------------------------------
> If you use the term "garbage collection" in a more expansive way, so that you say Rust "is a garbage collected language", then most people are going to misunderstand you.

‘Garbage collection’ is a technical term with a specific, precise meaning. This meaning is generally understood and accepted throughout the literature. It's also the thing that's specifically interesting here: manually managing object lifetimes is error-prone and tends to lead to bugs, and bugs in systems software tend to be far-reaching, so a way to eliminate those bugs categorically is considered valuable.

> You can't just say "well, these languages were derived from C in part, THEREFORE they must be easy to adopt at scale", that doesn't follow at all.

That's fair as such, but I think the situation is a bit more nuanced than that. The semantics of ats and cyclone are largely designed to augment c directly. Ats's proof semantics in particular map very well to the semantics of c programs as written. Which, true, doesn't prove anything, but shows that there is much less to be proved: the existing paradigm can still be used.

> Yes, Cyclone and ATS were research projects, that's why they were never able to accumulate the real-world experience needed to demonstrate that their ideas work at scale.

Are the several multi-100-kloc ats compilers out there not real-world enough? If not then, on the topic of proof languages, ada/spark and isabelle/hol had proven themselves long before rust.

Re: Why asynchronous Rust doesn't work

#440

Post by ragall »

saurik wrote: post #183
newpavlov wrote: post #87

Earlier quoted context omitted.

It's certainly possible to pave over the difference between models to a certain extent, but the resulting solution will not be zero-cost. Yes, there is a fundamental difference between those models (otherwise we would not have two separate models). In a poll-based model interactions between task and runtime look roughly like this: - task to runtime: I want to read data on this file descriptor. - runtime: FD is ready,…

FWIW, I'd bet almost anything that this problem isn't solvable in any general way without linear types, at which point I bet it would be a somewhat easy modification to what Rust has already implemented. (Most of my development for a long time now has been in C++ using co_await with I/O completion and essentially all of the issues I run into--including the things analogous to "async Drop", which I would argue is actu…

ATS has linear types.
Post reply on HN