Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why asynchronous Rust doesn't work

theta.eu.org

Post reply on HN
Print view View original

351–360 of 499 posts

Re: Why asynchronous Rust doesn't work

#351

Post by steveklabnik »

eru wrote: post #64
newpavlov wrote: post #45

Earlier quoted context omitted.

Yes, you can encode state machines manually, but it will be FAR less ergonomic than the async syntax. Rust has started with a library-based approach, but it was... not great. Async code was littered with and_then methods and it was really close to the infamous JS callback hell. The ergonomic improvements which async/await brings is essentially a raison d'être for incorporating this functionality into the language.

Interesting! For comparison, Haskell went with the library approach but has the syntactic sugar of the equivalent of `and_then` built into the language. (I am talking about Monads and do-notation.) It's a bit like iterating in Python: for-loops are a convenient syntactic sugar to something that can be provided by a library.

It is a little old, but for the general gist of it, http://aturon.github.io/tech/2018/04/24/async-borrowing/ is an amazing description of the problem here.

It is a PhD level research problem to know if monads and do notation would be able to work in Rust. The people who are most qualified to look into it (incidentally: a lot of the same crew was who was working on async) believe that it may literally be impossible.

Re: Why asynchronous Rust doesn't work

#352

Post by steveklabnik »

saurik wrote: post #183
newpavlov wrote: post #87

Earlier quoted context omitted.

It's certainly possible to pave over the difference between models to a certain extent, but the resulting solution will not be zero-cost. Yes, there is a fundamental difference between those models (otherwise we would not have two separate models). In a poll-based model interactions between task and runtime look roughly like this: - task to runtime: I want to read data on this file descriptor. - runtime: FD is ready,…

FWIW, I'd bet almost anything that this problem isn't solvable in any general way without linear types, at which point I bet it would be a somewhat easy modification to what Rust has already implemented. (Most of my development for a long time now has been in C++ using co_await with I/O completion and essentially all of the issues I run into--including the things analogous to "async Drop", which I would argue is actu…

> at which point I bet it would be a somewhat easy modification to what Rust has already implemented.

You'd lose that bet: https://gankra.github.io/blah/linear-rust/

Re: Why asynchronous Rust doesn't work

#353

Post by _xrjp »

jstrong wrote: post #30

I've been primarily coding in rust since 2018. I never cared for async/await, and I've never used it. (at some point, coding event loops became very natural/comfortable for me, and I have no trouble writing "manual" epoll code with mio/mio_httpc). one nice thing about rust's async/await is, you don't have to use it, and if you don't, you don't pay for it in any way. sure, I run into crates that expect me to bring in…

+2

Yes! Using `sans-async` must NOT diminish our productivity in comparison with `async` counterpart.

> at some point, coding event loops became very natural/comfortable for me

In fact it is, I'm very happy with it too.

> one nice thing about rust's async/await is, you don't have to use it, and if you don't, you don't pay for it in any way.

You've said it!

Re: Why asynchronous Rust doesn't work

#354

Post by stjohnswarts »

newpavlov wrote: post #334
withoutboats2 wrote: post #269

Earlier quoted context omitted.

This post is completely and totally wrong. At least you got to ruin my day, I hope that's a consolation prize for you. There is NO meaningful connection between the completion vs polling futures model and the epoll vs io-uring IO models. comex's comments regarding this fact are mostly accurate. The polling model that Rust chose is the only approach that has been able to achieve single allocation state machines in Rus…

Please, calm down. I do appreciate your work on Rust, but people do make mistakes and I strongly belive that in the long term the async stabilization was one of them. It's debatable whether async was essential or not for Rust, I agree it gave Rust a noticeable boost in popularity, but personally I don't think it was worth the long term cost. I do not intend to change your opinion, but I will keep mine and reserve the…

So why did you not present your own solutions to the issues that you criticized or better yet fix it with an RFC rather than declaring a working system as basically a failure (per your title). I think you wouldn't have 10% of the saltiness if you didn't have such an aggressive title to your article.

Re: Why asynchronous Rust doesn't work

#355

Post by steveklabnik »

mratsim wrote: post #246
newpavlov wrote: post #173

Earlier quoted context omitted.

>Why did polling have to be baked into the language? See this comment: https://news.ycombinator.com/item?id=26407440 >Meanwhile C and C++ can easily adopt any async system call style because it made no assumptions in the standards about how that would be done. Do you know about co_await in C++20? AFAIK (I only have a very cursory knowledge about it, so I may be wrong) it also makes some trade-offs, e.g. it requires a…

It requires allocation if the coroutine outlives the scope that created it. Otherwise compiler are free to implement heap allocation elision (which is done in Clang). Now compared to Rust, assuming you have a series of coroutines to process a deferred event, Rust will allocate once for the whole series while C++ would allocate once per coroutine to store them in the reactor/proactor.

Rust never implicitly allocates, even with async/await. I have written Rust programs on a microcontroller with no heap, using async/await for it.

Re: Why asynchronous Rust doesn't work

#356

Post by kibwen »

saurik wrote: post #260
zelly wrote: post #178

Earlier quoted context omitted.

C++20 coroutines are not async in the standard. They are just coroutines. Actually they have no implementation. The user has to write classes to implement the promise type and the awaitable type. You could just as easily write a coroutine library wrapping epoll as you could io_uring. The only thing it does behind your back (other than compile to stackless coroutines) is allocate memory, which also goes for a lot of o…

Is this not also true of Rust? Are you saying Rust in some sense hardcodes an implementation to await in a way C++ doesn't? (I am not a Rust programmer, but I am very very curious about this and would appreciate any insight; I do program in C++ with co_await daily, with my own promise/task classes.)

Rust's async/await support is not intended as a general replacement of coroutines. In fact, async/await is built on top of coroutines (what Rust calls "generators"), but these are not yet stable. https://github.com/rust-lang/rust/issues/43122

Re: Why asynchronous Rust doesn't work

#357

Post by steveklabnik »

saurik wrote: post #260
zelly wrote: post #178

Earlier quoted context omitted.

C++20 coroutines are not async in the standard. They are just coroutines. Actually they have no implementation. The user has to write classes to implement the promise type and the awaitable type. You could just as easily write a coroutine library wrapping epoll as you could io_uring. The only thing it does behind your back (other than compile to stackless coroutines) is allocate memory, which also goes for a lot of o…

Is this not also true of Rust? Are you saying Rust in some sense hardcodes an implementation to await in a way C++ doesn't? (I am not a Rust programmer, but I am very very curious about this and would appreciate any insight; I do program in C++ with co_await daily, with my own promise/task classes.)

You may want to watch/read my talk: https://www.infoq.com/presentations/rust-2019/

I also did a follow up, walking through how you would implement all of the bits: https://www.infoq.com/presentations/rust-async-await/

TL;DR: rust makes you bring some sort of executor along. You can write your own, you can use someone else's. I have not done enough of a deep dive into what made it into the standard to give you a great line-by-line comparison.

Re: Why asynchronous Rust doesn't work

#358

Post by kibwen »

zelly wrote: post #167
newpavlov wrote: post #37

A bigger problem in my opinion is that Rust has chosen to follow the poll-based model (you can say that it was effectively designed around epoll), while the completion-based one (e.g. io-uring and IOCP) with high probability will be the way of doing async in future (especially in the light of Spectre and Meltdown). Instead of carefully weighing advantages and disadvantages of both models, the decision was effectively…

Why did polling have to be baked into the language? Seems bizarre for a supposedly portable language to assume the functionality of an OS feature which could change in the future. Meanwhile C and C++ can easily adopt any async system call style because it made no assumptions in the standards about how that would be done. Rust also didn't solve the colored functions problem. Most people think that's an impossible prob…

> Meanwhile C and C++ can easily adopt any async system call style because it made no assumptions in the standards about how that would be done.

This is comparing apples to oranges; Rust's general, no-assumptions-baked-in coroutines feature is called "generators", and are not yet stable. It is this feature that is internally used to implement async/await. https://github.com/rust-lang/rust/issues/43122

Re: Why asynchronous Rust doesn't work

#359

Post by kristoff_it »

withoutboats2 wrote: post #269
newpavlov wrote: post #37

A bigger problem in my opinion is that Rust has chosen to follow the poll-based model (you can say that it was effectively designed around epoll), while the completion-based one (e.g. io-uring and IOCP) with high probability will be the way of doing async in future (especially in the light of Spectre and Meltdown). Instead of carefully weighing advantages and disadvantages of both models, the decision was effectively…

This post is completely and totally wrong. At least you got to ruin my day, I hope that's a consolation prize for you. There is NO meaningful connection between the completion vs polling futures model and the epoll vs io-uring IO models. comex's comments regarding this fact are mostly accurate. The polling model that Rust chose is the only approach that has been able to achieve single allocation state machines in Rus…

For what it's worth, I agree 100% with the premise of withoutboats' post, based on the experience of having worked a little on Zig's event loop.

My recommendation to people that don't see how ridiculous the original post was, is to write more code and look more into things.

Re: Why asynchronous Rust doesn't work

#360

Post by ragnese »

steveklabnik wrote: post #340
ragnese wrote: post #331

Earlier quoted context omitted.

I agree that Rust should have embraced HKTs instead of resisting them every step of the way. It's been my observation that the Rust lang team is pretty much against HKTs as a general feature, which makes me sad. I'm also happy to meet the only other person who had reservations about NLL! :p I have mixed feelings about it, still. It really is super ergonomic and convenient, but I really value language simplicity and N…

HKTs were considered, and do not actually solve the problem, in Rust. Rust is not Haskell. The differences matter. Nobody is against HKTs on some sort of conceptual level. They just literally do not work to solve this problem in Rust.

Which problem are we referring to? I was only making a general statement that I think Rust would have benefited from HKTs instead of doing umpteen ad-hoc implementations of specific higher-kinded types. I'm far from an expert, so please correct me if I'm wrong:

Wouldn't HKTs help us abstract over function/types more easily, including closures?

Aren't GATs a special case of HKTs?

If Rust somehow had HKTs and "real" monads, we could have a do-notation instead of the Try trait+operator and Future trait+async+await, right?

I'm not saying that HKTs would fix (most of) the issues mentioned in the article around tricky closure semantics and whatnot.

Post reply on HN