Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why asynchronous Rust doesn't work

theta.eu.org

Post reply on HN
Print view View original

471–480 of 499 posts

Re: Why asynchronous Rust doesn't work

#471

Post by yxhuvud »

mratsim wrote: post #465
yxhuvud wrote: post #462

Earlier quoted context omitted.

Well the link I posted is older than your link from 2019 so I doubt that. The other direction may be possible. However, enither are definitely not the first having ideas along these lines - structures and logic like this can also be found in Erlang supervisors after all. And as for the quote, it is quite explicitly referring to Dijkstra and structured programming constructs in nonconcurrent settings.

X10 Programming language is from 2004, http://x10-lang.org/x10-community/publications-using-x10.htm... Habanero Java from 2011 https://www.cs.rice.edu/~vs3/PDF/hj-pppj11.pdf

No doubt. In any case, the author notes his main influences in footnote 3, and those are not part of that. It seems he has a more practical than academic background in this.

Re: Why asynchronous Rust doesn't work

#472

Post by mratsim »

yxhuvud wrote: post #471
mratsim wrote: post #465

Earlier quoted context omitted.

X10 Programming language is from 2004, http://x10-lang.org/x10-community/publications-using-x10.htm... Habanero Java from 2011 https://www.cs.rice.edu/~vs3/PDF/hj-pppj11.pdf

No doubt. In any case, the author notes his main influences in footnote 3, and those are not part of that. It seems he has a more practical than academic background in this.

The author of the talk, Vivek Sarkar, is co-author in both the X10 paper from 2004 and the Habanero paper of 2011.

That's why those are not part of his influences, he co-wrote those papers.

Re: Why asynchronous Rust doesn't work

#473

Post by pcwalton »

johnnycerberus wrote: post #220
newpavlov wrote: post #37

A bigger problem in my opinion is that Rust has chosen to follow the poll-based model (you can say that it was effectively designed around epoll), while the completion-based one (e.g. io-uring and IOCP) with high probability will be the way of doing async in future (especially in the light of Spectre and Meltdown). Instead of carefully weighing advantages and disadvantages of both models, the decision was effectively…

In all this time, maestro Andrei Alexandrescu was right when he said Rust feels like it "skipped leg day" when it comes to concurrency and metaprogramming capabilities. Tim Sweeney was complaining about similar things, saying about Rust that is one step forward, one step backward. These problems will be evident at a later time, when it will be already too late. I will continue experimenting with Rust, but Zig seems t…

D and Zig have dynamically typed generics (templates/"comptime thingy"), while Rust has statically typed generics. A lot of people confuse this for Rust having less powerful generics. It's simply a different approach: the dynamic vs. static types distinction, at the type level instead of the value level.

Re: Why asynchronous Rust doesn't work

#474

Post by pcwalton »

jedisct1 wrote: post #251

If Rust had to be rewritten from scratch today, it would probably be done differently, learning from experience since it was invented. Cargo, macros, const generics and `#[cfg]` wouldn’t exist any more, replaced with comptime evaluation. The standard library would be half its current size (considering its redundant functions, deprecated functions, or constructions that Clippy wants to be replaced with other construct…

None of this is true.

Zig generics are dynamically typed; Rust generics are statically typed. Everyone in Rust was absolutely aware of dynamically-typed generics and they were actively rejected.

Explicit allocators were likewise rejected. They were deemed to add too much noise in the 99% case in which you just want to use the system malloc. You might disagree with that decision, but none of Rust's designers consider it a mistake.

I'm also very skeptical that the ability to automatically switch between async and sync would be the right decision for Rust. It may be the right decision for Zig. But in Rust we have to, for example, work properly with OS TLS and any sort of automatic async/sync switching would be a footgun.

Re: Why asynchronous Rust doesn't work

#475

Post by pcwalton »

newpavlov wrote: post #87
vlovich123 wrote: post #54

Earlier quoted context omitted.

Since you clearly have expertise, I'm curious if you might provide some insight into what would roughly be different in an async completion-based model & why that might be at a fundamental odds with the event-based one? Like is it an incompatibility with the runtime or does it change the actual semantics of async/await in a fundamental way to the point where you can't just swap out the runtime & reuse existing async…

It's certainly possible to pave over the difference between models to a certain extent, but the resulting solution will not be zero-cost. Yes, there is a fundamental difference between those models (otherwise we would not have two separate models). In a poll-based model interactions between task and runtime look roughly like this: - task to runtime: I want to read data on this file descriptor. - runtime: FD is ready,…

How does this proposal address the problems with "blocking on drop does not work" here? https://without.boats/blog/io-uring/

The problem here is not with Rust's async design. It's that Rust has affine types and not linear types. This is not something that could have been solved with more work on the design. It is not that there was "a decision" to allow dropping tasks; it's a constraint on the design that the language requires. (Personally, I'm unsure as to whether a practical language with true linear types is possible, but it's worth experimenting with. Rust is not and never will be that language, however.)

Re: Why asynchronous Rust doesn't work

#476

Post by pcwalton »

saurik wrote: post #183
newpavlov wrote: post #87

Earlier quoted context omitted.

It's certainly possible to pave over the difference between models to a certain extent, but the resulting solution will not be zero-cost. Yes, there is a fundamental difference between those models (otherwise we would not have two separate models). In a poll-based model interactions between task and runtime look roughly like this: - task to runtime: I want to read data on this file descriptor. - runtime: FD is ready,…

FWIW, I'd bet almost anything that this problem isn't solvable in any general way without linear types, at which point I bet it would be a somewhat easy modification to what Rust has already implemented. (Most of my development for a long time now has been in C++ using co_await with I/O completion and essentially all of the issues I run into--including the things analogous to "async Drop", which I would argue is actu…

> FWIW, I'd bet almost anything that this problem isn't solvable in any general way without linear types

I think this part of your comment is absolutely right and it's fatal to the argument that Rust made the wrong decision about I/O models for Rust. Maybe in the context of some other language, Rust's decision was not the best one, but not for Rust, because Rust just doesn't have linear types.

Re: Why asynchronous Rust doesn't work

#477

Post by [deleted] »

api wrote: post #433
leshow wrote: post #417

Earlier quoted context omitted.

I skimmed some of this, but are you asking why you need to clone in the closure? Because "async closures" don't exist at the moment, the closest you can get is a closure that returns a future, this usually has the form: where F: Fn() -> Fut, Fut: Future i.e. you call some closure f that returns a future that you can then await on. when writing that out it will look like: || { // closure async move { // returned futur…

Wait so you're saying "|| async move {}" is equivalent to "|| move { async move {} }"? If so then mystery solved, but that is not obvious at all and should be documented somewhere more clearly. In that case all I'm doing vs. their example is explicitly writing the function that returns the promise instead of letting it be "inferred?"

[deleted]

Re: Why asynchronous Rust doesn't work

#478

Post by pcwalton »

newpavlov wrote: post #394
nemothekid wrote: post #368

Earlier quoted context omitted.

> Of course, the described FSM state transition functions can be rightfully called callbacks, which adds a certain amount of confusion. No, I'm not talking about the state transition functions. I'm talking about the runtime - the thing that will call the state transition function. In the current design, abstractly, the runtime polls/checks every if future if it's in a runnable state, and if so executes it. In an comp…

>In an completion based design the future itself tells the runtime that the value is ready I think there is a misunderstanding. In a completion-based model (read io-uring, but I think IOCP behaves similarly, though I am less familiar with it) it's a runtime who "notifies" tasks about completed IO requests. In io-uring you have two queues represented by ring buffers shared with OS. You add submission queue entries (SQ…

Doesn't work because it relies on caller-managed buffers. See withoutboats' post: https://without.boats/blog/io-uring/

Re: Why asynchronous Rust doesn't work

#479

Post by pas »

mathw wrote: post #461
ragnese wrote: post #328

Earlier quoted context omitted.

I'm a giant Rust fanboy and have been since about 2016. So, for context, this was literally before Futures existed in Rust. But, I only work on Rust code sporadically, so I definitely feel the pros and cons of it when I switch back and forth to/from Rust and other languages. The problem, IMO, isn't about allocations or ownership. In fact, I think that a lot of the complaints about async Rust aren't even about async R…

I wonder if most of the pain is actually caused by Rust async being an MVP, so things like async trait functions (which would be very nice) don't exist... yet. I don't know if anybody has shown that they can't ever exist, it's just that they weren't considered necessary to get the initial async features out of the door. Rather like how you can't use impl Trait in trait method signatures either (there's definitely som…

This macro goes a very long way toward solving the problem: https://github.com/dtolnay/async-trait

Re: Why asynchronous Rust doesn't work

#480

Post by pas »

valand wrote: post #130
thombles wrote: post #43

I'm not totally sure what the author is asking for, apart from refcounting and heap allocations that happen behind your back. In my experience async Rust is heavily characterised by tasks (Futures) which own their data. They have to - when you spawn it, you're offloading ownership of its state to an executor that will keep it alive for some period of time that is out of the spawning code's control. That means all dat…

^this IMO, Rust already provides a decent amount of way to simplify and skip things. Reference counting, async, proc_macro, etc. In my experience, programming stuffs at a higher-level-language where things are heavily abstracted, like (cough) NodeJS, is easy and simple up to a certain point where I have to do a certain low-level things fast (e.g. file/byte patching) or do a system call which is not provided by the ru…

This is where Scala (and JVM based languages) would shine in theory, the JVM has a well defined memory model, provides great low-level tools, etc. (But JVM-based software is always very bulky to deploy, both in terms of memory and size, so this shining is rarely seen in practice.)
Post reply on HN