Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why asynchronous Rust doesn't work

theta.eu.org

Post reply on HN
Print view View original

271–280 of 499 posts

Re: Why asynchronous Rust doesn't work

#271

Post by SkiFire13 »

saurik wrote: post #183
newpavlov wrote: post #87

Earlier quoted context omitted.

It's certainly possible to pave over the difference between models to a certain extent, but the resulting solution will not be zero-cost. Yes, there is a fundamental difference between those models (otherwise we would not have two separate models). In a poll-based model interactions between task and runtime look roughly like this: - task to runtime: I want to read data on this file descriptor. - runtime: FD is ready,…

FWIW, I'd bet almost anything that this problem isn't solvable in any general way without linear types, at which point I bet it would be a somewhat easy modification to what Rust has already implemented. (Most of my development for a long time now has been in C++ using co_await with I/O completion and essentially all of the issues I run into--including the things analogous to "async Drop", which I would argue is actu…

I'm not familiar with linear types but as far as I can see they're pretty much the same as rust's ownership rules. Is there something I'm missing?

Re: Why asynchronous Rust doesn't work

#272

Post by jedisct1 »

topspin wrote: post #112

> projects that depend on like 3 different versions of tokio and futures I ended up here recently. This part is truly awful. I can forgive a lot, and much of what appears in this blog post falls under than umbrella, but the tokio/std futures schism is offensively discouraging. I strongly suspect it is a symptom of some great dysfunction.

I abandoned some projects due to that as well. That, and constant breaking changes in major libraries such as `hyper`.

Maintaining Rust code is expensive. More than completing the first iteration of a project. Language such as Go or C are boring, but don’t have that issue.

Re: Why asynchronous Rust doesn't work

#273

Post by jedisct1 »

ha_jo_331 wrote: post #250

Of course a fiber/green threads based solution would be easier to use, but Rust is supposed to provide zero cost abstractions for concurrency: Maximum performance and efficiency (and safety) coming first, usability second. THat is perfectly fine. Whereas others (Go for example) sacrifice some efficiency and performance to ergonomics.

Is it still zero cost if the price to pay is a steep increase of code complexity?

The current model also still requires a lot of boxing and cloning, so it is far from being “zero cost” as advertised.

Re: Why asynchronous Rust doesn't work

#274

Post by pjmlp »

zelly wrote: post #178
newpavlov wrote: post #173

Earlier quoted context omitted.

>Why did polling have to be baked into the language? See this comment: https://news.ycombinator.com/item?id=26407440 >Meanwhile C and C++ can easily adopt any async system call style because it made no assumptions in the standards about how that would be done. Do you know about co_await in C++20? AFAIK (I only have a very cursory knowledge about it, so I may be wrong) it also makes some trade-offs, e.g. it requires a…

C++20 coroutines are not async in the standard. They are just coroutines. Actually they have no implementation. The user has to write classes to implement the promise type and the awaitable type. You could just as easily write a coroutine library wrapping epoll as you could io_uring. The only thing it does behind your back (other than compile to stackless coroutines) is allocate memory, which also goes for a lot of o…

Which makes them quite powerful, as they allow for other kinds of patterns.

Re: Why asynchronous Rust doesn't work

#275

Post by littlestymaar »

jedisct1 wrote: post #251

If Rust had to be rewritten from scratch today, it would probably be done differently, learning from experience since it was invented. Cargo, macros, const generics and `#[cfg]` wouldn’t exist any more, replaced with comptime evaluation. The standard library would be half its current size (considering its redundant functions, deprecated functions, or constructions that Clippy wants to be replaced with other construct…

I see you're a fan of Zig's design, but I don't really think Rust would be as you describe even if it was made today, for several reasons:

- comptime evaluation aren't something Rust designers are interested in, as it doesn't fit their functional programming mindset[1], (and it's not a new concept to Zig, D has had something similar for long).

- An IDE-first compiler is a good idea (and a must have for a production language today) but I'm not sure how feasible it is to start with something like that. If it's going to slow down early development, then it's definitely not a good move to go there first, but I don't know.

- per function allocation is a pain in the ass from a usability perspective. In fact it recreates the same “function color” problem you'd like to avoid. Just think of your non-allocating functions as blue and allocating ones as red functions, you can't call a red function from a blue one.

- Zig's “colorblind” asynchronous function rely on automagically switching between async and sync functionn at compile-time, and I really don't see that fitting with Rust's thrive for explicitness.

[1]: https://twitter.com/pcwalton/status/1369114008045772804

Re: Why asynchronous Rust doesn't work

#276

Post by jokethrowaway »

zelly wrote: post #171
lmm wrote: post #13

Async isn't really the problem - the same issue pops up with error handling, with resource management, with anything where you want to pass functions around. The real problem is that Rust's ownership semantics and limited abstractions mean it doesn't really have first-class functions: there are three different function types and the language lacks the power to abstract over them, so you can't generally take an expres…

Rust programs can theoretically be fast, but most of the ones I've used are slow. I tried two high profile implementations of the same type of software, one in Rust and one in Java. The Java one was faster and used less memory. Rust programmers tend to do all kinds of little hacks here and there to make the borrow checker happy. It can add up. The borrow checker is perfectly happy when you copy everything. Rust is be…

> Rust programmers tend to do all kinds of little hacks here and there to make the borrow checker happy. It can add up. The borrow checker is perfectly happy when you copy everything.

Do you mind sharing an example? If you're talking about using to_owned or clone without reason, then it's fully on the developer. Some more pitfalls to avodi: https://llogiq.github.io/2017/06/01/perf-pitfalls.html

What you say definitely doesn't match my experience. I would say Java code and Rust code are roughly in the same order of processing speed but the JVM "wastes" some memory. You also have garbage collection complicating performance in some scenarios.

I'm pretty sure you can get in the same processing speed ballpark with careful programming in both languages.

Outperforming C++ is definitely harder but Java should be doable.

Re: Why asynchronous Rust doesn't work

#277

Post by withoutboats2 »

newpavlov wrote: post #259
the_mitsuhiko wrote: post #210

Earlier quoted context omitted.

> As far as I understand the situation, the completion-based API simply was not on the table 3 years ago Completion APIs were always considered. They are just significantly harder for Rust to support.

Can you provide any public sources for that? From that I've seen Rust async story was always developed primarily around epoll.

Alex Crichton started with a completion based Future struct in 2015. It was even (unstable) in std in 1.0.0:

https://doc.rust-lang.org/1.0.0/std/sync/struct.Future.html

Our async IO model was based on the Linux industry standard (then and now) epoll, but that is not at all what drove the switch to a polling based model, and the polling based model presents no issues whatsoever with io-uring. You do not know what you are talking about.

Re: Why asynchronous Rust doesn't work

#278

Post by Dr_Emann »

orf wrote: post #241
lmm wrote: post #140

Earlier quoted context omitted.

It's not about "abstracting away" ownership, it's about being polymorphic over it. I want to keep the distinction between Fn, FnOnce, and FnMut. But I want to be able to write a `compose` function that works on all three, returning the correct type in each case. That's not taking away from my ability to manage ownership, it's letting me abstract over it.

You could do the polymorphic return with an enumeration. But those are distinct types with very different semantics: you can’t just say “it returns a function that you can call once or maybe a function you can call more than once. Shrug”.

I believe they're asking for "this function returns a function of the same type (Fn/FnMut/FnOnce) as its first argument, but with different arguments. A generic way to write something like `fn bind(f: F, arg: T) -> Fn(...)`

Re: Why asynchronous Rust doesn't work

#279

Post by withoutboats2 »

comex wrote: post #110
stormbrew wrote: post #61

Earlier quoted context omitted.

I'm also curious about this. Boats wrote some about rust async and io-uring a while ago that's interesting[1], but also points out a very clear path forward that's not actually outside the framework of rust's Future or async implementation: using interfaces that treat the kernel as the owner of the buffers being read into/out of, and that seems in line with my expectations of what should work for this. But I haven't…

There's a workaround, but it's unidiomatic, requires more traits, and requires inefficient copying of data if you want to adapt from one to the other. However, I wouldn't call this a problem with a polling-based model. At least part of the goal here must be to avoid allocations and reference counting. If you don't care about that, then the design could have been to 'just' pass around atomically-reference-counted buff…

> I am not sure whether this is actually viable.

Having investigated this myself, I would be very surprised to discover that it is.

The only viable solution to make AsyncRead zero cost for io-uring would be to have required futures to be polled to completion before they are dropped. So you can give up on select and most necessary concurrency primitives. You really want to be able to stop running futures you don't need, after all.

If you want the kernel to own the buffer, you should just let the kernel own the buffer. Therefore, AsyncBufRead. This will require the ecosystem to shift where the buffer is owned, of course, and that's a cost of moving to io-uring. Tough, but those are the cards we were dealt.

Re: Why asynchronous Rust doesn't work

#280

Post by pjmlp »

zelly wrote: post #167
newpavlov wrote: post #37

A bigger problem in my opinion is that Rust has chosen to follow the poll-based model (you can say that it was effectively designed around epoll), while the completion-based one (e.g. io-uring and IOCP) with high probability will be the way of doing async in future (especially in the light of Spectre and Meltdown). Instead of carefully weighing advantages and disadvantages of both models, the decision was effectively…

Why did polling have to be baked into the language? Seems bizarre for a supposedly portable language to assume the functionality of an OS feature which could change in the future. Meanwhile C and C++ can easily adopt any async system call style because it made no assumptions in the standards about how that would be done. Rust also didn't solve the colored functions problem. Most people think that's an impossible prob…

> but people also thought garbage collection was impossible in a systems language until Rust solved it?

What?

Several OSes have proven their value written in GC enabled systems programming languages.

They aren't as mainstream as they should due to UNIX cargo cult and anti-GC Luddites.

Rust only proved that affine types can be easier than cyclone and ATS.

Post reply on HN