Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Maybe Rust isn’t a good tool for massively concurrent, userspace software

bitbashing.io

Post reply on HN
Print view View original

281–290 of 624 posts

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#281

Post by colinsane »

half-kh-hacker wrote: post #231
Sytten wrote: post #120

Earlier quoted context omitted.

A lot of that pain could have been avoided if the language had better primitives for async in the std or in the futures crate. Like a trait that executor must implement and a "default" blocking executor to execute async code from sync. Right now even building a library that support multiple async runtimes is a PITA, I have done it a couple times. So you end up supporting either just tokio and maybe async-std.

so it's clear to non-Rust devs, we do have basic primitives for "running async code from sync": https://docs.rs/futures/latest/futures/executor/fn.block_on.... imagine you have an: async fn do_things() -> Something { /* ... */ } you can: use futures::executor::block_on; fn my_normal_code() { let something = block_on(do_things()); } but this does get messy if the async code you're running isn't runtime-agnostic :(

> You can break the chain by commanding the entire runtime to block on the completion of a future, but you probably shouldn’t do this pervasively since it isn’t composable. If a function blocks on a future, and that future calls a function that blocks on a future, congrats! The runtime panics!

article says you can panic if you use the pattern you show. specifically, if you call `my_normal_code()` from an async context.

is the author just talking about a quirk in tokio? or is this sort of wrapping intrinsically dangerous somehow?

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#282

Post by Pauan »

awused wrote: post #271
Pauan wrote: post #160

Earlier quoted context omitted.

Rust embraces abstractions because Rust abstractions are zero-cost. So you can liberally create them and use them without paying a runtime cost. That makes abstractions far more useful and powerful, since you never need to do a cost-benefit analysis in your head, abstractions are just always a good idea in Rust.

"Zero-cost abstractions" can be a confusing term and it is often misunderstood, but it has a precise meaning. Zero-cost abstractions doesn't mean that using them has no runtime cost, just that the abstraction itself causes no additional runtime cost. These can also be quite narrow: Rc is a zero-cost abstraction for refcounting with both strong and weak references allocated with the object on the heap. You cannot impl…

I am very aware of the definition of zero-cost.

We're talking about the comparison between using an abstraction vs not using an abstraction.

When I said "doesn't have a runtime cost", I meant "the abstraction doesn't have a runtime cost compared to not using the abstraction".

If you want your computer to do anything useful, then you have to write code, and that code has a runtime cost.

That runtime cost is unavoidable, it is a simple necessity of the computer doing useful work, regardless of whether you use an abstraction or not.

Whenever you create or use an abstraction, you do a cost-benefit analysis in your head: "does this abstraction provide enough value to justify the EXTRA cost of the abstraction?"

But if there is no extra cost, then the abstraction is free, it is truly zero cost, because the code needed to be written no matter what, and the abstraction is the same speed as not using the abstraction. So there is no cost-benefit analysis, because the abstraction is always worth it.

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#283

Post by hgomersall »

vilunov wrote: post #247
hgomersall wrote: post #236

Earlier quoted context omitted.

It's nonsense. Async in rust is just syntactic sugar around a function signature. You can merrily call async functions from sync rust, you just have to know what to do with the future you get back.

"you just have to know what to do" is the problem. You can call any color from any color, but for some colors it's trivial, e.g. sync function from a sync or async one, or a non-failing function from a failing or non-failing one. I don't want to be able to call fallible function from an infallible one trivially, I want the compiler to force me to specify what exactly I'm going to do with an error if it happens. Likew…

Well yes to all that. I still don't see the problem. An async function isn't really an async function, it's a sync function that returns a future. Would it be better if all that was manual? I've done quite a bit of stuff using manual async traits and it's painful and I highly value the syntax sugar that async brings. That said, I certainly don't want some executor running quietly behind the scenes doing async stuff for me without my explicit and full control. If I want to manually poll a future, that's for me to decide.

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#284

Post by saghm »

vore wrote: post #238
saghm wrote: post #227

Earlier quoted context omitted.

You're not wrong, but I don't really see the problem? Even well before async Rust, closures worked the same way with not being able to specify a concrete type, and `impl Trait` syntax didn't even exist for a while. Annotating local variable types is a way to fix certain things that would otherwise be ambiguous; it's a means to an end, only an end itself.

Ah that's true, but I think it ends up hairier when you combine the two together and have closures that are async, e.g.: let x = || -> i32 { 1 }; // fine let x = || -> impl Future { async { 1 } }; // error: `impl Trait` only allowed in function and inherent method return types, not in closure return types Unless I'm missing something, sometimes you do have to name the return type of an async closure if it's returning…

I think part of the disconnect is that that what you're calling an "async closure" is more directly an analog of a "regular" function that happens to return a future rather than an "async fn" declared function; you'd need similar syntactic boilerplate for annotating a function that's not declared as "async". Currently, there is no closure version of an "async fn" in Rust, but it's arguably not particularly necessary because you can use a Future as the async version of a closure in a lot of cases due to them being lazy already. For example, spawning a task with tokio just takes a Future, not a closure like spawning a sync thread.

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#286

Post by riku_iki »

staunton wrote: post #14
mmastrac wrote: post #3

I love Rust, but async is a hot mess and you cannot just write async code the same way that you write sync code. I'm getting more convinced that mixing the two is a bad idea, and that Go's approach of making everything sync with a single async channel primitive might be right. I'm currently plumbing through some logic to call a sync method on a struct that implements Future and it's... an interesting challenge. While…

Library developers can afford to deal with complexity much more than users of libraries. Offloading such work on highly skilled people developing the basic infrastructure is surely the right approach.

many of these libraries then are debugged and troubleshooted by users..

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#287

Post by Pauan »

tel wrote: post #152
simias wrote: post #92

Earlier quoted context omitted.

It really is, but I still favour "unsexy" manual poll/select code with a lot of if/elseing if it means not having to deal with async. I fully acknowledge that I'm an "old school" system dev who's coming from the C world and not the JS world, so I probably have a certain bias because of that, but I genuinely can't understand how anybody could look at the mess that's Rust's async and think that it was a good design for…

I think Rust’s async stuff is a little half baked now but I have hope that it will be improved as time goes on. In the mean time it is a little annoying to use, but I don’t mind designing against it by default. I feel less architecturally constrained if more syntactically constrained.

I'm curious what things you consider to be half-baked about Rust async.

I've used Rust async extensively for years, and I consider it to be the cleanest and most well designed async system out of any language (and yes, I have used many languages besides Rust).

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#288

Post by vilunov »

hgomersall wrote: post #283
vilunov wrote: post #247

Earlier quoted context omitted.

"you just have to know what to do" is the problem. You can call any color from any color, but for some colors it's trivial, e.g. sync function from a sync or async one, or a non-failing function from a failing or non-failing one. I don't want to be able to call fallible function from an infallible one trivially, I want the compiler to force me to specify what exactly I'm going to do with an error if it happens. Likew…

Well yes to all that. I still don't see the problem. An async function isn't really an async function, it's a sync function that returns a future. Would it be better if all that was manual? I've done quite a bit of stuff using manual async traits and it's painful and I highly value the syntax sugar that async brings. That said, I certainly don't want some executor running quietly behind the scenes doing async stuff f…

You seem to raise valid points and I don't disagree with you, however I don't see how it's relevant to the original concern regarding colored functions.

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#289

Post by awused »

Pauan wrote: post #282
awused wrote: post #271

Earlier quoted context omitted.

"Zero-cost abstractions" can be a confusing term and it is often misunderstood, but it has a precise meaning. Zero-cost abstractions doesn't mean that using them has no runtime cost, just that the abstraction itself causes no additional runtime cost. These can also be quite narrow: Rc is a zero-cost abstraction for refcounting with both strong and weak references allocated with the object on the heap. You cannot impl…

I am very aware of the definition of zero-cost. We're talking about the comparison between using an abstraction vs not using an abstraction. When I said "doesn't have a runtime cost", I meant "the abstraction doesn't have a runtime cost compared to not using the abstraction". If you want your computer to do anything useful, then you have to write code, and that code has a runtime cost. That runtime cost is unavoidabl…

The way you used it in your parent comment didn't make it clear that you were using it properly, hence my clarification. I'm honestly still not sure you've got it right, because Rust abstractions, in general, are not zero-cost. Rust has some zero-cost abstractions in the standard library and Rust has made choices, like monomorphization for generics, that make writing zero-cost abstractions easier and more common in the ecosystem. But there's nothing in the language or compiler that forces all abstractions written in Rust to be free of extra runtime costs.

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#290

Post by artursapek »

dralley wrote: post #36
artursapek wrote: post #15

Earlier quoted context omitted.

this is what Go got right like 10 years ago

In the sense that green threads are easier, sure. But green threads were not and are not the right solution for Rust, so it's kind of beside the point. Async Rust is difficult, but it will eventually be possible to use Async Rust inside the Linux kernel, which is something you can't do with the Go approach.

in the sense that sharing memory by communicating is the right approach
Post reply on HN