Why asynchronous Rust doesn't work • Hacker News Board

Re: Why asynchronous Rust doesn't work

#161

Post by mplanchard » Sat, Nov 13, 2021, 3:09 PM UTC

I write async rust for my day job, and while it is more complicated to write than synchronous rust, the complications generally make sense once you understand how the compiler is managing the remarkably tricky task of async with no GC. Rust remains significantly more pleasurable to write than e.g. typescript for me.

Some patterns around error handling still a bit awkward, but the FuturesExt and TryFuturesExt traits help a lot there.

My only real “I have no clue what’s going on” moment so far has been with an error about traits not being general enough, but someone on the rust forums helped me out: https://users.rust-lang.org/t/trait-is-not-general-enough-fo...

Setting up clippy to disallow non-send-sync futures throughout the codebase has prevented that particular thing from recurring.

Re: Why asynchronous Rust doesn't work

#162

Post by dndx » Sat, Nov 13, 2021, 3:11 PM UTC

999900000999 wrote: post #62

Maybe I'm stupid. But why is Rust so much harder than any other newish programs language. Dart is like all of my dreams come true at once, Rust still gives me nightmares. I seriously tried to learn it multiple times and failed repeatedly. I've created several Dart/ Flutter projects for myself and friends. Multiple C#/Unity projects. Python and JavaScript have paid my rent for the better part of a decade. But Rust, I…

Depends on the kind of programs you write, you may not actually need the "features" Rust provides (e.g. no-GC and high performance). In that case writing in a GC managed language certainly removes a lot of mental burden compared to writing in Rust.

However, anyone came from a systems programming background and wrote any non-trivial async network applications in C/C++ will most certainly appreciate the abstraction and safety Rust provides. Productivity grows significantly when writing in Rust because a lot of the low level details are handled by library authors instead of the programmer.

Re: Why asynchronous Rust doesn't work

#163

Post by pjmlp » Sat, Nov 13, 2021, 3:11 PM UTC

tialaramex wrote: post #107

lordnacho wrote: post #71

Earlier quoted context omitted.

> intermittently doesn’t work in weird and impossible to debug ways This is my major reason for using Rust. It's far better to beat your head against a wall when you're writing than when you're debugging. In both c++ and c# it's possible to write subtly wrong code that is basically undebuggable. Often these are intermittent things that show up once every million or more runs. There's no amount of time that will satis…

One of the very particular "undebuggable" issues (safe) Rust solves is data races. Experience tells us that humans can't successfully reason about non-trivial concurrent programs unless they exhibit Sequential Consistency. In Rust you're promised this is what you get. Maybe what you wrote is stupid and wrong, but it has Sequential Consistency. "Oh," you exclaim during debugging, "A might happened before B and then we…

Only for data races via threads on the same process, it does nothing to prevent data races via shared memory using IPC mechanisms across processes.

Re: Why asynchronous Rust doesn't work

#164

Post by thinkharderdev » Sat, Nov 13, 2021, 3:17 PM UTC

CodesInChaos wrote: post #47

mrfox321 wrote: post #27

Earlier quoted context omitted.

If you are going to wrap everything in Rc, just use kotlin or c-sharp or python?

Rust has advantages over C#, even for high level programming: * Rust traits are more flexible than C# interfaces, especially when combined with generics (implementing traits for foreign types, associated types, each method can have its own constraints, conditional trait implementation, #derive) * Rust has much stronger thread safety guarantees (absence of data races, preventing access to a mutex's data without lockin…

If you don't need the low-level control of Rust then I think Scala would check all of your boxes (aside from distributing dependencies as code instead of binaries). Specifically, an effect library like ZIO looks a lot like Rust without the complexity of managing lifetimes (because you have a GC).

Re: Why asynchronous Rust doesn't work

#165

Post by tptacek » Sat, Nov 13, 2021, 3:18 PM UTC

Aissen wrote: post #15

It was heavily discussed previously. In particular, it triggered this response from Rust contributor withoutboats: https://news.ycombinator.com/item?id=26410487 And this blog post from someone who did spend a lot of time working with async rust: https://tomaka.medium.com/a-look-back-at-asynchronous-rust-d...

Is that provoked by the author of this post, or just a random commenter on the thread that post spawned? It looks like the latter.

Re: Why asynchronous Rust doesn't work

#166

Post by jffry » Sat, Nov 13, 2021, 3:20 PM UTC

gerbilly wrote: post #148

jffry wrote: post #131

Earlier quoted context omitted.

If you mean non-blocking in general, the benefit is that your system can do something else useful while it's waiting for some operation to complete (usually things accessing files or a database or a network service) If you specifically mean async/await syntax, let me illustrate with a contrived example. It can let you express a sequence of asynchronous operations in a more natural way: function promised(cache, db, me…

It's not that I don't see the benefit of the CPU doing something useful when waiting for I/O, my confusion comes from the fact that people like to express this using promises/await. Why not just arrange it like this? function non_awaited(cache, db, metrics) { let result = cache.query(...); if (!result) { result = db.query(...); cache.store(result); } metrics.log(...); return result; } Basically doesn't a good threadi…

I can't speak authoritatively, but I can think of some good reasons you might not want to automatically and implicitly await every invocation of an async function.

As designed, calling an async function just returns a Promise, and any Promise can be awaited. This means that I can pass that Promise around, and it also means I can use a Promise-based library (of which there are many) easily from within my async code.

An example? What if I want to launch multiple asynchronous tasks in parallel, and then either wait until the first one finishes (a race) or wait until they all finish? Without explicit await, we'd need some syntax to express this. With explicit await, I can store the Promise and then await it when desired, like this:

  //start both tasks in parallel
  let fileDataPromise = getFileDataAsync();
  let netDataPromise = getNetDataAsync();
  //wait until both are finished
  let fileData = await fileDataPromise;
  let netData = await netDataPromise;

Fortunately there are nice standard library functions for transforming collections of Promises, so we can also just write:

  let [fileData, netData] = await Promise.all([
    getFileDataAsync(),
    getNetDataAsync()
  ]);

Re: Why asynchronous Rust doesn't work

#167

Post by edude03 » Sat, Nov 13, 2021, 3:21 PM UTC

scotty79 wrote: post #127

quotemstr wrote: post #87

Earlier quoted context omitted.

Naive synchronous reference counting can lead to large pauses as well. What happens when you drop the last reference to the root of a 10,000-node search tree? You do 10,000 reference deferments and free()s. Reference counting might feel more incremental than GC, but really is not. There are tricks you can use, but you're better off with a fast, modern, pauseless real GC that comes with tons of other benefits. Look: m…

I wonder if you could just pass this 10000 tree node root to another thread that will just drop it. This should result in no pause for main thread. Apparently you need Arc for that not Rc which shouldn't have much overhead over Rc in reasonable scenarios.

There is a thread I came across on hacker news that does exactly that https://news.ycombinator.com/item?id=23362518 - this question reminded me of it

Re: Why asynchronous Rust doesn't work

#168

Post by kibwen » Sat, Nov 13, 2021, 3:22 PM UTC

diegocg wrote: post #91

999900000999 wrote: post #62

Maybe I'm stupid. But why is Rust so much harder than any other newish programs language. Dart is like all of my dreams come true at once, Rust still gives me nightmares. I seriously tried to learn it multiple times and failed repeatedly. I've created several Dart/ Flutter projects for myself and friends. Multiple C#/Unity projects. Python and JavaScript have paid my rent for the better part of a decade. But Rust, I…

> But why is Rust so much harder than any other newish programs language. For one, Rust has manual memory management. You are aided by the type system and the compiler, but it's the programmer who has to deal with the mental load of thinking about the lifetime aspects of every variable. Compare to a GCed language, where you just free your mind and can focus on your program.

To be more precise, I consider Rust to have "automatic static memory management", in contrast to C's "manual static memory management", or Java's "automatic dynamic memory management". The static part makes it harder than Java, because you do need to think about how to structure things, but the automatic part makes it easier than C, because the Rust compiler does the nitty-gritty for you.

Re: Why asynchronous Rust doesn't work

#169

Post by hmrr » Sat, Nov 13, 2021, 3:24 PM UTC

GordonS wrote: post #65

hmrr wrote: post #12

Earlier quoted context omitted.

It’s a pain with GC as well, coming from a C# background. It’s incredibly easy to write something that intermittently doesn’t work in weird and impossible to debug ways.

Hmm, C# is my main language, and I don't think I've had an issue like you describe since back when async/await was new and I was still learning about it. And nowadays, Roslyn analyzers, like those in VS and Rider, will warn you about many problems.

Yep same. When I had w3wp crashing 50 times a day with half an async stack somewhere inside the .Net framework several continuations after I did something stupid. Sleep well .Net developers :)

Re: Why asynchronous Rust doesn't work

#170

Post by smabie » Sat, Nov 13, 2021, 3:27 PM UTC

dgan wrote: post #99

999900000999 wrote: post #62

Maybe I'm stupid. But why is Rust so much harder than any other newish programs language. Dart is like all of my dreams come true at once, Rust still gives me nightmares. I seriously tried to learn it multiple times and failed repeatedly. I've created several Dart/ Flutter projects for myself and friends. Multiple C#/Unity projects. Python and JavaScript have paid my rent for the better part of a decade. But Rust, I…

Anecdotally, I found Rust easier to grasp than OCaml..

I would say that Rust is a significantly more complex language than OCaml. As someone who knows both decently, I found Rust much harder to learn.

What problems did you encounter when learning OCaml?