Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Maybe Rust isn’t a good tool for massively concurrent, userspace software

bitbashing.io

Post reply on HN
Print view View original

531–540 of 624 posts

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#531

Post by sznio »

throw10920 wrote: post #410
Animats wrote: post #223

OK, I suppose I should write to this. As I've mentioned before, I'm writing a high performance metaverse client. Here's a demo video.[1] It's about 40,000 lines of Rust so far. If you are doing a non-crappy metaverse, which is rare, you need to wrangle a rather excessive amount of data in near real time. In games, there's heavy optimization during game development to prevent overloading the play engine. In a metavers…

> Async contamination I've always wondered why the "color" of a function can't be a property of its call site instead of its definition . That would completely solve this problem - you declare your functions once , colorlessly, and then can invoke them as async anywhere you want.

`std::thread::spawn()` and `.join()` are the ultimate async implementation.

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#532

Post by tel »

mplanchard wrote: post #448
simias wrote: post #92

Earlier quoted context omitted.

It really is, but I still favour "unsexy" manual poll/select code with a lot of if/elseing if it means not having to deal with async. I fully acknowledge that I'm an "old school" system dev who's coming from the C world and not the JS world, so I probably have a certain bias because of that, but I genuinely can't understand how anybody could look at the mess that's Rust's async and think that it was a good design for…

I read comments like this and feel like I’m living in some weird parallel universe. The vast majority of Rust I write day in and day out for my job is in an async context. It has some rough edges, but it’s not particularly painful and is often pleasant enough. Certainly better than promises in JS. I have also used system threads, channels, etc., and indeed there are some places where we communicate between long runni…

Honestly, this is me too. The only thing I’d like to also see is OTP-like supervisors and Trio-like nurseries. They each have their use and they’re totally user land concerns.

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#533

Post by pjmlp »

imtringued wrote: post #512
pjmlp wrote: post #470

Earlier quoted context omitted.

I have found someone that never introduces logic errors, and found out a way to use dependent types in Rust. /s

95% of my "logic errors" are related to surprise nulls (causing a data leak of sensitive data) or surprise mutability. The idea that there is no debug cycle is ridiculous but I am confident that there will be less of them in Rust.

I bet it won't survive a pentest attack, and there are more things missing on program expectations than only nullability.

On the type system theory, Rust still has quite something to catch up to theorem provers, which even those aren't without issues.

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#534

Post by Galanwe »

maccard wrote: post #517
imtringued wrote: post #505

Earlier quoted context omitted.

>use fancy smart pointers, etc. The thing is, you think your code is safe and it most likely is, but mathematically speaking, what you are doing is difficult or even impossible to prove correct. It is akin to running an NP complete algorithm on a problem that is easier than NP. Most practical problem instances are easy to solve, but the worst case which can't be ruled out is utterly, utterly terrible, which forces yo…

Don't let perfect be the enemy of good. Since smart pointers because ubiquitous in c++, I've (personally) had only a handful of memory and lifetime issues. They were all deduceable by looking at where we "escape hatched" and stored a raw ptr that was actually a unique pointer, or something similar. I'll take having one of those every 18 months over throwing away my entire language, toolchain,ecosystem and iteration t…

I don't think it's a matter of putting one versus the other.

If you can get away with smart pointers and such, life is beautiful, nothing wrong there!

The debate here is rather for the cases where you cannot afford such things.

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#535

Post by SkiFire13 »

TwentyPosts wrote: post #364
SkiFire13 wrote: post #135

Earlier quoted context omitted.

Memory pinning in Rust is not a problem that has to do with concurrency because the compiler will never relocate memory when something is referencing it. The problem is however with how stackless coroutines in general (even single-threaded ones, like generators) work. They are inherently self-referential structures, and Rust's memory model likes to pretend such structures don't exist, so you need library workarounds…

>(and the discussion on whether they are actually sound is still open!) Do you have a reference for this? Frankly, maybe I shouldn't ask since I still don't even understand why stackless coroutines are necessarily self-referential, but I am quite curious!

See for example https://github.com/rust-lang/rust/issues/63818 and https://github.com/rust-lang/rfcs/pull/3467

Basically the problem is that async blocks/fns/generators need to create a struct that holds all the local variables within them at any suspension/await/yield point. But local variables can contain references to other local variables, so there are parts of this struct that reference other parts of this struct. This creates two problems:

- once you create such self-references you can no longer move this struct. But moving a struct is safe, so you need some unsafe code that "promises" you this won't happen. `Pin` is a witness of such promise.

- in the memory model having an `&mut` reference to this struct means that it is the only way to access it. But this is no longer true for self referential structs, since there are other ways to access its contents, namely the fields corresponding to those local variables that reference other local variables. This is the problem that's still open.

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#536

Post by pkolaczk »

kprotty wrote: post #528
pkolaczk wrote: post #514

Earlier quoted context omitted.

> Golang supports running asynchronous code in defers, similar with Zig when it still had async. So does Rust. You can run async code inside `drop`.

To run async in Drop in rust, you need to use block_on() as you can't natively await (unlike in Go). This is the "blocking on Drop" mentioned and can result in deadlocks if the async logic is waiting on the runtime to advance, but the block_on() is preventing the runtime thread from advancing. Something like `async fn drop(&mut self)` is one way to avoid this if Rust supported it.

You need to `block_on` only if you need to block on async code. But you don't need to block on order to run async code. You can spawn async code without blocking just fine and there is no risk of deadlocks.

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#537

Post by stolsvik »

kprotty wrote: post #530
throw10920 wrote: post #437

Earlier quoted context omitted.

> could be async IO and require being called in the context of an async runtime The compiler already has knowledge that a function is being called as async - what prevents it from ensuring that a runtime is present when it does? > blocking synchronously on an async task in an async runtime can result in deadlocks from task waiting on runtime IO polling but the waiting preventing the runtime from being polled What pre…

> what prevents it from ensuring that a runtime is present when it does? The runtime being a library instead of a language/compiler level feature. Custom runtimes is necessary for systems languages as they can have specialized constraints. EDIT: Note that it's the presence of a supported runtime for the async operation (e.g. it relies on runtime-specific state like non-blocking IO, timers, priorities, etc.), not only…

> _green thread inefficiencies_

Ron Pressler (@pron) from Loom @ Java had an interesting talk on the Java Language Summit just recently, talking about Loom’s solution to the stack copying: https://youtu.be/6nRS6UiN7X0

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#539

Post by ilyt »

hot_gril wrote: post #139
marcosdumay wrote: post #41

Earlier quoted context omitted.

Yes. That atomized model of concurrency where your state goes everywhere and you somehow collect it back at some point was always (literally) the textbook example of how not to do it. It was created to be an improvement over the Javascript situation, and somehow every language that had a sane structure adopted it as if it was not only good, but the way to do things. This is insane.

And yet, people are going to use async in Rust. The feature has already proven itself useful long ago in other languages, beyond the timespan a fad could survive. Everyone started out doing it the other way and got sick of it.

"Many people use it so it is good" is an idiotic argument

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#540

Post by mratsim »

Animats wrote: post #223

OK, I suppose I should write to this. As I've mentioned before, I'm writing a high performance metaverse client. Here's a demo video.[1] It's about 40,000 lines of Rust so far. If you are doing a non-crappy metaverse, which is rare, you need to wrangle a rather excessive amount of data in near real time. In games, there's heavy optimization during game development to prevent overloading the play engine. In a metavers…

Rust is not race condition free unless it the compiler does formal verification like Ada/Sparks?

It is data-race free however.

Post reply on HN