Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Maybe Rust isn’t a good tool for massively concurrent, userspace software

bitbashing.io

Post reply on HN
Print view View original

321–330 of 624 posts

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#321

Post by dilippkumar »

riku_iki wrote: post #273
dilippkumar wrote: post #182

Earlier quoted context omitted.

> In the same sense that the lifetime of an object in a GC'd system has a lower bound of, "as long as it's referenced", sure. These are not the same. The problem with GC'd systems is that you don't know when the GC will run and eat up your cpu cycles. It is impossible to determine when the memory will actually be freed in such systems. With ARC, you know exactly when you will release your last reference and that's wh…

> In terms of performance, ARC offers massive benefits but it also has big disadvantage, that it communicates to actual malloc for memory management, which is usually much less performant than GC from various reasons.

> which is usually much less performant than GC from various reasons.

Can you elaborate?

I've seen a couple of malloc implementations, and in all of them, free() is a cheap operation. It usually involves setting a bit somewhere and potentially merging with an adjacent free block if available/appropriate.

malloc() is the expensive call, but I don't see how a GC system can get around the same costs for similar reasons.

What am I missing?

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#322

Post by jstx1 »

pshc wrote: post #313
jstx1 wrote: post #280

Earlier quoted context omitted.

> Yes, sometimes you can get stuck for a day, trying to express something within the ownership rules. This is a big problem. Fast iteration time is very valuable. And who likes doing this to themselves anyway? Isn't it a very frustrating experience? How is this the most loved language?

Debugging rare crashes and heisenbugs is more frustrating, and in non-safe languages, a chronic problem. Whereas after you prove the safety of a design once, it stays with you.

It stays with you until you need to change something and find yourself unable to make incremental changes.

And in many use cases people are throwing Rust (and especially async Rust) on problems solved just fine with GC languages so the safety argument doesn’t apply there.

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#323

Post by dgb23 »

brigadier132 wrote: post #278
dgb23 wrote: post #210

Earlier quoted context omitted.

To me this kind of sounds like circular reasoning. Without function coloring there's no distinction that you need to know of. Can you elaborate?

> Without function coloring there's no distinction that you need to know of Why do you think you don't need to know of it? I want to know if the function I'm calling is going to make a network request. Just because I can have a programming language that hides that distinction from me doesn't mean I want that. Ideally I want to have the fundamental behavior of any function I call encoded in the function signature. So…

> I want to know if the function I'm calling is going to make a network request.

That has nothing to do with function coloring.

> Ideally I want to have the fundamental behavior of any function I call encoded in the function signature.

There is no distinction of async functions if you don't have function coloring that you can encode in type signatures.

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#324

Post by SkiFire13 »

hedora wrote: post #312
SkiFire13 wrote: post #263

Earlier quoted context omitted.

> which is because of a design mistake that's baked deep into the rust ecosystem: Leaking memory is 'safe'. Would you prefer not to have internal mutability, not to have `Rc`, or have them but with infectious unsafe trait bounds, or something else?

I don’t see why it would be an either/or. For instance, Rc does not leak memory, and neither do many cases of interior mutability. If an API leaks memory, then I’d like it to be deemed unsafe. That way, leaking a future would be unsafe, so the borrow checker could infer (transitively) that freeing the future means that any references it had are now dead (as it can already infer when a synchronous function call pops r…

> I don’t see why it would be an either/or. For instance, Rc does not leak memory, and neither do many cases of interior mutability.

`Rc` and internal mutability together do allow creating cycles and thus leaking with only safe code. I suggest you to read https://cglab.ca/~abeinges/blah/everyone-poops/ if you haven't done already, it explains the historical reasons for why `std::mem::forget` was changed to be safe.

> Edit 2: maybe an auto trait could statically disallow Rc cycles somehow?

It could be done with an auto and implied trait that is implemented in the opposite case (the type can be safely leaked), but then all the ecosystem must be changed to use `?Leakable` and it would be a pain.

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#325

Post by bullen »

I don't know if I'm adding to the noise or signal but there is a definite solution to this problem: Put your C data in Arrays of 64 byte Structs.

Java is ok too if you want object oriented atomic joint parallelism, but I only recommend using it on the server where you need a VM anyhow.

C from 1970 and Java from 1990 still got things right.

Also Vulkan/Metal/DX12 does not really help, OpenGL 3 with VAO is enough.

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#326

Post by JoshTriplett »

Animats wrote: post #223

OK, I suppose I should write to this. As I've mentioned before, I'm writing a high performance metaverse client. Here's a demo video.[1] It's about 40,000 lines of Rust so far. If you are doing a non-crappy metaverse, which is rare, you need to wrangle a rather excessive amount of data in near real time. In games, there's heavy optimization during game development to prevent overloading the play engine. In a metavers…

> Rust is race condition free, but not deadlock free. It needs a static deadlock analyzer, one that tracks through the call chain and finds that lock A is locked before lock B on path X, while lock B is locked before path A on path Y.

That sounds like a great idea. Something in the style of lockdep, that (when enabled) analyzes what locks are currently held while any other lock is taken, and reports any potential deadlocks (even if they haven't actually deadlocked).

That would require some annotation to handle cases of complex locking, so that the deadlock detection knows (for instance) that a given class of locks are always obtained in address order so they can't deadlock. But it's doable.

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#327

Post by jose_zap »

lelanthran wrote: post #293
simias wrote: post #92

Earlier quoted context omitted.

It really is, but I still favour "unsexy" manual poll/select code with a lot of if/elseing if it means not having to deal with async. I fully acknowledge that I'm an "old school" system dev who's coming from the C world and not the JS world, so I probably have a certain bias because of that, but I genuinely can't understand how anybody could look at the mess that's Rust's async and think that it was a good design for…

> It really is, but I still favour "unsexy" manual poll/select code with a lot of if/elseing if it means not having to deal with async. > I fully acknowledge that I'm an "old school" system dev who's coming from the C world and not the JS world, so I probably have a certain bias because of that, but I genuinely can't understand how anybody could look at the mess that's Rust's async and think that it was a good design…

That’s how Haskell async works. You mark the call as async, not the function itself.

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#328

Post by Someone »

RetpolineDrama wrote: post #272
Animats wrote: post #223

OK, I suppose I should write to this. As I've mentioned before, I'm writing a high performance metaverse client. Here's a demo video.[1] It's about 40,000 lines of Rust so far. If you are doing a non-crappy metaverse, which is rare, you need to wrangle a rather excessive amount of data in near real time. In games, there's heavy optimization during game development to prevent overloading the play engine. In a metavers…

>The normal development process is that it's hard to get things to compile, and then it Just Works. That's great! I hate using a debugger, especially on concurrent programs. Yes, sometimes you can get stuck for a day, trying to express something within the ownership rules. Beats debugging. This is a far superior workflow when you factor in outcomes. More up front time to get a "correct"/more-reliable output scales in…

> This is a far superior workflow when you factor in outcomes.

I’m a strong-typing enthousiast, too, but still, I’m not fully convinced that’s true.

It seems you can’t iterate fast at all in Rust because the code wouldn’t compile, but can iterate fast in C++, except for the fact that the resulting code may be/often is unstable.

If you need to try out a lot of things before finding the right solution, the ability to iterate fast may be worth those crashes.

Maybe, using C++ for fast iterations, and only using various tools to hunt down issues the borrow checker would catch on the iteration you want to keep beats using Rust.

Or do Rust programmers iterate fast using unsafe where needed and then fix things once they’ve settled on a design?

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#329

Post by brigadier132 »

dgb23 wrote: post #323
brigadier132 wrote: post #278

Earlier quoted context omitted.

> Without function coloring there's no distinction that you need to know of Why do you think you don't need to know of it? I want to know if the function I'm calling is going to make a network request. Just because I can have a programming language that hides that distinction from me doesn't mean I want that. Ideally I want to have the fundamental behavior of any function I call encoded in the function signature. So…

> I want to know if the function I'm calling is going to make a network request. That has nothing to do with function coloring. > Ideally I want to have the fundamental behavior of any function I call encoded in the function signature. There is no distinction of async functions if you don't have function coloring that you can encode in type signatures.

> That has nothing to do with function coloring.

Sure, in the same way that types have nothing to do with enforcing logical correctness of software.

> There is no distinction of async functions if you don't have function coloring that you can encode in type signatures.

What are you trying to say with this statement?

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#330

Post by JoshTriplett »

Sytten wrote: post #120
lionkor wrote: post #19

Async is also spread through so many crates that your program will have to be async in its entirety, or at least depend on the tokio crate for a lot of things. Want a web server? Async + tokio or gtfo. Want an sql connector? You better write your own, unless you want async. Each with a different solution to the various problems async brings -- and dont even get me started on async closures and such shit, thats where…

A lot of that pain could have been avoided if the language had better primitives for async in the std or in the futures crate. Like a trait that executor must implement and a "default" blocking executor to execute async code from sync. Right now even building a library that support multiple async runtimes is a PITA, I have done it a couple times. So you end up supporting either just tokio and maybe async-std.

> A lot of that pain could have been avoided if the language had better primitives for async in the std or in the futures crate. Like a trait that executor must implement and a "default" blocking executor to execute async code from sync.

This is one of the goals of the async working group. Hopefully, when ready, that'll make it possible to swap out async runtimes underneath arbitrary code without issues.

Post reply on HN