Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Maybe Rust isn’t a good tool for massively concurrent, userspace software

bitbashing.io

Post reply on HN
Print view View original

431–440 of 624 posts

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#431

Post by vacuity »

tumdum_ wrote: post #256
raincole wrote: post #49

Earlier quoted context omitted.

Go: it turns out that generic is actually useful Rust: it turns out that not every concurrency needs to be zero-cost abstraction

Except that Rust hasn’t yet realised it

If Rust had gone with green threads as the core async strategy (I know it was a thing pre-1.0), that would be terrible. You're not understanding Rust's design goals. Rust's async model, while it has several major pain points at present, is still undoubtedly superior for what Rust was made for. It would be a shame to throw all that away. Go can go do it's own thing (it has, evidently).

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#432

Post by o11c »

Animats wrote: post #223

OK, I suppose I should write to this. As I've mentioned before, I'm writing a high performance metaverse client. Here's a demo video.[1] It's about 40,000 lines of Rust so far. If you are doing a non-crappy metaverse, which is rare, you need to wrangle a rather excessive amount of data in near real time. In games, there's heavy optimization during game development to prevent overloading the play engine. In a metavers…

> Single ownership with a back reference is a very common need, and it's too hard to do.

I've been collecting a list[1] of what memory-management policies programmers actually want in their code; it is far more extensive than any particular language actually implements. Contributions are welcome!

I already had back reference on the list, but added some details. When the ownership is indirect (really common) it is difficult to automate.

One thing that always irritates me: Rust's decision to make all objects moveable really hurts it at times.

[1] https://gist.github.com/o11c/dee52f11428b3d70914c4ed5652d43f...

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#433

Post by lmm »

throw10920 wrote: post #410
Animats wrote: post #223

OK, I suppose I should write to this. As I've mentioned before, I'm writing a high performance metaverse client. Here's a demo video.[1] It's about 40,000 lines of Rust so far. If you are doing a non-crappy metaverse, which is rare, you need to wrangle a rather excessive amount of data in near real time. In games, there's heavy optimization during game development to prevent overloading the play engine. In a metavers…

> Async contamination I've always wondered why the "color" of a function can't be a property of its call site instead of its definition . That would completely solve this problem - you declare your functions once , colorlessly, and then can invoke them as async anywhere you want.

> I've always wondered why the "color" of a function can't be a property of its call site instead of its definition. That would completely solve this problem - you declare your functions once, colorlessly, and then can invoke them as async anywhere you want.

If you have a non-joke type system (which is to say, Haskell or Scala) you can. I do it all the time. But you need HKT and in Rust each baby step towards that is an RFC buried under a mountain of discussion.

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#434

Post by lmm »

bombela wrote: post #234
0xDEF wrote: post #232

Earlier quoted context omitted.

"I've learned to live without objects, but the trait system is somewhat convoluted. There's one area of asset processing that really wants to be object oriented, and I have more duplicate code there than I like. I could probably rewrite it to use traits more, but it would take some bashing to make it fit the trait paradigm." Can you expand on this? I come from the C# world and the Rust trait system feels expressive e…

I understand this not as objects are missing, after all, struct with methods and traits are objects aren't they? But more like the lack of hierarchical inheritance, that is most often used in OOP to conveniently share common code with added specialization. Override only the methods you want. You can do it with Traits of course, but it's much more verbose. You can technically use the defer trait to simulate a sort of…

Does delegate support delegating everything (except what you're specifically implementing for your own struct) yet? That's the way to do it.

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#435

Post by neonsunset »

za3faran wrote: post #395
jiggawatts wrote: post #367

I wonder if Rust should have gone down the same path as Java’s Project Loom and implemented async I/O using the same memory model that is used with operating system threads. I suspect that to take advantage of 1024-thread systems the only sane programming model will be structured concurrency with virtual threads instead of coroutines. It’s the same progression as we saw in the industry going from unstructured imperat…

I believe rust started with green threads early on, before ditching them. To your point, the C# guys seem to be interested in experimenting with green threads: https://twitter.com/davidfowl/status/1532880744732758018

It was looked at and deemed an inferior design. Especially so given that existing async/await paradigm in .NET works really well with existing language features that would make adoption of green-threads-like approach problematic.

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#436

Post by lmm »

throw10920 wrote: post #427
buildartefact wrote: post #414

Earlier quoted context omitted.

>There's no option to "turn the borrow checker off" - which means that when you're in prototyping mode, you pay this huge productivity penalty for no benefit That’s not really true. The standard workaround for this is just to .clone() or Rc > to unblock yourself, then come back later and fix it. It is true that this needs to be done with some care otherwise you can end up infecting your whole codebase with the “worka…

> It is true that this needs to be done with some care otherwise you can end up infecting your whole codebase with the “workaround” It's a "workaround" precisely because the language does not support it. My statement is correct - you cannot turn the borrow-checker off, and you pay a significant productivity penalty for no benefit. "Rc" can't detect cycles. ".clone()" doesn't work for complex data structures.

You can use unsafe if you really want to "turn the borrow-checker off", no?

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#437

Post by throw10920 »

kprotty wrote: post #417
throw10920 wrote: post #410

Earlier quoted context omitted.

> Async contamination I've always wondered why the "color" of a function can't be a property of its call site instead of its definition . That would completely solve this problem - you declare your functions once , colorlessly, and then can invoke them as async anywhere you want.

If a function calls something that does something async, that can't be evaluated synchronously due to 1) no setup; could be async IO and require being called in the context of an async runtime (library feature, not language feature) and 2) blocking synchronously on an async task in an async runtime can result in deadlocks from task waiting on runtime IO polling but the waiting preventing the runtime from being polled…

> could be async IO and require being called in the context of an async runtime

The compiler already has knowledge that a function is being called as async - what prevents it from ensuring that a runtime is present when it does?

> blocking synchronously on an async task in an async runtime can result in deadlocks from task waiting on runtime IO polling but the waiting preventing the runtime from being polled

What prevents the runtime from preempting a task?

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#438

Post by Galanwe »

ghosty141 wrote: post #334
jstx1 wrote: post #280

Earlier quoted context omitted.

> Yes, sometimes you can get stuck for a day, trying to express something within the ownership rules. This is a big problem. Fast iteration time is very valuable. And who likes doing this to themselves anyway? Isn't it a very frustrating experience? How is this the most loved language?

> And who likes doing this to themselves anyway? Isn't it a very frustrating experience? How is this the most loved language? The thing is, these dependencies do exist no matter what language you use if they stem from an underlying concept. In that case rust just makes you explicitly write them which is a good thing since in C++ all these dependencies would be more or less implicit and everytime somebody edits the co…

> The thing is, these dependencies do exist no matter what language you use

Sure, but in a lot of cases, these invariants can be trivially explained, or intuitive enough that it wouldn't even need explanation. While in Rust, you can easily spend a full day just explaining it to the compiler.

I remember spending litteral _days_ tweaking intricate lifetimes and scopes just to promise Rust that some variables won't be used _after_ a thread finishes.

Some things I even never managed to be able to express in Rust, even if trivial in C, so I just rely on having a C core library for the hot path, and use it from Rust.

Overall, performance sensitive lifetime and memory management in Rust (especially in multithreaded contexts) often comes down to:

1) Do it in _sane_ Rust, and copy everything all over the place, use fancy smart pointers, etc.

2) Do it in a performant manner, without useless copies, without over the top memory management, but prepare a week of frustrating development and a PhD in Rust idiosyncrasies.

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#439

Post by throw10920 »

lmm wrote: post #436
throw10920 wrote: post #427

Earlier quoted context omitted.

> It is true that this needs to be done with some care otherwise you can end up infecting your whole codebase with the “workaround” It's a "workaround" precisely because the language does not support it. My statement is correct - you cannot turn the borrow-checker off, and you pay a significant productivity penalty for no benefit. "Rc" can't detect cycles. ".clone()" doesn't work for complex data structures.

You can use unsafe if you really want to "turn the borrow-checker off", no?

No, because that doesn't give you automatic memory management, which is the point. When I'm prototyping, there's zero reason for me to care about lifetimes - I just want to allocate an object and let the runtime handle it. When you mark everything in your codebase unsafe (a laborious and unnecessary process that then has to be laboriously undone), you still have to ask the Rust runtime for dynamic memory manually, and then track the lifetimes in your head.

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#440

Post by Turskarama »

throw10920 wrote: post #419
buildartefact wrote: post #412

Earlier quoted context omitted.

You’re seriously suggesting writing a game engine in Python?

You accidentally responded to the wrong comment. I never mentioned a game engine.

It doesn't really matter, there doesn't exist a problem space where both Rust and Python are reasonable choices.

Case in point, I once wrote a program to take a 360 degree image and rotate it so that the horizon followed the horizontal line along the middle, and it faced north. I wrote it in python first and running it on a 2k image took on the order of 5 minutes. I rewrote it in rust and it took on the order of 200ms.

Could I iterate in Python faster? Yes, but the end result was useless.

Post reply on HN