Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Maybe Rust isn’t a good tool for massively concurrent, userspace software

bitbashing.io

Post reply on HN
Print view View original

441–450 of 624 posts

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#441

Post by OvermindDL1 »

lmm wrote: post #433
throw10920 wrote: post #410

Earlier quoted context omitted.

> Async contamination I've always wondered why the "color" of a function can't be a property of its call site instead of its definition . That would completely solve this problem - you declare your functions once , colorlessly, and then can invoke them as async anywhere you want.

> I've always wondered why the "color" of a function can't be a property of its call site instead of its definition. That would completely solve this problem - you declare your functions once, colorlessly, and then can invoke them as async anywhere you want. If you have a non-joke type system (which is to say, Haskell or Scala) you can. I do it all the time. But you need HKT and in Rust each baby step towards that is…

You can do it without HKTs with an effects system, which you can think of as another kind of generics that causes the function to be sliced in different ways depending on how it's called. There is movement in Rust to try to do this, but I wish it was done before async was implemented considering async could be implemented within it...

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#442

Post by Turskarama »

radicalbyte wrote: post #300
bombela wrote: post #234

Earlier quoted context omitted.

I understand this not as objects are missing, after all, struct with methods and traits are objects aren't they? But more like the lack of hierarchical inheritance, that is most often used in OOP to conveniently share common code with added specialization. Override only the methods you want. You can do it with Traits of course, but it's much more verbose. You can technically use the defer trait to simulate a sort of…

I'm from the C# world and am working through learning Rust... in C# we've largely moved away from using inheritance. Not sure that it's a good thing but "best practise" results in serialisation being implemented differently (serialisers which use attributes, or for more advanced teams serialisation wired in at compile time targeted by attributes - advantage here being that the state doesn't have to be public).

It's "prefer composition over inheritance" though, not "never use inheritance".

There is a time and a place for it.

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#443

Post by RickHull »

FridgeSeal wrote: post #318
KRAKRISMOTT wrote: post #149

Earlier quoted context omitted.

We need a way to bridge the gap. Having a runtime may not be suitable for all apps but it can easily allow you to reach 95%+ concurrency performance. The async compile-to-state-machine model is only necessary for the last 5%. Most userland apps rarely need to maximize concurrency efficiency. They need concurrency yes, but performance at the 95th percentile is more than sufficient.

I really don’t buy this argument that only some small “special” fraction of apps “actually” need async, and for the rest of us “plebs” we should be relegated to blocking. Async is just hard . That’s it. It’s fundamentally difficult. In my experience language implementations of async fall into 2-axes: clarity and control. C# is straightforward-enough (having cribbed its async design off functional languages) but I fin…

I learned concurrency and parallelism by confronting blocking behavior: waiting on a networking or filesystem request stops the world, so we need a new execution context to keep things moving.

What I realized, eventually, is that blocking is a beautiful thing. Embrace the thread of execution going to sleep, as another thread may now execute on the (single core at the time) CPU.

Now you have an organization problem, how to distribute threads across different tasks, some sequential, some parallel, some blocking, some nonblocking. Thread-per-request? Thread-per-connection?

And now a management problem. Spawning threads. Killing threads. Thread pools. Multithreaded logging. Exceptions and error handling.

Totally manageable in mild cases, and big wins in throughput, but scaling limits will present themselves.

I confront many of these tradeoffs in a fun little exercise I call "Miner Mover", implemented in Ruby using many different concurrency primitives here: https://github.com/rickhull/miner_mover

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#444

Post by OvermindDL1 »

throw10920 wrote: post #420
jjnoakes wrote: post #418

Earlier quoted context omitted.

Lifetimes are still important in managed languages. You just have to track them in your head, which is fallible. The difference is that if you get it wrong in a managed language, you get leaks or stale objects or other logic bugs. In rust you get compile time errors.

> The difference is that if you get it wrong in a managed language, you get leaks or stale objects or other logic bugs. Can you provide concrete examples of this? I've literally never had a bug due to the nature of a memory-managed language.

CMEs in Java are a constant thorn to many Java programmers as a lifetime violation bug. Hell even NPEs are too for that matter, lol.

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#445

Post by throw10920 »

Turskarama wrote: post #440
throw10920 wrote: post #419

Earlier quoted context omitted.

You accidentally responded to the wrong comment. I never mentioned a game engine.

It doesn't really matter, there doesn't exist a problem space where both Rust and Python are reasonable choices. Case in point, I once wrote a program to take a 360 degree image and rotate it so that the horizon followed the horizontal line along the middle, and it faced north. I wrote it in python first and running it on a 2k image took on the order of 5 minutes. I rewrote it in rust and it took on the order of 200m…

> there doesn't exist a problem space where both Rust and Python are reasonable choices

This thread, and many other threads about Rust, are filled with people arguing the exact opposite - that Rust is a good, productive language for high-level application development. I agree with you, there's relatively little overlap - that's what I'm arguing for!

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#446

Post by jandrewrogers »

queuebert wrote: post #425
throw10920 wrote: post #403

Earlier quoted context omitted.

> It seems you can’t iterate fast at all in Rust because the code wouldn’t compile Yup, this is correct - and the reason is because Rust forces you to care about efficiency concerns (lifetimes) everywhere . There's no option to "turn the borrow checker off" - which means that when you're in prototyping mode, you pay this huge productivity penalty for no benefit. A language that was designed to be good at iteration wo…

The dev cycle is slower, yes, but once it compiles, there is no debug cycle.

So then tests are optional?

Most bugs are elementary logic bugs expressible in every programming language.

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#448

Post by mplanchard »

simias wrote: post #92
tel wrote: post #20

Earlier quoted context omitted.

Waiting asynchronously on multiple channels/signals. Heterogenous select is really nice.

It really is, but I still favour "unsexy" manual poll/select code with a lot of if/elseing if it means not having to deal with async. I fully acknowledge that I'm an "old school" system dev who's coming from the C world and not the JS world, so I probably have a certain bias because of that, but I genuinely can't understand how anybody could look at the mess that's Rust's async and think that it was a good design for…

I read comments like this and feel like I’m living in some weird parallel universe. The vast majority of Rust I write day in and day out for my job is in an async context. It has some rough edges, but it’s not particularly painful and is often pleasant enough. Certainly better than promises in JS. I have also used system threads, channels, etc., and indeed there are some places where we communicate between long running async tasks with channels, which is nice, and some very simple CLI apps and stuff where we just use system threads rather than pulling in tokio and all that.

Anyway, while I have some issues with async around futur composition and closures, I see people with the kind of super strong reaction here and just feel like I must not be seeing something. To me, it solves the job well, is comprehensible and relatively easy to work with, and remains performant at scale without too much fiddling.

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#449

Post by jonstewart »

Animats wrote: post #223

OK, I suppose I should write to this. As I've mentioned before, I'm writing a high performance metaverse client. Here's a demo video.[1] It's about 40,000 lines of Rust so far. If you are doing a non-crappy metaverse, which is rare, you need to wrangle a rather excessive amount of data in near real time. In games, there's heavy optimization during game development to prevent overloading the play engine. In a metavers…

This is very interesting. How do you manage latency of events coming over the network?

Do... you... wind up having to set TCP_NODELAY?

•͡˘㇁•͡˘

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#450

Post by efficax »

Galanwe wrote: post #438
ghosty141 wrote: post #334

Earlier quoted context omitted.

> And who likes doing this to themselves anyway? Isn't it a very frustrating experience? How is this the most loved language? The thing is, these dependencies do exist no matter what language you use if they stem from an underlying concept. In that case rust just makes you explicitly write them which is a good thing since in C++ all these dependencies would be more or less implicit and everytime somebody edits the co…

> The thing is, these dependencies do exist no matter what language you use Sure, but in a lot of cases, these invariants can be trivially explained, or intuitive enough that it wouldn't even need explanation. While in Rust, you can easily spend a full day just explaining it to the compiler. I remember spending litteral _days_ tweaking intricate lifetimes and scopes just to promise Rust that some variables won't be u…

> Some things I even never managed to be able to express in Rust, even if trivial in C, so I just rely on having a C core library for the hot path, and use it from Rust.

i can’t think of anything you can do in c that you can’t do in unsafe rust, and that has the advantage that you can both narrow it down to exactly where you need it and only there, and your can test it in miri to find bugs

Post reply on HN