Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Maybe Rust isn’t a good tool for massively concurrent, userspace software

bitbashing.io

Post reply on HN
Print view View original

101–110 of 624 posts

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#101

Post by jzelinskie »

ncruces wrote: post #90
radlad wrote: post #33

Earlier quoted context omitted.

Go also uses goroutines and channels to facilitate message passing, or as they describe it, "sharing memory by communicating." I imagine Rust to be a language far more similar to Go, in both use cases and functionality, than JS.

And in the end, almost everything ends up using Mutex, RWMutex, WaitGroup, Once, and some channels that exist only to ever be closed (like Context.Done), and only if you need to select around them. It's great, but message passing it is not.

As a quite senior Go developer, I'd like to +1 this a ton. You're far more likely to have shocking edge cases unaccounted for when using channels. I consider every usage very, very carefully. Just like every other language, I think the ultimate solution is to build higher-level abstractions for concurrency patterns (e.g. errgroup) and, now that Go has generics, it's the right time to start building them.

If you haven't seen this paper, I bet you'll find at least one or two new bugs that you didn't know about: https://songlh.github.io/paper/go-study.pdf

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#102

Post by fnordpiglet »

brundolf wrote: post #72
fnordpiglet wrote: post #42

Earlier quoted context omitted.

I find the criticisms a little strange - async doesn’t imply multithreaded, and you don’t need to annotate everything shared with magic keywords if you’re async within the same thread because there’s no sharing. Only one future at a time is running on the thread and they’re within the same context. When moving between threads I do what you suggest here and use channels to send signals rather than having a lot of shar…

> Sometimes there is a crucial global state something that’s easier to just directly access, but I just write struct that manages all the Arc/RwLock or whatever other exclusive access mechanism I need for the access patterns. From the callers point of view everything is just a simple function call. When writing the struct I need to be thoughtful of sharing semantics but it’s a very small struct and I write it once an…

Yeah it was a bit of a block for me as well, I don’t know where it came from, but I resisted wrapping things. Reality is breaking things up into crates is encouraged anyway, and just abstracting complexity away is Not That Hard, and can usually be pretty small and concise to boot.

I think I’m used to other languages provided a lot of these abstractions or having some framework that manages it all. The frameworks in rust tend to be pretty low level (with a few notable exceptions) so perhaps that’s where it comes from.

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#103

Post by omnicognate »

divyekapoor wrote: post #80

We do not want red and blue functions. Any language that implements async / await as coroutines instead of green threads is making a fundamental CS mistake. https://journal.stuffwithstuff.com/2015/02/01/what-color-is-... Concurrency's correct primitive is Hoare's Communicating Sequential Processes mapped onto green threads. Some languages that have it right are Java (since JDK17 - Java Virtual Threads), Go, Kotlin.

Always with the blooming red and blue functions. You can say exactly the same thing about const.

The fact that a function can perform asynchronous operations matters to me and I want it reflected in the type system. I want to design my system on such a way that the asynchronous parts are kept where they belong, and I want the type system's help in doing that. "May perform asynchronous operations" is a property a calling function inherits from its callee and it is correctly modelled as such. I don't want to call functions that I don't know this about.

Now you can make an argument that you don't want to design your code this way and that's great if you have another way to think about it all that leads to code that can be maintained and reasoned about equally well (or more so). But calling the classes of functions red and blue and pretending the distinction has no more meaning than that is not such an argument. It's empty nonsense.

"We" don't all agree on this.

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#104

Post by the__alchemist »

I'm a bit on my own for HAL-level tooling in embedded Rust, partly because many of the libs and discussion in OSS Rust mediums focus on async. I've tried to grokk and use async a few times, and have come off disliking it each.

Not my cup of tea.

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#105

Post by atomicnumber3 »

grug_htmx_dev wrote: post #68

Yes, async is effectively a much harder version of Rust, and it's regrettable how it's been shoved down the throats of everyone, while only 1% of projects using it really need it. Hover, async is also amazing in these 1% of cases when it's useful. If you have a service that handles massive amounts of network calls at the core (think linkerd, nginx, etc.), or you want to have a massive amount of lightweight tasks in y…

Is there any reason to use async when your platform supports virtual threads?

I ask as someone who uses java and is about to rewrite a bunch of code to be able to chuck the entire async paradigm into the trash can and use a blocking model but on virtual threads where blocking is ok.

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#106

Post by hot_gril »

nemetroid wrote: post #29
miguelmurca wrote: post #5

This is bound to get some criticism (or some tangent-at-best discussion), but it seems like a pretty fair discussion to me. What I'm missing at the end of the article is the author's point: I believe they're advocating for the use of raw threads and manual management of concurrency, and doing away with the async paraphernalia. But, at the same time, earlier in the article they give the example of networking-related t…

I thought the author's point was relatively clear: Rust might not be a good fit for the kind of tasks that need more concurrency than raw threads can give you. Such programs should be written in some other language instead. > Maybe Rust isn’t a good tool for massively concurrent, userspace software. We can save it for the 99% of our projects that don’t have to be.

So 99% of projects need raw threads only, according to the author. I doubt that.

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#107

Post by lowbloodsugar »

I don’t know enough rust to comment on that part of the article but I’ve run GC-based systems at scale and load and the remarks about that do not match my experience at all.

I've coded performant applications on an OS that used channels and it sucked. It just got in the way and was confusing to engineers used to lower level constructs. "Just get out of my way!"

I think rust async is hard.

And that's what it comes down to. 99.9% (maybe more nines) of people do not need that level of control. They need conceptually simple things, like channels, and GC, and that will work for nearly everyone. The ones who need to drop to rust either have the engineers to do that, or their problem is intractable (for them). I pity those who drop to rust prematurely because it's cool.

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#109

Post by grug_htmx_dev »

HippoBaro wrote: post #89
grug_htmx_dev wrote: post #68

Yes, async is effectively a much harder version of Rust, and it's regrettable how it's been shoved down the throats of everyone, while only 1% of projects using it really need it. Hover, async is also amazing in these 1% of cases when it's useful. If you have a service that handles massive amounts of network calls at the core (think linkerd, nginx, etc.), or you want to have a massive amount of lightweight tasks in y…

The argument here is that Rust chose to implement coroutines the wrong way. It went the route of stackless coroutines that need async/await and colored functions. This creates all the friction the article laments over. But it also praises Go for its implementation, which is also based on a coroutine of a different kind. Stackful coroutines, which do not have any of these problems. Rust considered using those (and, at…

Threads are driven by the OS. Something needs to drive couritines, so there's no way around needing some (even rudimentary, like in embedded) executor. But to be a versatile and universal systems language, Rust can't just build-in executor into a language.

I think that stackless coroutines are better than stackfull, in particular for Rust. Everything was done correctly by the Rust team.

Again, this is all fair and good, as long as people understand the tradeoff and make good technical decisions around. If they all jump on async bandwagon blind o the obvious limitations, we get where Rust ecosystem is now.

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#110

Post by jeroenhd »

lionkor wrote: post #26
cmrdporcupine wrote: post #6

Now, I think async is bad syntactic sugar that hides what's really going on under the surface. And I rail against it all the time. Especially the way dropping in async contaminates code bases by building tendrils across call-sites all through the application. ... But the tools that have been built around it are very useful and there's some good stuff there. I have some quibbles with this article: "Rust comes at this…

> It allows for that This is like saying C++ allows for templates, and theres a big community around it. Sure, but its the entire community.

Practically speaking: the only libraries I regularly use that demand me to use async stuff are HTTP clients, and those have optional blocking imports. I still need to sprinkle [tokio::main] in front of the entry point, but from that point on, everything is blocking.

I don't think it's "the entire community" at all. Dealing with futures across library calls is a pain and almost every library that can avoid it, will avoid it.

I try to avoid async code because of its annoying pain points and I rarely see any circumstances where spawning a new thread doesn't work. Sure, there's more overhead, and you need some kind of limiting factor to prevent spawning a billion of them, but async isn't really required in most circumstances.

It's like saying Go allows for generics. Very few people and libraries bother with them. Working with them is kind of a pain. They're there jf you want to use them, but you generally don't.

Post reply on HN