Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Maybe Rust isn’t a good tool for massively concurrent, userspace software

bitbashing.io

Post reply on HN
Print view View original

501–510 of 624 posts

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#501

Post by awused »

Pauan wrote: post #390
awused wrote: post #289

Earlier quoted context omitted.

The way you used it in your parent comment didn't make it clear that you were using it properly, hence my clarification. I'm honestly still not sure you've got it right, because Rust abstractions, in general, are not zero-cost. Rust has some zero-cost abstractions in the standard library and Rust has made choices, like monomorphization for generics, that make writing zero-cost abstractions easier and more common in t…

I never said that ALL abstractions in Rust are zero-cost, though the vast majority of them are, and you actually have to explicitly go out of your way to use non-zero-cost abstractions.

Are you sure about that?

>Rust embraces abstractions because Rust abstractions are zero-cost. So you can liberally create them and use them without paying a runtime cost.

>you never need to do a cost-benefit analysis in your head, abstractions are just always a good idea in Rust

Again though, and ignoring that, "zero-cost abstraction" can be very narrow and context specific, so you really don't need to go out of your way to find "costly" abstractions in Rust. As an example, if you have any uses of Rc that don't use weak references, then Rc is not zero-cost for those uses. This is rarely something to bother about, but rarely is not never, and it's going to be more common the more abstractions you roll yourself.

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#502

Post by anonymoushn »

awused wrote: post #498
anonymoushn wrote: post #495

Earlier quoted context omitted.

Reading from a socket, as in the linked post, is an example of not performing I/O? I'm not familiar with tokio so I did not know that it maintained buffers in userspace and filled them before the user called read(), but this is unimportant, it could still have read() yield and return the contents of the buffer. I assumed that users would issue reads of like megabytes at a time and usually receive less. Does the examp…

You're confusing IO not happening because it's not needed with IO never happening. Just because a method can perform IO doesn't mean it actually does every time you call it. If I call async_read(N) for the next N bytes, that isn't necessarily going to touch the IO driver. If your task can make progress without polling, it doesn't need to poll. >I'm not familiar with tokio so I did not know that it maintained buffers…

> You're confusing IO not happening because it's not needed with IO never happening. Just because a method can perform IO doesn't mean it actually does every time you call it. If I call async_read(N) for the next N bytes, that isn't necessarily going to touch the IO driver.

Maybe you can read the linked post again? The problem in the example in the post is that data keeps coming from the network. If you were to strace the program, you would see it calling read(2) repeatedly. The runtime chooses to starve all other tasks as long as these reads return more than 0 bytes. This is obviously not the only option available.

I apologize for charitably assuming that you were correct in the rest of my reply and attempting to fill in the necessary circumstances which would have made you correct

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#503

Post by reacharavindh »

Is it possible to use Rust and all the important 3rd party crates without using async?

Out of curiosity, could Rust be limited to a language subset to mimic the simplicity of Golang (with channels and message passing) and trade-off some of the powerful features that seem to be causing pain?

Pardon a naïve question. I’m a systems engineer who occasionally dabbles with simple cli tools in all languages for fun, but don’t have a serious need for them.

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#504

Post by rcxdude »

tgv wrote: post #480
likeabbas wrote: post #456

Earlier quoted context omitted.

People love(d) rust because it’s a pleasant language to write code for while also being insanely performant. Async is taking away the first point and making it miserable to write code for. If this trend continues, it’ll ultimately destroy the credibility of the language and people will choose other languages. The proposers of async did not take this into account when they were proposing async

Naive question, since I tried my hand at rust years ago, but haven't looked at it since: isn't it possible to write another crate to build go-like channels? A kind of "write, then lose the reference" function call that places a value on a queue, and an accompanying receiver. That could make life easier for "normal" software development.

There are many such primitives in Rust (including one in the standard library). And it's effectively the default, the only annoying thing is the libraries which use async (it is possible to just wrap the async code in sync code, just a little annoying. But I think it's what most users of the language should do.)

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#505

Post by imtringued »

Galanwe wrote: post #438
ghosty141 wrote: post #334

Earlier quoted context omitted.

> And who likes doing this to themselves anyway? Isn't it a very frustrating experience? How is this the most loved language? The thing is, these dependencies do exist no matter what language you use if they stem from an underlying concept. In that case rust just makes you explicitly write them which is a good thing since in C++ all these dependencies would be more or less implicit and everytime somebody edits the co…

> The thing is, these dependencies do exist no matter what language you use Sure, but in a lot of cases, these invariants can be trivially explained, or intuitive enough that it wouldn't even need explanation. While in Rust, you can easily spend a full day just explaining it to the compiler. I remember spending litteral _days_ tweaking intricate lifetimes and scopes just to promise Rust that some variables won't be u…

>use fancy smart pointers, etc.

The thing is, you think your code is safe and it most likely is, but mathematically speaking, what you are doing is difficult or even impossible to prove correct. It is akin to running an NP complete algorithm on a problem that is easier than NP. Most practical problem instances are easy to solve, but the worst case which can't be ruled out is utterly, utterly terrible, which forces you to use a more general solution than is actually necessary.

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#506

Post by awused »

anonymoushn wrote: post #502
awused wrote: post #498

Earlier quoted context omitted.

You're confusing IO not happening because it's not needed with IO never happening. Just because a method can perform IO doesn't mean it actually does every time you call it. If I call async_read(N) for the next N bytes, that isn't necessarily going to touch the IO driver. If your task can make progress without polling, it doesn't need to poll. >I'm not familiar with tokio so I did not know that it maintained buffers…

> You're confusing IO not happening because it's not needed with IO never happening. Just because a method can perform IO doesn't mean it actually does every time you call it. If I call async_read(N) for the next N bytes, that isn't necessarily going to touch the IO driver. Maybe you can read the linked post again? The problem in the example in the post is that data keeps coming from the network. If you were to strac…

Actually, no, I misread it trying to make sense of what you were posting so this post is edited.

This is just mundane non-blocking sockets. If the socket never needs to block, it won't yield. Why go through epoll/uring unless it returns EWOULDBLOCK?

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#507

Post by imtringued »

jstx1 wrote: post #322
pshc wrote: post #313

Earlier quoted context omitted.

Debugging rare crashes and heisenbugs is more frustrating, and in non-safe languages, a chronic problem. Whereas after you prove the safety of a design once, it stays with you.

It stays with you until you need to change something and find yourself unable to make incremental changes. And in many use cases people are throwing Rust (and especially async Rust) on problems solved just fine with GC languages so the safety argument doesn’t apply there.

The safety argument is actually the reason why you can use Rust in those cases to begin with. If it was C or C++ you simply couldn't use it for things like webservers due to the safety problems inherent to these languages. So Rust creeps into the part of the market that used to be exclusive to GC languages.

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#508

Post by pkolaczk »

brundolf wrote: post #307
Pauan wrote: post #276

Earlier quoted context omitted.

The whole point of an abstraction is to remove complexity for the user. So I assume you mean "implementation complexity" but that's irrelevant, because that cost only needs to be paid once, and then you put the abstraction into a crate, and then millions of people can benefit from that abstraction.

You've got a very narrow view that I'd encourage you to be more open-minded about No abstraction is perfect. Every abstraction, when encountered by a user, requires them to ask "what does this do?", because they don't have the implementation in front of their eyes This may be an easy question to answer- maybe it maps very obviously to a pattern or domain concept they already know, or maybe they've seen this exact abs…

The whole point of abstraction is to replace the need of understanding all the details of the implementation with a more general and simpler concept. So while the abstraction itself may have a non zero cognitive cost for the end user, this cost should be lower than the cognitive cost of the full implementation that the abstraction hides. Hence the net cognitive cost of proper abstraction is negative.

Abstractions allow systems to scale. Without them, it would be impossible to work on a system that's 1M lines of code long, because you'd have to read and understand all 1M lines before doing anything.

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#509

Post by mtsr »

throw10920 wrote: post #427
buildartefact wrote: post #414

Earlier quoted context omitted.

>There's no option to "turn the borrow checker off" - which means that when you're in prototyping mode, you pay this huge productivity penalty for no benefit That’s not really true. The standard workaround for this is just to .clone() or Rc > to unblock yourself, then come back later and fix it. It is true that this needs to be done with some care otherwise you can end up infecting your whole codebase with the “worka…

> It is true that this needs to be done with some care otherwise you can end up infecting your whole codebase with the “workaround” It's a "workaround" precisely because the language does not support it. My statement is correct - you cannot turn the borrow-checker off, and you pay a significant productivity penalty for no benefit. "Rc" can't detect cycles. ".clone()" doesn't work for complex data structures.

You can’t turn off undefined behavior in C++ either. Lifetimes exist whether the language acknowledges them or not.

Except if you go to a GC language, but then you’re prototyping other types of stuff than you’d probably pick Rust for.

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#510

Post by kasdi »

Taek wrote: post #151

Rust made a critical safety mistake when it chose its async paradigm. It gave the code the option to decide when to yield. What that means is that when I'm writing async code, I have to audit every library I import to make sure that library is guaranteed to yield after a few microseconds of execution, otherwise my own core loops starve. Importing unknown code when using async rust is not safe for any application that…

Naive question: Why not use OS concurrency then?
Post reply on HN