Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Maybe Rust isn’t a good tool for massively concurrent, userspace software

bitbashing.io

Post reply on HN
Print view View original

391–400 of 624 posts

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#391

Post by Pauan »

brundolf wrote: post #307
Pauan wrote: post #276

Earlier quoted context omitted.

The whole point of an abstraction is to remove complexity for the user. So I assume you mean "implementation complexity" but that's irrelevant, because that cost only needs to be paid once, and then you put the abstraction into a crate, and then millions of people can benefit from that abstraction.

You've got a very narrow view that I'd encourage you to be more open-minded about No abstraction is perfect. Every abstraction, when encountered by a user, requires them to ask "what does this do?", because they don't have the implementation in front of their eyes This may be an easy question to answer- maybe it maps very obviously to a pattern or domain concept they already know, or maybe they've seen this exact abs…

Even the costs you are talking about are a one-time cost to read the documentation and learn the abstraction. And the long-term benefits of the abstraction are far greater than the one-time costs. That's why we create abstractions, because they are a net gain. If they were not a net gain, we would simply not create them.

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#392

Post by seabrookmx »

marcosdumay wrote: post #317
seabrookmx wrote: post #246

Earlier quoted context omitted.

> It was created to be an improvement over the Javascript situation I see this repeated everywhere in this thread. async/await originated in C# not JS.

The C# implementation is clearly an attempt of putting type-safety over exactly the same implementation JS promises use. Done because MS wanted to port the same behavior.

I honestly can't tell if you're trolling.

- the C# implementation predates even Promises in JS, so it is not "the same implementation" and your implication that C# was inspired by JS as opposed to the other way around is false. More background: [0]

- Typescript works fine with the JS implementation so any differences aren't for type safety reasons, but largely because C# has a multithreaded event loop unlike JS

Also promises (or "futures" as they're called elsewhere) aren't unique to any language. They're used in lots of places that predate both C# and JS's use, for example the twisted framework in Python.

[0]: https://news.ycombinator.com/item?id=37438486

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#393

Post by Pauan »

likeabbas wrote: post #298
Pauan wrote: post #213

Earlier quoted context omitted.

No, tokio does not require your Futures to be thread-safe. Every executor (including tokio) provides a `spawn_local` function that spawns Futures on the current thread, so they don't need to be Send: https://docs.rs/tokio/1.32.0/tokio/task/fn.spawn_local.html I have used Rust async extensively, and it works great. I consider Rust's Future system to be superior to JS Promises.

So you’re stuck choosing a single CPU or having to write send and sync everywhere. There’s a lot of use cases where you would want a thread-per-core model like Glommio to take advantage of multiple cores while still being able to write code like it’s a single thread. > I have used Rust async extensively, and it works great. I consider Rust's Future system to be superior to JS Promises. Sure, but it’s a major headache…

> So you’re stuck choosing a single CPU or having to write send and sync everywhere. There’s a lot of use cases where you would want a thread-per-core model like Glommio to take advantage of multiple cores while still being able to write code like it’s a single thread.

thread_local! exists, and you can just call spawn_local on each thread. You can even call spawn_local multiple times on the same thread if you want.

You can have some parts of your programs be multi-threaded, and then other parts of your program can be single-threaded, and the single-threaded and multi-threaded parts can communicate with an async channel...

Rust gives you an exquisite amount of control over your programs, you are not "stuck" or "locked in", you have the flexibility to structure your code however you want, and do async however you want.

You just have to uphold the basic Rust guarantees (no data races, no memory corruption, no undefined behavior, etc.)

The abstractions in Rust are designed to always uphold those guarantees, so it's very easy to do.

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#394

Post by kevingadd »

randyrand wrote: post #384
hgomersall wrote: post #233

Earlier quoted context omitted.

We do not want functions that take floating point arguments, only u32 should be used. And don't get me started on more than one argument!

you can convert a float to a u32. you cannot convert an function that calls async code into a sync function.

You can call .Wait on the Task it returns :)

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#395

Post by za3faran »

jiggawatts wrote: post #367

I wonder if Rust should have gone down the same path as Java’s Project Loom and implemented async I/O using the same memory model that is used with operating system threads. I suspect that to take advantage of 1024-thread systems the only sane programming model will be structured concurrency with virtual threads instead of coroutines. It’s the same progression as we saw in the industry going from unstructured imperat…

I believe rust started with green threads early on, before ditching them.

To your point, the C# guys seem to be interested in experimenting with green threads: https://twitter.com/davidfowl/status/1532880744732758018

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#396

Post by Pauan »

tel wrote: post #366
Pauan wrote: post #287

Earlier quoted context omitted.

I'm curious what things you consider to be half-baked about Rust async. I've used Rust async extensively for years, and I consider it to be the cleanest and most well designed async system out of any language (and yes, I have used many languages besides Rust).

Async traits come to mind immediately, generally needing more capability to existentially quantify Future types without penalty. Async function types are a mess to write out. More control over heap allocations in async/await futures (we currently have to Box/Pin more often than necessary). Async drop. Better cancellation. Async iteration.

> Async traits come to mind immediately,

I agree that being able to use `async` inside of traits would be very useful, and hopefully we will get it soon.

> generally needing more capability to existentially quantify Future types without penalty

Could you clarify what you mean by that? Both `impl Future` and `dyn Future` exist, do they not work for your use case?

> Async function types are a mess to write out.

Are you talking about this?

    fn foo() -> impl Future
Or this?

    async fn foo() -> u32
> More control over heap allocations in async/await futures (we currently have to Box/Pin more often than necessary).

I'm curious about your code that needs to extensively Box. In my experience Boxing is normally just done 1 time when spawning the Future.

> Async drop.

That would be useful, but I wouldn't call the lack of it "half-baked", since no other mainstream language has it either. It's just a nice-to-have.

> Better cancellation.

What do you mean by that? All Futures/Streams/etc. support cancellation out of the box, it's just automatic with all Futures/Streams.

If you want really explicit control you can use something like `abortable`, which gives you an AbortHandle, and then you can call `handle.abort()`

Rust has some of the best cancellation support out of any async language I've used.

> Async iteration.

Nicer syntax for Streams would be cool, but the combinators do a good job already, and StreamExt already has a similar API as Iterator.

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#397

Post by wvenable »

radicalbyte wrote: post #300
bombela wrote: post #234

Earlier quoted context omitted.

I understand this not as objects are missing, after all, struct with methods and traits are objects aren't they? But more like the lack of hierarchical inheritance, that is most often used in OOP to conveniently share common code with added specialization. Override only the methods you want. You can do it with Traits of course, but it's much more verbose. You can technically use the defer trait to simulate a sort of…

I'm from the C# world and am working through learning Rust... in C# we've largely moved away from using inheritance. Not sure that it's a good thing but "best practise" results in serialisation being implemented differently (serialisers which use attributes, or for more advanced teams serialisation wired in at compile time targeted by attributes - advantage here being that the state doesn't have to be public).

I still use inheritance in C# although it is only used for is-a relationships and those aren't that common. But when you need it for that, it's usually pretty important.

I also think it's much more common to see it in library / framework code and not in application code.

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#398

Post by randyrand »

kevingadd wrote: post #394
randyrand wrote: post #384

Earlier quoted context omitted.

you can convert a float to a u32. you cannot convert an function that calls async code into a sync function.

You can call .Wait on the Task it returns :)

Right, but now you are forced to convert the calling function to async.

u32 / float does not have the problem. It does not "bubble up", unless you want it to.

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#399

Post by vulcan01 »

lowbloodsugar wrote: post #107

I don’t know enough rust to comment on that part of the article but I’ve run GC-based systems at scale and load and the remarks about that do not match my experience at all. I've coded performant applications on an OS that used channels and it sucked. It just got in the way and was confusing to engineers used to lower level constructs. "Just get out of my way!" I think rust async is hard. And that's what it comes dow…

> an OS that used channels

I'm very curious; what OS is this?

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#400

Post by Pauan »

brabel wrote: post #196
lordnacho wrote: post #9

I find myself in this weird corner when it comes to async rust. The guy's got a point in that doing a bunch of Arc, RwLock, and general sharing of state is going to get messy. Especially once you are sprinkling 'static all over the place, it infects everything, much like colored functions. I did this whole thing once back when I was starting off where I would Arc stuff, and try to be smart about borrow lifetimes. Tot…

The author does mention that you should probably stop at using Threads and passing data around via channels... but then mentions the C10K problem and says that sometimes you need more... but does not answer the question that I think is begging to be asked: does using Rust async with all the complications (Arc, cloning, Mutex whatever) does actually outperform Threads/channels?? Even if it does, by how much? It would…

Threads cannot scale at all, because you're limited to the number of threads (which is usually quite small).

Async code can scale essentially infinitely, because it can multiplex thousands of Futures onto a single thread. And you can have millions of Futures multiplexed onto a dozen threads.

This makes async ideal for situations where your program needs to handle a lot of simultaneous I/O operations... such as a web server:

http://aturon.github.io/blog/2016/08/11/futures/

Async wasn't invented for the fun of it, it was invented to solve practical real world problems.

Post reply on HN