Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Local async executors and why they should be the default

maciej.codes

Post reply on HN
Print view View original

211–220 of 327 posts

Re: Local async executors and why they should be the default

#211

Post by jacquesm »

lossolo wrote: post #206
jacquesm wrote: post #199

Earlier quoted context omitted.

Any design pattern can be applied in a wrong way.

Sure it can, but all design patterns have their use cases. Generalizing and saying that one is just "terrible" (which your previous comment implied), and people "should learn" is just untrue and an ignorant take. Sometimes it's the only sane solution compared to others, considering project constraints.

Yes, that one is terrible compared to the alternatives. If there is one thing that really irritates me about the way we go about IT then it is that stuff that works and is reliable and well understood gets tossed and replaced by 'new shiny thing' which then eventually undergoes the same treatment. We are eternally stuck in reinventing wheels, rather than that we make real progress in for instance reliability and are able to accept liability for what we create.

Reducing scope and simplification, increasing reliability and ultimately aiming for software as a true engineering profession should be our goal. Not fashion and running after certain other eco-systems because they are perceived to be in competition. Rust has promise, it is potentially a game changer and this sort of distraction can be done without. It feels as if over time the Rust project is being hijacked by people that want it to be everything for everybody, and now places itself not only in competition with C (and possibly C++) but also with Node (which is itself in competition with C).

Especially considering project constraints of the Rust project it might pay off to see what ultimately is what makes C++ a problematic language. This can be traced back to Stroustrup dragging in everything and the kitchen sink rather than to stick to his original vision, which was 'C but with classes'. The end result is a giant hairball and fifty (ok, exaggerated) ways of achieving the same thing.

Re: Local async executors and why they should be the default

#212

Post by pdimitar »

Nullabillity wrote: post #200
pdimitar wrote: post #104

Earlier quoted context omitted.

I do that of course, and that's one of the easiest ways to use async Rust. In real projects you need much more however. F.ex. I had to code an example of how to add tasks to an already running pool of tasks and posted my findings here: https://github.com/dimitarvp/rust-async-examples/blob/main/e... (there's #2 as well with some more comments and a different approach). The fact that I needed to make a GitHub repo and…

`flowmatic.Do` is equivalent to `future::try_join` ( https://docs.rs/futures/latest/futures/future/fn.try_join.ht... ), and `.Each` is equivalent to `stream::iter().map().buffer_unordered()` ( https://docs.rs/futures/latest/futures/stream/fn.iter.html , https://docs.rs/futures/latest/futures/stream/trait.StreamEx... , https://docs.rs/futures/latest/futures/stream/trait.StreamEx... ). `.ManageTasks` doesn't seem to ha…

That's on me for making blanket statements -- my apologies. Yep, I know about the first two and I used them regularly.

My issue in general is that async Rust involves using several things at the same time (not just the two keywords) and this should be surfaced much earlier in any intro material and the maintainers of the language (or the libraries, or both) should just double down on whatever they feel is the best way to do async Rust.

The stance of "you have freedom, make your choice and assemble your own LEGO" is not very productive. I want Rust to start being a bit more opinionated.

Though I'll recognize this is just a personal taste but I still have to defend it by saying that as a programmer who is paid to deliver, I want to be able to deliver in predictable timelines and not having to go off on an adventure to learn all the intricacies of async Rust before I am able to write a feature.

Re: Local async executors and why they should be the default

#213

Post by lenkite »

pdimitar wrote: post #96

Just a personal take: after not coding with Rust for several months, I find it more and more difficult to return to an async code I was writing. The whole thing just reads... ugly and inconsistent. It needs too much already-accumulated knowledge. As the article correctly points out, you need a bunch of stuff that are seemingly unrelated (and syntactically speaking you would never guess they belong together). And as o…

No, it is completely fair to say that reading and comprehending Golang concurrent code is far-far easier than reading Rust Async code. With Virtual Threads and Structured Concurrency, even concurrent Java code is easy to read nowadays.

Rust Async is far more difficult to understand than Rust Sync. You can't just dive in. You need to have coffee and truly focus your mind like you are solving a Math Olympiad problem. I find it difficult to believe that a community language came to this ridiculous level of complexity.

Re: Local async executors and why they should be the default

#214

Post by lycopodiopsida »

packetlost wrote: post #178
surajrmal wrote: post #172

Earlier quoted context omitted.

Lower overhead. It's the same reason you would need to avoid a GC. It may not be for everyone, but that isn't a rust goal necessarily.

I'm aware that Rust's threading has higher overhead because they're system threads, but what about green/user threads? Is there intrinsic overhead to userspace threads that async doesn't have? I think every "task"/thread needs it's own callstack space, and you're paying scheduling overhead no matter what. Is there literature on the topic?

As far as I understand, green/user threads were on the table in rust team initially, but were sorted out because having green threads also means there is a runtime managing them which rust tries to avoid.

What I do not understand is where the principal difference with async lies: it has no runtime, but you have to bring your own for it to work anyway in form of e.g. tokio. What rust goal conflicts with a similar solution for green threads?

Re: Local async executors and why they should be the default

#215

Post by marcosdumay »

rklaehn wrote: post #198
pjmlp wrote: post #158

Earlier quoted context omitted.

Rust sweetspot is really for use cases where any kind of automatic memory management is forbidden, either due to real use case requirements (high integrity computing, kernel drivers,...), or due to existing domain culture that frowns upon any other kind of alternatives. For everything else, there are more productive alternatives, even Go, which after generics is kind of ok.

Automatic memory management is not exclusively positive, even if you don't care about performance. If you lose deterministic destructors and therefore the possibility of RAII, for me it is a loss. I worked with lots of garbage collected languages (Java, C#, Scala) before rust, and I don't miss garbage collection at all...

> If you lose deterministic destructors and therefore the possibility of RAII, for me it is a loss.

Hum... Haskell does the "getting out of scope means it's clean-up" thing quite fine. Just like it's async handling is only paralleled to the agent-based languages.

It's just a matter of looking at a language where people want to do good things, instead of "we always did it this way".

Re: Local async executors and why they should be the default

#216

Post by pdimitar »

jacquesm wrote: post #137
pdimitar wrote: post #96

Just a personal take: after not coding with Rust for several months, I find it more and more difficult to return to an async code I was writing. The whole thing just reads... ugly and inconsistent. It needs too much already-accumulated knowledge. As the article correctly points out, you need a bunch of stuff that are seemingly unrelated (and syntactically speaking you would never guess they belong together). And as o…

Any kind of mechanism like that requires a mental model that carries a ton of state because it is no longer immediately visible what the scope of execution is. Continuations and the way Erlang handles this have far less mental overhead and help keeping the model and the mental representation of that model in sync. Differences between the two is where bugs will hide. The web isn't asynchronous it's synchronous in almo…

Oh, hey again. :)

And yes I work mostly with Elixir for years and Erlang's model is just irreplaceable so far. You're quite right that everything is synchronous but all the Erlang "processes" (green threads, fibers or w/e people want to call them) are preemptively and forcibly switched. And that has basically eliminated 95% of all parallel programming problems.

I would kill for Erlang's concurrency / parallel primitives in Rust, especially if they come with their own DSL (likely via Rust proc macros?) and are much shorter and comprehensible than they are right now. :|

> It essentially gets you all of the complexity of interrupt driven code for very little gain

Yep, sadly. I've written some fairly impressive commercial Rust code but it added gray hairs and left me wondering whether that was the best way to do it.

Re: Local async executors and why they should be the default

#217

Post by pdimitar »

lenkite wrote: post #213
pdimitar wrote: post #96

Just a personal take: after not coding with Rust for several months, I find it more and more difficult to return to an async code I was writing. The whole thing just reads... ugly and inconsistent. It needs too much already-accumulated knowledge. As the article correctly points out, you need a bunch of stuff that are seemingly unrelated (and syntactically speaking you would never guess they belong together). And as o…

No, it is completely fair to say that reading and comprehending Golang concurrent code is far-far easier than reading Rust Async code. With Virtual Threads and Structured Concurrency, even concurrent Java code is easy to read nowadays. Rust Async is far more difficult to understand than Rust Sync. You can't just dive in. You need to have coffee and truly focus your mind like you are solving a Math Olympiad problem. I…

Yikes. You summarized it much better than I could, kudos.

Re: Local async executors and why they should be the default

#218

Post by marcosdumay »

morelisp wrote: post #188
tjoff wrote: post #182

Earlier quoted context omitted.

The promise with a good architecture is that the doom part is more infrequent though. And that when the Go/Java doom scenario bites you you'll have a whole lot of technical debt to pay back.

But you're arguing in circles. If attention is paid, if the architecture is good, all else like programmer salary/skill being equal, any of those languages (and plenty of others) will work fine. (And honestly, not a huge difference in the work to achieve that, a good design will look about the same in all four.) It's only an interesting question which to use if attention is not paid, or you don't want to invest enoug…

> any of those languages (and plenty of others) will work fine

Well, C won't.

Any large C code needs a decade or two of bug-fixing before it is safe, and that only happens if the developers are competent and wiling to fix it.

Re: Local async executors and why they should be the default

#219

Post by ikekkdcjkfke »

gpderetta wrote: post #81
notamy wrote: post #69

Earlier quoted context omitted.

Sometimes you want to spawn a [future/task/promise/...] and not immediately await it, but instead pass it around. Suppose something like: should_be_future = async_function() should_execute = async_function() # While the compiler could figure out where to insert awaits automatically, it adds cognitive overhead for the developer -- suddenly the same way of calling a function can result in either a [future/task/promise/…

Then you should annotate the tricky diverging call site with async (see my other comment), instead of making it implicit and requiring annotation of the common expected case: should_be_future = async async_function() should_execute = async_function() #

I just don't run into many cases where I'm not immediately awaiting a task, so instead of littering the code base with async await and their return types all the way up, there could be a lighweight version that free's the thread up when waiting on a blocking call (I'm assuming the main reason we want to use async in C# is to free the thread up to do other tasks).

NotAPromiseObject ReadCsv(FileFile file){ string content = waitfor File.ReadAsync(file) ...

Re: Local async executors and why they should be the default

#220

Post by pdimitar »

insanitybit wrote: post #145
jacquesm wrote: post #137

Earlier quoted context omitted.

Any kind of mechanism like that requires a mental model that carries a ton of state because it is no longer immediately visible what the scope of execution is. Continuations and the way Erlang handles this have far less mental overhead and help keeping the model and the mental representation of that model in sync. Differences between the two is where bugs will hide. The web isn't asynchronous it's synchronous in almo…

> Continuations and the way Erlang handles this have far less mental overhead and help keeping the model and the mental representation of that model in sync. Differences between the two is where bugs will hide. You still have a lot to think about in Erlang. For example, you need an entire supervisory system to handle the fact that an actor can die. You need to handle the fact that an actor A might send a message to a…

You seem to be mostly arguing that all the building blocks are in place and you can assemble your own LEGO.

Which is exactly what I started disliking lately. I want async Rust more opinionated with much more huge red arrows pointing in the right directions for cases A, B and C, and links to docs and even 3rd party crates for cases from D to Z.

Post reply on HN