Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Local async executors and why they should be the default

maciej.codes

Post reply on HN
Print view View original

161–170 of 327 posts

Re: Local async executors and why they should be the default

#161

Post by insanitybit »

jerf wrote: post #154
insanitybit wrote: post #147

Earlier quoted context omitted.

Saying that Rust chose async because node did is such an absurdly ignorant, incorrect statement.

Rust didn't choose async because of Node. Async's massive popularity is because of Node. It existed long before then, and that's precisely part of my point . It had a bad reputation before then, and it's slowly-but-surely reacquiring it now.

Rust didn't choose async because it was popular. You can go ahead and read the massive, multi-year discussions on the topic if you'd like. They've been going on since before 1.0.

Re: Local async executors and why they should be the default

#162

Post by insanitybit »

jacquesm wrote: post #156
insanitybit wrote: post #145

Earlier quoted context omitted.

> Continuations and the way Erlang handles this have far less mental overhead and help keeping the model and the mental representation of that model in sync. Differences between the two is where bugs will hide. You still have a lot to think about in Erlang. For example, you need an entire supervisory system to handle the fact that an actor can die. You need to handle the fact that an actor A might send a message to a…

The supervisory system is an extra, you don't technically need it but it can help make your application bullet proof and I would definitely recommend if you go the Erlang route to use it to your advantage. Every other language and/or runtime will need something similar anyway, but there is a good chance that it won't be nearly as elegant (as as solid) as the way Erlang does this. Agreed that actors are not something…

> The supervisory system is an extra, you don't technically need it but it can help make your application bullet proof and I would definitely recommend if you go the Erlang route to use it to your advantage.

An actor system without supervisors is throwing away a lot. How do you handle an actor that crashes? The thing is, again, Actors are very low level - they're a foundational model. You end up having to build protocols and systems on top.

> If you kept it at the process level that simply could never happen, your process would error out, the supervisor would take over and that would be that.

I don't think you're going to get an orphaned resource by timing out and dropping the future, which drops its state.

Re: Local async executors and why they should be the default

#163

Post by masklinn »

menaerus wrote: post #50
rklaehn wrote: post #21

Earlier quoted context omitted.

Multi threading and concurrency are not the same. You can have a very high performance server that handles thousands of requests concurrently on a single thread. That's how node/deno do things. But the way to do things in async rust is that if you want concurrency you also have to use multithreading. At least that is what you see in all the examples and docs. As soon as you require your futures to be Send you have to…

> So e.g. you have to use Arc > In terms of C++ code that would equate to std::shared_ptr > which ... sounds quite wasteful in terms of scalability/performance. Why is it not possible to simply return a Rust promise? That's the way I do it in my C++ async (executor) library backed by work-stealing queues under the hood.

> Why is it not possible to simply return a Rust promise?

It… is? An Arc is what you need to share mutable data between “promises”.

Re: Local async executors and why they should be the default

#164

Post by surajrmal »

jerf wrote: post #142
seanhunter wrote: post #115

Earlier quoted context omitted.

Async in its current incarnation did not come from node. Eg Haskell had async/await in 1999. https://softwareengineering.stackexchange.com/questions/3774... Node wasn't released until 10 years later.

And I have code still in production written in the "Perl Object Environment", one of several async frameworks for Perl, written before Node even existed. Also before Node existed, I had written code in the Twisted framework for Python. Node did not invent async by any means and I know it in the strongest possible sense, having used it before Node existed. But those libraries, in use for over a decade, did not make ev…

Async await isn't useful because it's more ergonomic. It's useful because it's low overhead. If you don't care about that then rust might not be the right language for you the same way a language with a GC might be better suited for most folks. For my domain, any other choice would have made rust untenable. Rust has made all the right choices that make it an excellent choice for code I was otherwise forced to use c or c++ for. Not every language must bend its ways for every possible user, and most folks get upset when that inevitable happens because it dilutes the qualities that made the language useful for the original set of folks it was primarily useful for. Being popular unfortunately comes with the curse of not pleasing everyone.

Re: Local async executors and why they should be the default

#165

Post by tjoff »

pjmlp wrote: post #158
pdimitar wrote: post #96

Just a personal take: after not coding with Rust for several months, I find it more and more difficult to return to an async code I was writing. The whole thing just reads... ugly and inconsistent. It needs too much already-accumulated knowledge. As the article correctly points out, you need a bunch of stuff that are seemingly unrelated (and syntactically speaking you would never guess they belong together). And as o…

Rust sweetspot is really for use cases where any kind of automatic memory management is forbidden, either due to real use case requirements (high integrity computing, kernel drivers,...), or due to existing domain culture that frowns upon any other kind of alternatives. For everything else, there are more productive alternatives, even Go, which after generics is kind of ok.

Maybe for prototypes or small scripts but disagree otherwise. The promise of automatic memory management is that you don't have to think about memory.

But the second you don't think about memory you are doomed to write bad code anyway. Maybe because of performance but most probably due to architecture. To have a language that forces you to think about memory is not a curse, it is a blessing.

Re: Local async executors and why they should be the default

#166

Post by datadeft »

duped wrote: post #76
datadeft wrote: post #22

For me the biggest issue with Async is the management of multiple dependent async calls. It has some weird thing going on and I am not sure which pattern to use exactly. Some functions expect exactly same async fn signature some not and I am not sure why and which one to use.

I'm confused what you mean here. If you have a function that "depends on" another function you call it within the other function. If it's async you .await it. Do you mean something about spawning tasks or passing callbacks around?

I have this function that I still struggle to write.

  async fn function_handler(_event: Request) -> Result, Error> {
    let aws_clients = AwsClients {
        s3_client: S3_CLIENT.get().await.to_owned(),
        glue_client: GLUE_CLIENT.get().await.to_owned(),
        athena_client: ATHENA_CLIENT.get().await.to_owned(),
    };

    println!("{}", aws_clients);

    let config = CONFIG.get().await;
    let version = VERSION.get().await;

    let db_eid_cache = create_db_eid_cache(aws_clients.clone(), config);
    let table_eid_cache = create_table_eid_cache(aws_clients.clone(), config);
    let job_eid_cache = create_job_eid_cache(aws_clients.clone(), config);
    let table_summaries_cache = create_table_summaries_cache(aws_clients.clone(), config);
    let job_summaries_cache = create_job_summaries_cache(aws_clients.clone(), config);
    let query_summaries_cache = create_query_summaries_cache(aws_clients, config);

    let rez = tokio::join!(
        db_eid_cache,
        table_eid_cache,
        job_eid_cache,
        table_summaries_cache,
        job_summaries_cache,
        query_summaries_cache,
    );

    let six_pack = SixPack {
        db_eid_cache: rez.0,
        table_eid_cache: rez.1,
        job_eid_cache: rez.2,
        table_summaries_cache: rez.3,
        job_summaries_cache: rez.4,
        query_summaries_cache: rez.5,
    };

    let lambda_response = LambdaResponse {
        config,
        result: &six_pack,
        version: &version,
        now: utc_now(),
    };

    let response_body = serde_json::to_string(&lambda_response)?;

    Ok(Response::builder()
        .status(200)
        .header("content-type", "application/json")
        .body(response_body.into())
        .map_err(Box::new)?)
}

Not sure how to do error handling properly or what would be the idiomatic way to implement it.

Re: Local async executors and why they should be the default

#167

Post by pjmlp »

kibwen wrote: post #125
jerf wrote: post #113

Earlier quoted context omitted.

I really think in the end, in another decade or two, the community consensus is going to be that async as it is conceived of today is simply a mistake, full stop. Think about it. Where did it come from in its current incarnation? Node. Why did Node choose it? Did it have a multiplicity of options and carefully choose the best one based on years of experience with each choice? No. Async was "chosen" because it was the…

> Where did it come from in its current incarnation? Node. Rust's async/await is inspired by C# (which was inspired by F#, which was inspired by Haskell). > right now threaded code is straight-up a better option on almost every metric, Agreed that people are generally too eager to reach for async when they could easily get away with threads, and fortunately Rust makes threads extremely pleasant to work with. The whol…

And yet it didn't learn that it took almost 10 years for .NET community to sort out async/await support across all layers of the stack.

.NET archictects have spent last year researching Go and Java Loom approaches, and have acknowledge if it had been today, most likely that would have been the approach taken instead of async/await, as many .NET devs still get it wrong.

During the "ASP.NET Core and Blazor futures, Q&A" at BUILD 2023.

This is the best practices of async/await in .NET as written by one of the ASP.NET architects,

https://github.com/davidfowl/AspNetCoreDiagnosticScenarios/b...

Re: Local async executors and why they should be the default

#168

Post by pjmlp »

cmrdporcupine wrote: post #89
lionkor wrote: post #35

I find all the async stuff in Rust incredibly ugly, cumbersome, and its one of the biggest reasons I prefer C++, still. C++ lets me just write single- or multithreaded code, because none of the dependencies force their `async` stuff on me. Yeah, its up to me to ensure things are synchronized, but I'd rather do that than try to figure out how to get some dependency that isnt meant to use async to work in some async mo…

I find the async stuff obnoxious, too, and I avoid it. Helps that I don't work in the web space. But in general the dependency story with Rust is better than with C++. I don't miss the integration story with C++; just bringing in a dep in the first place is a roll of the dice on whether it's going to work with your build system. And then whether it brings with it some other lifestyle assumptions (exceptions, some thi…

Or just stick with cmake + vcpkg/conan, and enjoy the existing ecosystem of C++ libraries.

Nowadays it is impensable to do a Java project without Maven/Gradle, yet it took about 10 years for Ant to be relevant (counting from 1996), and couple more for Maven, and yet another few for Gradle.

Similarly with NuGET and MSBuild evolution.

Yet they were eventually adopted, same is happening with vcpkg/conan.

Re: Local async executors and why they should be the default

#169

Post by surajrmal »

cmrdporcupine wrote: post #133
jerf wrote: post #113

Earlier quoted context omitted.

I really think in the end, in another decade or two, the community consensus is going to be that async as it is conceived of today is simply a mistake, full stop. Think about it. Where did it come from in its current incarnation? Node. Why did Node choose it? Did it have a multiplicity of options and carefully choose the best one based on years of experience with each choice? No. Async was "chosen" because it was the…

We wrote asynchronous driven network code for decades without syntactic sugar for it, and it was fine. That 5% performance gain can be had without it. Async syntax is a mistake. Not just because of the mess it makes across the program tree, but also because it brings with it a specific notion of how to do asynchronous I/O. io_uring for example gives an entirely different model, one with some lovely performance benefi…

Having written decades of async code without rust (and still doing so in c++), I don't understand your point. It is really hard to write safe highly performant asynchronous code. Rust makes it safe, but without the additional "syntax sugar" it makes it very unergonomic. The developer velocity I can achieve with async rust is unbelievable compared to what I had to do previously.

I think your problem may be with tokio, not async. io_uring and async are not incompatible.

Re: Local async executors and why they should be the default

#170

Post by seanhunter »

jerf wrote: post #142
seanhunter wrote: post #115

Earlier quoted context omitted.

Async in its current incarnation did not come from node. Eg Haskell had async/await in 1999. https://softwareengineering.stackexchange.com/questions/3774... Node wasn't released until 10 years later.

And I have code still in production written in the "Perl Object Environment", one of several async frameworks for Perl, written before Node even existed. Also before Node existed, I had written code in the Twisted framework for Python. Node did not invent async by any means and I know it in the strongest possible sense, having used it before Node existed. But those libraries, in use for over a decade, did not make ev…

A bear indeed. I too used twisted circa about 2001 or so.
Post reply on HN