Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A four year plan for async Rust

without.boats

Post reply on HN
Print view View original

101–110 of 236 posts

Re: A four year plan for async Rust

#101

Post by vorticalbox »

munificent wrote: post #85

> AsyncIterator and async generators For what it's worth, Dart has had synchronous and asynchronous generators (including `await for` statements) for as long as its had async/await. They are neat features. I've definitely written code using synchronous generators that would be hard to manually transform into a custom Iterable implementation. But they add a large amount of complexity to the language implementations an…

I've just used an async generator when upgrading to aws-sdk v3, the function lists all the items in a s3 bucket, if its truncated then it yeilds a recursive generator.

Stops when it's got all the keys.

First time in 5 years writing typescript I've actually used a generator.

Re: A four year plan for async Rust

#102

Post by no_wizard »

insanitybit wrote: post #80
no_wizard wrote: post #64

Earlier quoted context omitted.

Thats red herring IMO. Sure, they're GC'd languages, but that does not preclude the fact that both of these approaches from a DX perspective are easier to work with, and that matters in language design. I didn't say we need goroutines or we need Loom style asynchronous primitives I am however pointing out, that they did a really good job of making async approachable and feel like you're writing the same code as you w…

> I didn't say we need goroutines or we need Loom style asynchronous primitives I am however pointing out, that they did a really good job of making async approachable and feel like you're writing the same code as you would in a traditional synchronous model. You created a dichotomy - languages with a native construct for async versus languages that provide this as libraries. But that dichotomy does not exist - both…

I agree, it should be taken really seriously.

I think 4 years worth of a feature being out in the wild and in use is enough time to start having conversations about what went well and what didn't and how to address that. Its very clear to me, at least, that async in Rust is becoming more and more dependent on tokio. Most of the major async supporting crates support tokio and/or only leverage tokio. Just a cursory glance at the crates registry supports this much.

I'm not against a "pull in a crate" mentality mind you (though, careful what you wish for here, see: NPM / Node ecosystem) however, it is worth identifying when something is becoming / has become / is considered to be such a core feature of the ecosystem that it would benefit greatly from stdlib support, and I think this fits that definition based on the evidence I've seen, at least. Though I realize others may not share this sentiment, I think its a viewpoint that has evidentiary backing (see all the talk about async Rust in the communicate, issues etc surrounding it. Its already a pretty big buzz topic relative to other things surrounding the language)

All this is to say, maybe its time to seriously start thinking about what first party support will look if we bring in a first party async / non-blocking I/O platform into the stdlib

Re: A four year plan for async Rust

#103

Post by bad_user »

andrewstuart wrote: post #84
sargun wrote: post #56

I was a huge opponent of Rust, but finally decided to give it a try in anger once again. I began writing a large application, and noticed that many of my libraries only offered async versions, and the promise was quite appealing -- not having to worry about threads or concurrency as long as I followed certain rules. What I ended up with was an incredibly slow application because of the limitations of Rust async, and…

In the words of Steve Jobs, “you’re holding it wrong”. I’m guessing your code is blocking. You absolutely cannot do blocking code in async in any language, unless the blocking is super quick. No blocking io. async code should yield great performance if you are doing everything the right way. Yes it is single threaded but either run multiple threads in rust or run multiple instances of your program with systemd. That…

Java, C#, probably Go and others, are able to do async I/O on multiple threads. It doesn't help with the actual async I/O being performed, but it does help with parallelising CPU work, or with the fact that not all I/O can be async and the runtime is faking it, such as local file I/O, which is going to be blocking system threads.

Async I/O is all about multiplexing work on few CPU cores efficiently, and multi-threading is still required. Python or JavaScript are seriously limiting environments and should not be given as an example when we're talking about a language that does 1:1 scheduling.

Re: A four year plan for async Rust

#104

Post by mplanchard »

withoutboats3 wrote: post #39
dymk wrote: post #24

Earlier quoted context omitted.

This article is by a Rust contributor. I'm sure they want to see Rust be more widely adopted as well. Async is pretty great, but it's not perfect, and in some situations its shortcomings cannot be worked around easily.

I think the comment this person was responding to is completely off topic for my blog post, and extremely shallow. I share the person you're responding to's frustration with how async Rust is talked about on Hacker News.

I also agree with echelon, and I've been working with async rust daily for two and a half years. It was a bit painful before tokio 1.0, but by and large it's consistently great to work with, especially now. My only complaints remain having to restructure closure-based code into loops and occasional weird lifetime issues caused by the `#[async_trait]` macro, but hopefully the latter will go away with stabilization of `async_trait`.

The amount of "async rust is impossible to use," "async rust is fundamentally broken," etc. commentary that comes up on HN is absurd. I have trouble squaring my own experience with it, and it makes me think people are either complaining just to complain or that they haven't worked with it in anything other than toy contexts. I have a hard time imagining how difficult it would be to maintain our ~100k line rust codebase (http proxy that processes requests & responses, makes DB queries, and makes http requests to other services) and keep any kind of predictable performance by spawning threads, using channels, etc.

Like, not trying to be elitist, but I feel like people are missing the benefits for the sake of piling on or something.

Re: A four year plan for async Rust

#105

Post by no_wizard »

untitaker_ wrote: post #63
inamberclad wrote: post #2

Async is a huge wart and I try and avoid it wherever possible. It's simply not useful in a lot of cases that I encounter. However, if a library uses async, you have little choice but to make your whole project async. This adds to its horrible reputation.

This is a popular sentiment that I share to some degree, but I really wish we could just move on from this discussion. It happens in every post that is even just vaguely about Rust or async. It's like reading complaints about the GIL on a post that is loosely connected to Python! At some point it just gets boring. "I will forever argue that [...]" -- Why are you forever arguing? "I feel like I am alone with [...]" --…

to be honest, HN is one of the few venues where I think someone who is core to a project might actually see what is said about a language. I've tried getting involved in mailing lists and stuff, but they aren't often as welcoming to this discussion as one might assume.

My hope is that by raising these things on HN, someone will take notice in a different way and at least start considering / revisiting alternatives

Re: A four year plan for async Rust

#106

Post by dgacmu »

steveklabnik wrote: post #41
dgacmu wrote: post #37

Earlier quoted context omitted.

I'm curious why you don't like it for embedded - embassy has been an absolute delight for me thus far, and my primary complaint about it is just that it doesn't have the breadth of hardware support (yet?). It's been the thing that has redeemed Rust async for me, as otherwise, I tend to find the tension the author notes frustrating as well.

I am pro async Rust, but we don't use async in our embedded projects at Oxide. This is because of specific design constraints and goals: https://hubris.oxide.computer/reference/#_why_synchronous That being said, I am also a fan of embassy when you have different design constraints and goals, and consider the fact that is is able to exist and be successful is a massive testament to the design of async Rust. (We also u…

Oddly, I agree with you - but I think I may be approaching it differently. I use async as a mechanism to be able to have clearly-defined "tasks" in an embedded context, where tasks have straight-line code that handles something. I have most of the interaction between tasks be synchronous; in the case of embassy, the thing it brings is that it manages that otherwise-spaghetti-feeling mix of state machines in an easy-to-read kind of way.

Example from a current side project, since it's not work-encumbered: A wifi-enabled clock light for my 5yo. It has a task that sits there and every hour pings an SNTP server, updating a (mutex-protected) global with the time state. It has another task that listens for a telnet session for various control signals - which also updates a mutex-protected global config state. And it has a task that spins doing LED effects.

With embassy/async, I can write each of those as a separate task, without paying much attention to what gets invoked by interrupt handlers.

(This particular one is an rp2040, but I use it on stm-based systems as well).

I feel like this is kind of analogous to the discussion of threads-vs-events as a mechanism for structuring code vs. threads as a mechanism for achieving parallelism. :-)

Edited to add: Or, perhaps an alternative view of what I'm doing is that I'm using the embassy runtime as a really lightweight alternative to an RTOS, since I mostly haven't met an RTOS I don't want to throw across the room. An argument against what I'm saying here is, "well, use Hubris as your embedded OS and then you can have tasks and they can be synchronous" - which seems entirely fair.

Re: A four year plan for async Rust

#107

Post by andrewstuart »

bad_user wrote: post #103
andrewstuart wrote: post #84

Earlier quoted context omitted.

In the words of Steve Jobs, “you’re holding it wrong”. I’m guessing your code is blocking. You absolutely cannot do blocking code in async in any language, unless the blocking is super quick. No blocking io. async code should yield great performance if you are doing everything the right way. Yes it is single threaded but either run multiple threads in rust or run multiple instances of your program with systemd. That…

Java, C#, probably Go and others, are able to do async I/O on multiple threads. It doesn't help with the actual async I/O being performed, but it does help with parallelising CPU work, or with the fact that not all I/O can be async and the runtime is faking it, such as local file I/O, which is going to be blocking system threads. Async I/O is all about multiplexing work on few CPU cores efficiently, and multi-threadi…

I don’t really get your point.

You cannot, (should not), do blocking IO in async in any language.

The language provides libraries to do non blocking IO.

What is unclear about that?

Multi threading has nothing to do with async, except as a way to run certain things in an executor thread to avoid blocking.

Also I don’t really know what you mean by async IO…. even in the languages you reference I would guess IO operations are all run in a single thread.

Re: A four year plan for async Rust

#108

Post by wavemode »

dymk wrote: post #18
inamberclad wrote: post #2

Async is a huge wart and I try and avoid it wherever possible. It's simply not useful in a lot of cases that I encounter. However, if a library uses async, you have little choice but to make your whole project async. This adds to its horrible reputation.

> However, if a library uses async, you have little choice but to make your whole project async. This isn't true. I'm writing an application right now that is mostly sync, but has a small amount of async code in it. It's easy to spin up a tokio runtime which can run inline on the current thread. Then use it to evaluate a Future. tokio::runtime::Builder::new_current_thread().enable_all().build().unwrap().block_on(asyn…

The complexity of this code snippet almost seems satirical (though I do get the point you're making, and agree with you).

Re: A four year plan for async Rust

#109

Post by MajimasEyepatch »

vorticalbox wrote: post #101
munificent wrote: post #85

> AsyncIterator and async generators For what it's worth, Dart has had synchronous and asynchronous generators (including `await for` statements) for as long as its had async/await. They are neat features. I've definitely written code using synchronous generators that would be hard to manually transform into a custom Iterable implementation. But they add a large amount of complexity to the language implementations an…

I've just used an async generator when upgrading to aws-sdk v3, the function lists all the items in a s3 bucket, if its truncated then it yeilds a recursive generator. Stops when it's got all the keys. First time in 5 years writing typescript I've actually used a generator.

Generators are one of those things that pop up more often in library code than in application code, I've found.

Re: A four year plan for async Rust

#110

Post by mplanchard »

bad_user wrote: post #103
andrewstuart wrote: post #84

Earlier quoted context omitted.

In the words of Steve Jobs, “you’re holding it wrong”. I’m guessing your code is blocking. You absolutely cannot do blocking code in async in any language, unless the blocking is super quick. No blocking io. async code should yield great performance if you are doing everything the right way. Yes it is single threaded but either run multiple threads in rust or run multiple instances of your program with systemd. That…

Java, C#, probably Go and others, are able to do async I/O on multiple threads. It doesn't help with the actual async I/O being performed, but it does help with parallelising CPU work, or with the fact that not all I/O can be async and the runtime is faking it, such as local file I/O, which is going to be blocking system threads. Async I/O is all about multiplexing work on few CPU cores efficiently, and multi-threadi…

Tokio's default scheduler is multi-threaded, and Rust does this very well. You just have to be explicit when you want to run some non-async blocking code in a way that won't block a thread.
Post reply on HN