Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A four year plan for async Rust

without.boats

Post reply on HN
Print view View original

171–180 of 236 posts

Re: A four year plan for async Rust

#171

Post by insanitybit »

Elinvynia wrote: post #159
mplanchard wrote: post #87

Earlier quoted context omitted.

If you don't want to pull in a helper library to run async code in a sync context, then why pull in an async library at all? Rust is not a batteries-included language like Python. There are lots of libraries that are very commonly used in most projects (serde, thiserror, and itertools are in almost all of mine), but this is a conscious choice. They say in Python that the stdlib is where projects go to die. I'd rather…

The problem is that a large number of popular libraries has converted to async, 95+% of them to Tokio. So you are stuck with smaller, less battle tested products if you'd rather not pull in 100+ crates of dependencies that are doing nothing but inflating the build times and file sizes (for your particular usecase). Example: reqwest vs ureq

OK, but like, can we just be honest then that the problem here is that your build times go up? People act like it's an insurmountable problem rather than just a trivial trade-off where, yes, your build times will go up because of some extra dependencies on an async runtime.

Increased build times are not great but holy shit the way people talk you'd never know that that's the actual trade-off here, an extra 3 seconds on a clean build.

Re: A four year plan for async Rust

#172

Post by 0x457 »

tuetuopay wrote: post #111
nextaccountic wrote: post #71

Earlier quoted context omitted.

Why didn't you use the multithreaded executor from Tokio?

Tokio does sync I/O in the background with dedicated I/O threads if memory serves. (it is sync in the sense it does syscalls, which are pushed to a dedicated thread).

Tokio has a thread pool for blocking operations. It's not automatic, though, you have to explicitly use it.

I don't understand why some people think async is difficult to write. It's difficult if language doesn't have good support for it, but rust does. I remember async ruby was pretty weird to write initially.

Re: A four year plan for async Rust

#173

Post by geodel »

mplanchard wrote: post #90
couchand wrote: post #16

Earlier quoted context omitted.

Just because it's a network request doesn't mean it benefits from using async. EDIT: to add a specific example, postgres. I understand from sfackler's perspective why it makes sense to maintain just an async library and a sync wrapper around it. But from the user's perspective there'a absolutely no reason that all of tokio should be required to talk to their db.

Nothing is stopping anyone from writing DB drivers that don't require async. If there's as much of a market for it as threads like this suggest, it seems like it would be a fairly popular project.

Huh, thats rather odd argument. It is like saying no one is stopping anyone to write new low level, memory-safe programing language, that won't have Rust style async system. If there is real market demand it could fairly popular project.

DB drivers, http servers, runtimes and so many other complex components are preferences of high skilled developers and not some objective market choices. Once its developed most application developers have to use it however user unfriendly it maybe.

Re: A four year plan for async Rust

#174

Post by insanitybit »

chlorion wrote: post #167
withoutboats3 wrote: post #36

Earlier quoted context omitted.

As a user who has maintained network services that communicate with Postgres, I'm very glad that sfackler's postgres library doesn't perform blocking IO calls, as that would make it unfit for purpose for me.

"async" and threads are not the only options for concurrent IO. I've never saw anyone discuss using poll(2) style concurrency here which is interesting but also kind of sad. With the mio crate for example, it uses the operating systems "select/poll" interface. On Linix this is epoll(7), but other OSes have their own interface. "poll" style concurrency is a lot less complicated for me. The general idea is to register…

"mio" was the game in town for years and it was definitely not ideal at all, it was promising but very low level and most people didn't use it if they could help it. People wanted async/await.

Re: A four year plan for async Rust

#175

Post by cmrdporcupine »

mplanchard wrote: post #162
tuetuopay wrote: post #77

Earlier quoted context omitted.

I would not think it would be that good. Having multiple runtimes is great for specific use cases. I would also fear that having "one standard endorsed runtime" would lead to either the death of the alternative ones, or to the freeze of it once it's in the stdlib and fall to oblivion (like many packages in the python standard library). What the stdlib actually needs is the proper set of traits/facades/whatever to int…

Agree on both points. While I don't find async rust hard to use as a library consumer, trying to write a runtime-agnostic library is (IMO) more difficult than it needs to be. I would like to see most of `futures` either be incorporated into the stdlib or used more consistently throughout the various runtimes.

I think we'd need, at least, runtime agnostic locks, channels, and a 'spawn' function (trickier). And ideally some I/O primitives somehow not tied to the runtime?

Re: A four year plan for async Rust

#176

Post by jcranmer »

tuetuopay wrote: post #77
lucasyvas wrote: post #40

I think std needs a default runtime, and we might as well make it tokio, but maybe make the single-threaded executor the default instead and tweak the API where appropriate to align with this change. Swapping the executors out should absolutely be a feature, and the traits should be portable, but a way to start fixing the situation beyond the great suggestions in this proposal is to acknowledge that std and no-std us…

I would not think it would be that good. Having multiple runtimes is great for specific use cases. I would also fear that having "one standard endorsed runtime" would lead to either the death of the alternative ones, or to the freeze of it once it's in the stdlib and fall to oblivion (like many packages in the python standard library). What the stdlib actually needs is the proper set of traits/facades/whatever to int…

There's two sets of missing things here.

The first is interfaces to represent async versions of existing core sync traits (that's the AsyncRead/AsyncWrite/Stream/Sink/etc. you refer to). What makes this somewhat awkward is you're providing these traits without an implementation of them in the standard lib.

The other thing that's missing is something like GlobalAlloc. You need a generic executor runtime interface to handle some of the executor things you need like "schedule new task".

In general, I think there's a class of features where you need to have just one global (really, process-wide, not just library-wide) provider of some service, and it may be worth having language features to provide this functionality. Memory allocation is one specific area; async executors is another topical area. But you can also throw in stuff like signal handlers or logging or service providers or error handling or tracing features, etc.

Re: A four year plan for async Rust

#177

Post by jcranmer »

Kranar wrote: post #143
samus wrote: post #58

Earlier quoted context omitted.

The problem with C++ is that every feature - attempts to solve too many problems, - has significant papercuts that have to be considered, - has significant runtime cost, - interacts poorly with other features, or at least leaves significant edge cases open, - has non-uniform compiler support (although in OSS land only GCC and LLVM matter), and - creates arcane error messages that are anything but fun to analyze. Espe…

Don't forget that C++ features are also often rolled out before anyone even really understands them, and then after they are rolled out people find all kinds of footguns and issues with them. The C++ committee basically evaluates features almost entirely in the abstract, writing papers and discussing privately among a small group of people about things instead of working on proof of concept implementations, getting a…

> The C++ committee basically evaluates features almost entirely in the abstract, writing papers and discussing privately among a small group of people about things instead of working on proof of concept implementations, getting actual real world feedback on it, or having something along the lines of Rust's nightly where experimental features can be tried out.

Sitting on the C++ committee, we actually demand a lot of proof-of-concept implementation of proposals before they'll be accepted.

Re: A four year plan for async Rust

#178

Post by galangalalgol »

empath-nirvana wrote: post #114
inamberclad wrote: post #2

Async is a huge wart and I try and avoid it wherever possible. It's simply not useful in a lot of cases that I encounter. However, if a library uses async, you have little choice but to make your whole project async. This adds to its horrible reputation.

I genuinely don't understand the problems people have with async code... at least for anything involving http/api requests it's largely just a matter of decorating stuff with 'async/await'. It makes a few things difficult (iterators with futures, ugh), but mostly it's easy.

Because rust is a systems language, it is common for people to use it for things that don't involve http or even networking in general. Alternatively, people often use it for wasm. Before async and specifically tokio, a much larger percentage of crates were usable by those people. This feels like something nice has been taken away or that our use case has been marginalized. The answer is to spend the time to create tokio and/or async free alternatives to these crates, but that was not expected to be necessary, so it also is added work. All of that is irritating, and the constant stream of "just accept it and get with the program. Stop complaining!" Is really quite infuriating. The fair answer is that rust should be what the majority of its users want/need it to be. But people who just need a back end language have a dozen or more good ones to choose from. People who need a systems language have far fewer choices, especially if you want memory safety, so having the aims of the language diverted from that end is irritating to say the least. I do suspect a fork to occur someday for those reasons, probably centered around the use in linux android or windows.

Re: A four year plan for async Rust

#179

Post by bryanlarsen »

chlorion wrote: post #167
withoutboats3 wrote: post #36

Earlier quoted context omitted.

As a user who has maintained network services that communicate with Postgres, I'm very glad that sfackler's postgres library doesn't perform blocking IO calls, as that would make it unfit for purpose for me.

"async" and threads are not the only options for concurrent IO. I've never saw anyone discuss using poll(2) style concurrency here which is interesting but also kind of sad. With the mio crate for example, it uses the operating systems "select/poll" interface. On Linix this is epoll(7), but other OSes have their own interface. "poll" style concurrency is a lot less complicated for me. The general idea is to register…

Several of my rust programs use tokio's select! macro as their main loop. I really like that pattern.

Re: A four year plan for async Rust

#180

Post by gwbas1c »

starcraft2wol wrote: post #11
inamberclad wrote: post #2

Async is a huge wart and I try and avoid it wherever possible. It's simply not useful in a lot of cases that I encounter. However, if a library uses async, you have little choice but to make your whole project async. This adds to its horrible reputation.

The entire unix operating system is designed so that you can write sequential code. It abstracts concurrency away. It’s incredible.

The point of async is to move concurrency from the OS into the process.

The specific issue is the context switch. The compiler, with async is able to be much more performant than the context switch that the OS provides.

Depending on the kind of application you're writing, this is either splitting hairs or very, very important. Applications that handle many (hundreds, 1000s,) of concurrent IO will have a noticeable performance improvement using async versus the OS's context switching.

But, there's a more important thing to consider: "async" in a programming language communicates that a method can block. It allows the caller to start the call and do something while it's waiting for the result, without needing to get into the weeds of threading. Purely relying on the OS for context switching means that it's hard to know what methods block.

Post reply on HN