Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A four year plan for async Rust

without.boats

Post reply on HN
Print view View original

201–210 of 236 posts

Re: A four year plan for async Rust

#201

Post by [deleted] »

0x457 wrote: post #172
tuetuopay wrote: post #111

Earlier quoted context omitted.

Tokio does sync I/O in the background with dedicated I/O threads if memory serves. (it is sync in the sense it does syscalls, which are pushed to a dedicated thread).

Tokio has a thread pool for blocking operations. It's not automatic, though, you have to explicitly use it. I don't understand why some people think async is difficult to write. It's difficult if language doesn't have good support for it, but rust does. I remember async ruby was pretty weird to write initially.

[deleted]

Re: A four year plan for async Rust

#202

Post by cmrdporcupine »

steveklabnik wrote: post #200
lucasyvas wrote: post #193

Earlier quoted context omitted.

I agree with you 100%, but members of the Rust project have stated in the past that async, in not insignificant ways, is a play to deliberately target the higher level web/service based crowd. So, the cultural issue is partially self-inflicted. I'm not even sure what I want in Rust. All I know is that people have a legitimate gripe being sold on Rust async for high level work, having it ergonomically fall much flatte…

> the higher level web/service based crowd. Not everyone that writes network services is "higher level" or "web".

That is entirely fair. But back to the topic -- some of us building network services etc would like options, and not to be tied to a specific async runtime. E.g. I was looking at what it would take to move my code over to monoio. Or anything else. Not going to happen, because too many 3rd party deps mandate tokio.

That's not a good situ. I'd as soon rip out async, and optimize the concurrent multithreaded situation myself than be tied down that way.

FWIW my day job is on embedded Linux, small systems that sit in tractors. We don't do async Rust. It's all actor-style communicating components. And it works pretty much fine.

Re: A four year plan for async Rust

#203

Post by steveklabnik »

cmrdporcupine wrote: post #202
steveklabnik wrote: post #200

Earlier quoted context omitted.

> the higher level web/service based crowd. Not everyone that writes network services is "higher level" or "web".

That is entirely fair. But back to the topic -- some of us building network services etc would like options, and not to be tied to a specific async runtime. E.g. I was looking at what it would take to move my code over to monoio. Or anything else. Not going to happen, because too many 3rd party deps mandate tokio. That's not a good situ. I'd as soon rip out async, and optimize the concurrent multithreaded situation m…

I hear you, options would be nice for sure. We'll see if making things more generic is in the cards or not, I guess.

Re: A four year plan for async Rust

#204

Post by gnulinux »

kingofthehill98 wrote: post #70
dymk wrote: post #18

Earlier quoted context omitted.

> However, if a library uses async, you have little choice but to make your whole project async. This isn't true. I'm writing an application right now that is mostly sync, but has a small amount of async code in it. It's easy to spin up a tokio runtime which can run inline on the current thread. Then use it to evaluate a Future. tokio::runtime::Builder::new_current_thread().enable_all().build().unwrap().block_on(asyn…

Every day we stray further from god.

This is the kind of thing you write once to abstract out async things in a function and call it a day. It really isn't that bad. Besides you can just use:

    smol::block_on(async {
        println!("I'm async!");
    });
if smol is an option.

Re: A four year plan for async Rust

#205

Post by bryanlarsen »

ironhaven wrote: post #181
explodingwaffle wrote: post #92

Tangential to some comments here: Why does the standard library not have block_on?

block_on is a executor that polls a future until it returns ready. If the future is not ready then the executor wait until it is woken up. If you poll a future that relies on a specific runtime (I.g. A async library that uses Tokio) to wake up the executor, with a simple block_on you will get stuck. A generic block_on in std would be a footgun for new programmers I believe.

The proposals to add block_on generally include an executor like pollster.

Re: A four year plan for async Rust

#207

Post by dpc_01234 »

insanitybit wrote: post #168
dpc_01234 wrote: post #158

Earlier quoted context omitted.

Example from the recent past: I wanted a quick Rust code to stream s3 object, uncompress with zstd and untar the content to a directory. aws-sdk-s3 supports only async (tokio), tar crate only blocking, async-tar only async-std, async-compression only tokio. Hard to paper over it with `block_on` because of the streaming part. I don't remember what I actually did, but it doesn't matter - having to come up with a magic…

> And using Rust ecosystem was easier just a few years ago than it is now for these use-cases I used to write Rust web services (professionally) before async and it was definitely not easier for me. It was way harder. I'd end up with accidental hanging because some socket didn't have a timeout set on it properly, it was extremely leaky (why the hell am I talking to raw socket APIs just so that I can read from S3?), a…

> why the hell am I talking to raw socket APIs just so that I can read from S3

I don't know. https://doc.rust-lang.org/std/net/struct.TcpStream.html#meth... etc. were there for a while. Though I think that deadline-based timeouts would be way better to have. https://www.reddit.com/r/rust/comments/8b5krv/the_case_for_d... . Probably could be built around existing primitives. These things were never built/popularized because the community just jumped on async like some silver bullet.

I agree that building heavy duty networked services got easier with async, but at an expense of fracturing the ecosystem and dragging everything else into an MVP feature, which made things worse (at least in some respect) for other things. I personally don't do much web servers, and when I do I can just spawn lots of threads, terminate TLS with nginx anyway.

Again, I don't mind async on its own. I think it's great for what it is, and is useful when it's useful. But for decades tons of the web was built in Java, Python, RoR without async IO and it worked just fine. And I didn't have to play IO-type sudoku, and could expect that the most basic, native blocking IO is well supported, and not just an afterthought/wrapper.

Re: A four year plan for async Rust

#208

Post by qup »

insanitybit wrote: post #171
Elinvynia wrote: post #159

Earlier quoted context omitted.

The problem is that a large number of popular libraries has converted to async, 95+% of them to Tokio. So you are stuck with smaller, less battle tested products if you'd rather not pull in 100+ crates of dependencies that are doing nothing but inflating the build times and file sizes (for your particular usecase). Example: reqwest vs ureq

OK, but like, can we just be honest then that the problem here is that your build times go up? People act like it's an insurmountable problem rather than just a trivial trade-off where, yes, your build times will go up because of some extra dependencies on an async runtime. Increased build times are not great but holy shit the way people talk you'd never know that that's the actual trade-off here, an extra 3 seconds…

Adding dependencies is not easy in some organizations. You have to trust them, in addition to waiting 3 seconds to compile them.

Re: A four year plan for async Rust

#209

Post by insanitybit »

qup wrote: post #208
insanitybit wrote: post #171

Earlier quoted context omitted.

OK, but like, can we just be honest then that the problem here is that your build times go up? People act like it's an insurmountable problem rather than just a trivial trade-off where, yes, your build times will go up because of some extra dependencies on an async runtime. Increased build times are not great but holy shit the way people talk you'd never know that that's the actual trade-off here, an extra 3 seconds…

Adding dependencies is not easy in some organizations. You have to trust them, in addition to waiting 3 seconds to compile them.

OK, in some niche scenarios I can see the cost being larger, but I think this is totally overstated.

Re: A four year plan for async Rust

#210

Post by gwbas1c »

gwbas1c wrote: post #190
starcraft2wol wrote: post #186

Earlier quoted context omitted.

I agree with most of this. > without needing to get into the weeds of threading A thread is the exact concept needed to describe that and preserve “if else then” sequential code. > 1000s of threads Linux handles thousands of threads just fine. If you’re using a scripting language like python, context switching is the least of your performance concerns.

> A thread is the exact concept needed to describe that and preserve “if else then” sequential code. I suggest looking at C# and Javascript that implement async very, very well. The difference is that with conventional threading semantics, when you join a thread, it doesn't return a result. You still need to write some kind of "thing" to get your result from the subthread to whatever's waiting on it. (C# also provide…

Too late to edit: I should also point out that Rust has the same issue with joining a thread: It doesn't give you the result of the function; unlike awaiting on a promise.

(I really struggled with async rust, so I'll admit that I don't remember the name of the type that represents the promise.)

Post reply on HN