Asynchronous IO: the next billion-dollar mistake? • Hacker News Board

Re: Asynchronous IO: the next billion-dollar mistake?

#41

Post by csomar » Sat, Sep 07, 2024, 7:46 AM UTC

> Now imagine a parallel universe where instead of focusing on making asynchronous IO work, we focused on improving the performance of OS threads such that one can easily use hundreds of thousands of OS threads without negatively impacting performance

Isn't that why async I/O was created in the first place?

> Just use 100 000 threads and let the OS handle it.

How does the OS handle it? How does the OS know whether to give it CPU time or not?

I was expecting something from the OP (like a new networking or multi-threading primitive) but I have a feeling he lacks an understanding of how networking and async I/O works.

Re: Asynchronous IO: the next billion-dollar mistake?

#42

Post by dietr1ch » Sat, Sep 07, 2024, 7:46 AM UTC

dist1ll wrote: post #36

winternewt wrote: post #27

> Now imagine a parallel universe where instead of focusing on making asynchronous IO work, we focused on improving the performance of OS threads such that one can easily use hundreds of thousands of OS threads without negatively impacting performance I actually can't imagine how that would ever be accomplished at the OS level. The fact that each thread needs its own stack is an inherent limiter for efficiency, as sw…

> But at the programming language level the compiler does have insight into the dependencies of your continuation This is really the key point - coupled with the fact that certain I/O operations are just inherently asynchronous. The TX/RX queues in NICs are an async, message passing interface - regardless of whether you're polling descriptors or receiving completion interrupts. So really, async I/O is the natural abs…

Anything that happens far enough from the CPU is async, and here far probably means 10cm thanks to the speed of light not being fast enough, even in vacuum.

So, any computation that spans a machine the size of our hands needs async unless you are willing to drop the clocks to push "far" a bit further away (and bring in power, heat and noise with it).

Re: Asynchronous IO: the next billion-dollar mistake?

#43

Post by pizza » Sat, Sep 07, 2024, 7:46 AM UTC

Correct me if I'm wrong, but Microsoft's DirectStorage seems to me something like what the author is writing about. It lets you do eg massively parallel NVME file io ops from the GPU itself of lots of small files. This avoids the delay of the path through the CPU, any extra threads/saturation of the CPU, and even lets you do eg decompression of game assets on the GPU itself thereby saving even more CPU. This demo benchmark shows DEFLATE going from 1 GB/s on CPU to 7 GBs/ on GPU https://github.com/microsoft/DirectStorage/tree/main/Samples...

Re: Asynchronous IO: the next billion-dollar mistake?

#44

Post by alexgartrell » Sat, Sep 07, 2024, 7:47 AM UTC

> File IO is perhaps the best example of this (at least on Linux). To handle such cases, languages must provide some sort of alternative strategy such as performing the work in a dedicated pool of OS threads.

AIO has existed for a long time. A lot longer than io_uring.

I think the thing that the author misses here is that the majority of IO that happens is actually interrupt driven in the first place, so async io is always going to be the more efficient approach.

The author also misses that scheduling threads efficiently from a kernel context is really hard. Async io also confers a benefit in terms of “data scheduling.” This is more relevant for workloads like memcached.

Re: Asynchronous IO: the next billion-dollar mistake?

#45

Post by junon » Sat, Sep 07, 2024, 7:48 AM UTC

BenoitP wrote: post #11

> Now imagine a parallel universe where instead of focusing on making asynchronous IO work Funny choice of words. In the JVM world, Ron Pressler's first foray into fibers -quasar- was named "parallel universe". It worked with a java agent manipulating bytecode. Then Ron went to Oracle and now we have Loom, aka a virtual thread unmounted at each async IO request. Java's Loom is not even mentioned in the article. I won…

Quasar was awesome when it came out. Still remember trying to make it work when I was at Uber but it was really finicky.

Re: Asynchronous IO: the next billion-dollar mistake?

#46

Post by Someone » Sat, Sep 07, 2024, 7:49 AM UTC

FTA: “Need to call a C function that may block the calling thread? Just run it on a separate thread, instead of having to rely on some sort of mechanism provided by the IO runtime/language to deal with blocking C function calls.”

And then? How do you know when your call completed without “some sort of mechanism provided by the IO runtime/language”? Yes, you periodically ask the OS whether that thread completed, but that doesn’t come for free and is far from elegant.

There are solutions. The cheapest, resource-wise, are I/O completion callbacks. That’s what ”System” had on the original Mac in 1984, and there likely were even smaller systems before that had them.

Easier for programmers would be something like what we now have with async/await.

It might not be the best option, but AFAICT, this article doesn’t propose a better one. Yes, firing off threads is easy, but getting the parts together the moment they’re all available isn’t.

Re: Asynchronous IO: the next billion-dollar mistake?

#47

Post by ffsm8 » Sat, Sep 07, 2024, 7:49 AM UTC

dist1ll wrote: post #36

winternewt wrote: post #27

> Now imagine a parallel universe where instead of focusing on making asynchronous IO work, we focused on improving the performance of OS threads such that one can easily use hundreds of thousands of OS threads without negatively impacting performance I actually can't imagine how that would ever be accomplished at the OS level. The fact that each thread needs its own stack is an inherent limiter for efficiency, as sw…

> But at the programming language level the compiler does have insight into the dependencies of your continuation This is really the key point - coupled with the fact that certain I/O operations are just inherently asynchronous. The TX/RX queues in NICs are an async, message passing interface - regardless of whether you're polling descriptors or receiving completion interrupts. So really, async I/O is the natural abs…

> certain I/O operations are just inherently asynchronous.

That's technically not true.

The fact that its inherently async is an implementation detail. You either have blocking sync or non-blocking async. the implementation could be synchronous if the blocking didn't cause overhead and that was the proposed idea here - at least as far as I interpreted it.

Re: Asynchronous IO: the next billion-dollar mistake?

#48

Post by vbezhenar » Sat, Sep 07, 2024, 7:49 AM UTC

dist1ll wrote: post #36

winternewt wrote: post #27

> Now imagine a parallel universe where instead of focusing on making asynchronous IO work, we focused on improving the performance of OS threads such that one can easily use hundreds of thousands of OS threads without negatively impacting performance I actually can't imagine how that would ever be accomplished at the OS level. The fact that each thread needs its own stack is an inherent limiter for efficiency, as sw…

> But at the programming language level the compiler does have insight into the dependencies of your continuation This is really the key point - coupled with the fact that certain I/O operations are just inherently asynchronous. The TX/RX queues in NICs are an async, message passing interface - regardless of whether you're polling descriptors or receiving completion interrupts. So really, async I/O is the natural abs…

> So really, async I/O is the natural abstraction for networking.

Special pointer values are also natural abstraction.

Re: Asynchronous IO: the next billion-dollar mistake?

#49

Post by bjornsing » Sat, Sep 07, 2024, 7:52 AM UTC

haileys wrote: post #10

Asynchronous IO isn't about efficiency. The approach the author takes with their language is just threads, but scheduled in userland. This model allows a decoupling of the performance characteristics of runtime threads from OS threads - which can sometimes be beneficial - but essentially, the programming model is fundamentally still synchronous. Asynchronous programming with async/await is about revealing the time di…

> Take cancellation for example: cancelling tasks under the synchronous programming model requires passing a context object through every part of your code that might call down into an IO operation.

Does it? Wouldn’t you just kill the thread in the synchronous model?

Re: Asynchronous IO: the next billion-dollar mistake?

#50

Post by sedatk » Sat, Sep 07, 2024, 7:53 AM UTC

Many synchronous I/O operations under the hood are just async I/O + blocking waits, at least that's the case with Windows. Why? Because all I/O is inherently async. Even polling I/O requires timed waits which also makes it async.

That said, I like async programming model in general, not just for I/O. It makes modeling your software as separetely flowing operations that need to be synchronized occasionally quite easy. Some tasks need to run in parallel? Then, you just wait for them later.

I also like the channel concept of Golang and D in the same manner, but I heard it brought up some problems that async/await model didn't have. Can't remember what it was now. Maybe they are more susceptible to race conditions? Not sure.