Asynchronous IO: the next billion-dollar mistake? • Hacker News Board

Re: Asynchronous IO: the next billion-dollar mistake?

#141

Post by yencabulator » Sat, Sep 07, 2024, 5:41 PM UTC

BenoitP wrote: post #74

josefrichter wrote: post #28

Isn’t that parallel universe actually the Erlang BEAM?

Or Go. Or Java' virtual threads. It must happen at the language level; When it comes to execution context knowledge: what context to compile out (stackless), or what context to serialize to the heap (stackful). Programming language will always know much more about the program than the OS. If I'm not mistaken in Erlang the programmer will provide the exact context to serialize : the actor.

This abstraction that a heap is somehow different than stack is hurting understanding; they're just regions of memory.

Go stacks are allocated areas of memory like anything else. When a green thread is suspended, it's stack is not "serialized to the heap". Switching green threads is mostly a question of setting the stack pointer to point to the new green thread's stack.

What makes threads "green" is that they are not supposed to call blocking syscalls (they can queue work in a separate blocking threadpool and call the scheduler to suspend themselves). Go has a signal-based mechanism for non-cooperative scheduling of green threads, so green threads are not even required to cooperatively yield to the scheduler.

Re: Asynchronous IO: the next billion-dollar mistake?

#142

Post by yencabulator » Sat, Sep 07, 2024, 5:46 PM UTC

weinzierl wrote: post #62

"Not every IO operation can be performed asynchronously though. File IO is perhaps the best example of this (at least on Linux). To handle such cases, languages must provide some sort of alternative strategy such as performing the work in a dedicated pool of OS threads." Can someone explain, why this would be the case? - Why can't every IO op be async? - Why is file IO on Linux not async? - What does iouring have to…

In-kernel file or block I/O is async if the driver/file implemented the async operations. If not, it's blocking.

io_uring is a relatively new userspace syscall API to minimize the number of syscalls used for I/O, and to provide a common mechanism for doing many kinds of operations. Previously, large numbers of syscalls could become a bottleneck for busy applications, and some kinds of operations did not have a usable non-blocking interface.

Re: Asynchronous IO: the next billion-dollar mistake?

#143

Post by yencabulator » Sat, Sep 07, 2024, 5:53 PM UTC

Animats wrote: post #37

jiggawatts wrote: post #4

There are fundamental reasons for OS threads being slow, mostly to do with processor design. Changing the silicon would be hundreds of times more expensive than solving the problem in software in user mode. This is a billion-dollar solution to a hundred-billion dollar problem.

There is a history of trying to support context switching in CPU hardware. That was a hot idea decades ago. Intel had call gates and some other stuff. Some of the RISC machines had hardware to spill all the registers to memory in background. Some early machines, from the days where CPUs were slower than memory, just had a context pointer in hardware. Change that and you're running a different thread. None of this hel…

One interesting angle is Mill (note, vaporware). Their "portal" calls between security boundaries are essentially the same as their EBB function calls.

https://www.youtube.com/watch?v=XJasE5aOHSw

Re: Asynchronous IO: the next billion-dollar mistake?

#144

Post by nullindividual » Sat, Sep 07, 2024, 6:25 PM UTC

wtallis wrote: post #20

nullindividual wrote: post #6

We do live in the universe of high performance threads with asynchronous I/O. The author is looking for Windows NT.

The same Windows NT whose decades-old async IO capabilities are so good they immediately cloned io_uring?

Yes, it is nearly a 1:1 copy of io_uring, but unlike io_uring which applies to a single function, all I/O in the NT kernel is asynchronous. IOCP acts on file, network, mail slot, pipes, etc.

IoRing/io_uring is for files only. RegisteredIO is network only.

While Windows userland itself may be a 'mess' and archaic in it's own way (among the other anti-consumer bits), the NT kernel is technically quite advanced, not only for it's time, but at the present time.

David Cutler's team got the kernel architecture correct the first time. Linux still has a ways to catch up in certain aspects.

Re: Asynchronous IO: the next billion-dollar mistake?

#145

Post by OutOfHere » Sat, Sep 07, 2024, 6:42 PM UTC

> Languages such as Go and Erlang bake support for asynchronous IO directly into the language, while others such as Rust rely on third-party libraries such as Tokio.

This is so wrong. Go and Erlang have message passing, not async. Message passing is its own thing; it should not be mixed with threading or async.

Re: Asynchronous IO: the next billion-dollar mistake?

#146

Post by Animats » Sat, Sep 07, 2024, 7:38 PM UTC

harry8 wrote: post #99

Animats wrote: post #37

Earlier quoted context omitted.

There is a history of trying to support context switching in CPU hardware. That was a hot idea decades ago. Intel had call gates and some other stuff. Some of the RISC machines had hardware to spill all the registers to memory in background. Some early machines, from the days where CPUs were slower than memory, just had a context pointer in hardware. Change that and you're running a different thread. None of this hel…

> a consequence of C winning. C likes a single flat address space. Interesting. What other language did C beat? How was their address space handling different?

C allows casting to (void *). That implies all pointers are in the same address space. Segmented architectures, where that's not true, have mostly died out. Burroughs used to build segmented address space machines where an address was a path, something like programname/scopename/variablename[subscript]. IBM's System 38 had an "object" architecture. Both worked fine, but were not very C-compatible.

Re: Asynchronous IO: the next billion-dollar mistake?

#147

Post by torginus » Sat, Sep 07, 2024, 7:52 PM UTC

yencabulator wrote: post #137

torginus wrote: post #38

Asynchronous IO is just simply how the world works. Instead, the idea that changes happen only during CPU computation is the mistake. Your disk drive/network card exists in parallel to your CPU and can process stuff concurrently. Your CPU very likely has a DMA engine that works in parallel without consuming a hardware thread.

The "world" these days works by ringbuffers and interrupts, neither of which looks like async/await. Async/await is a chosen abstraction over what hardware/kernel interfaces really look like.

Well, async/await in languages is just promises/futures made to look like regular flow control via state machines. And promises are controlled via callbacks, which are called from some event.

It's essentially the same thing as when you get an interrupt from your microcontroller's DMA engine that the ring buffer needs to be refilled, just with 2 layers of abstractions on top to make it look nicer.

Re: Asynchronous IO: the next billion-dollar mistake?

#148

Post by quercus » Sat, Sep 07, 2024, 8:28 PM UTC

I thought this article was going to talk about async IO's potential to cause insidious bugs that you don't notice until they're crashing jet planes. That's definitely a topic worth of a blog post.

Re: Asynchronous IO: the next billion-dollar mistake?

#149

Post by ninkendo » Sat, Sep 07, 2024, 8:41 PM UTC

yencabulator wrote: post #138

YorickPeterse wrote: post #129

Earlier quoted context omitted.

That particular paragraph can be phrased better so I'll adjust that. What I meant to say is that you can't handle it asynchronously like you can with sockets, i.e. polling it for readiness. This is because for file IO, reads and writes are always reported as being available, making epoll/kqueue/etc effectively useless.

With io_uring, you don't poll a file descriptor for readiness and then attempt a non-blocking operation and hope it works. You submit a request and later get a response.

To be fair to the author, io_uring has not made its way very far into userspace libraries yet. Even if the language you use has async primitives, and it’s on paper a great fit for io_uring, there’s a slim to zero chance the underlying libraries you’re using are actually using it yet. (Unless you’ve gone out of your way to target io_uring and write your own code for it.)

Re: Asynchronous IO: the next billion-dollar mistake?

#150

Post by yencabulator » Sat, Sep 07, 2024, 8:49 PM UTC

ninkendo wrote: post #149

yencabulator wrote: post #138

Earlier quoted context omitted.

With io_uring, you don't poll a file descriptor for readiness and then attempt a non-blocking operation and hope it works. You submit a request and later get a response.

To be fair to the author, io_uring has not made its way very far into userspace libraries yet. Even if the language you use has async primitives, and it’s on paper a great fit for io_uring, there’s a slim to zero chance the underlying libraries you’re using are actually using it yet. (Unless you’ve gone out of your way to target io_uring and write your own code for it.)

Yeah, it's a huge challenge. A big part of the overhead of read(2) is that historically each read required a buffer allocated to it before you start the slow request, and the way languages implemented non-blocking I/O followed that API. io_uring avoids the problem with buffer pools, but that completely changes the I/O API that most languages have settled on.