Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Asynchronous IO: the next billion-dollar mistake?

yorickpeterse.com

Post reply on HN
Print view View original

81–90 of 165 posts

Re: Asynchronous IO: the next billion-dollar mistake?

#81

Post by dudeinjapan »

pizza wrote: post #43

Correct me if I'm wrong, but Microsoft's DirectStorage seems to me something like what the author is writing about. It lets you do eg massively parallel NVME file io ops from the GPU itself of lots of small files. This avoids the delay of the path through the CPU, any extra threads/saturation of the CPU, and even lets you do eg decompression of game assets on the GPU itself thereby saving even more CPU. This demo ben…

I interpreted it as mainly network I/O, but the core point in the article is less about the I/O itself and more about thread-based async I/O handling.

Re: Asynchronous IO: the next billion-dollar mistake?

#82

Post by ffsm8 »

Veserv wrote: post #54
ffsm8 wrote: post #47

Earlier quoted context omitted.

> certain I/O operations are just inherently asynchronous. That's technically not true. The fact that its inherently async is an implementation detail. You either have blocking sync or non-blocking async. the implementation could be synchronous if the blocking didn't cause overhead and that was the proposed idea here - at least as far as I interpreted it.

No, the hardware is frequently inherently asynchronous. You write some memory and then the hardware consumes the prepared data asynchronously, in parallel, until it informs you in some manner that the operation is complete (usually either a asynchronous interrupt, or asynchronous write to a location you are polling). You can do whatever you want after preparing the data without waiting for completion. That is a inher…

While that is technically true, it's also missing the point entirely. That's why I said you can have either blocking sync or non-blocking async.

The article explicitly talks about the API provided by the OS. As a matter of fact, they're even more specific talking about spawning os threads vs async non-blocking file access.

At this level, the async is an implementation detail.

I guess your comment confirms that you didn't read the article, and neither did the people down voting me.

As usual on HN. lots of people suffering from the Dunning Kruger complex

Re: Asynchronous IO: the next billion-dollar mistake?

#83

Post by seanhunter »

weinzierl wrote: post #62

"Not every IO operation can be performed asynchronously though. File IO is perhaps the best example of this (at least on Linux). To handle such cases, languages must provide some sort of alternative strategy such as performing the work in a dedicated pool of OS threads." Can someone explain, why this would be the case? - Why can't every IO op be async? - Why is file IO on Linux not async? - What does iouring have to…

File IO can absolutely be async on linux[1] and this has been supported since version 2.5 or something provided that the device supports it (which they all have since about 2000). io_uring was the syscall interface that was introduced in 5.1 to improve async file i/o performance so it is relevant in that the previous way to do it wasn't great.

[1] https://kkourt.io/blog/2017/10-14-linux-aio.html <- note the 2017 date.

Re: Asynchronous IO: the next billion-dollar mistake?

#84

Post by aenis »

thesnide wrote: post #33
kaba0 wrote: post #25

Earlier quoted context omitted.

Are you sure that the developer is the best at determining these context switches? I mean, for a low-level language like rust, sure. But for higher level programming, e.g. some CRUD backend, should the developer really care about all that added complexity, when the runtime knows just as much, if not more. Like, it’s a DB call? Then just use the async primitive of the OS in the background and schedule another job in i…

It is. Until it isn't anymore. Same as we used to do asm, but then the generated code is becoming good enough, or even better than hand written asm. I predict the async trend will fade, as hardware and software will improve. And synchronous programming is higher level than using async. And higer level always prevail given enough time, as management always wants to hire the cheapest devs for the task.

Not sure why the downvotes. Async programming is harder than sync, as one needs not only know one's code, but also all the dependencies. Since the benefits of async are in many scenarios limited[1] I'd expect the simpler abstractions to win.

I am a CTO at a large company and I routinely experience tech leads who dont understand what happens under the hood of an async event loop and act surprised when weird and hard to debug p99 issues occur in prod (because some obscure dependency of a dependench does sync file access for something trivial).

Abstractions win, people are lazy and most new developers lack understanding of lower level concepts such as interrupts or pooling, nor can predict what code may be cpu bound and unsafe in a async codebase. In a few years, explicit aync will be seen as C is seen today - a low level skill.

[1] if your service handles, say, 500qps on average the difference between async and threaded sync might be just 1-2 extra nodes. Does not register on the infra spend.

Re: Asynchronous IO: the next billion-dollar mistake?

#85

Post by anonymoushn »

haileys wrote: post #10

Asynchronous IO isn't about efficiency. The approach the author takes with their language is just threads, but scheduled in userland. This model allows a decoupling of the performance characteristics of runtime threads from OS threads - which can sometimes be beneficial - but essentially, the programming model is fundamentally still synchronous. Asynchronous programming with async/await is about revealing the time di…

Cancellation isn't possible in general. For example, if you've kicked off expensive work on another thread or passed a pointer to your future to the kernel via io_uring, you must add a layer of indirection that's quite similar to a cancelation context. You can't just accept that the expensive work you don't care about will keep happening or resume a future that has been dropped when the cqe entry bearing a pointer to it arrives. The cancellation facility provided by io_uring that guarantees that you won't receive such a thing does so by blocking your thread for a while, which is undesirable.

As implemented, async/await typically greatly harms composability. For example see here: https://nullderef.com/blog/rust-async-sync/

In the specific case of Rust, we might begin to be able to write composable libraries a couple decades from now: https://github.com/rust-lang/keyword-generics-initiative

Re: Asynchronous IO: the next billion-dollar mistake?

#86

Post by mmis1000 »

ayewo wrote: post #51
Yoric wrote: post #40

Earlier quoted context omitted.

> It's a simple, clean model - no need for locks. Nit: You can very easily have race conditions in async JS. There are all sorts of Mutex-style structures for async.

That's really interesting. Care to share a link to 1 or 2 real world examples of this that you've seen? Or even better, examples of how one would write such locks in JS that would be effective against these type of race conditions?

Having race condition in js is more about having questionable programming practice though.

Instead of write result of operations into separate variables and aggregate them later. You write them into the same variable with unspecified order and prey they will work correctly. There won't be memory corruption or something. But the results you got won't be correct either.

This type of problems is probably what rust try to address. (Rust will probably tell you to fxxk off because the write permission shouldn't be grant by two place at same time) But unfortunately there isn't rust for js. So only thing you can do is take care of it yourself.

Re: Asynchronous IO: the next billion-dollar mistake?

#87

Post by dfgdfg34545456 »

The post seems to be assuming that multi threaded code is easy to build and maintain. From my experience it is horrible, every new thread means going from n bugs to nn bugs. As a programmer I prefer* async constructs in languages, and do not want to spin up and manage threads and all the state synchronisation that involves.

Re: Asynchronous IO: the next billion-dollar mistake?

#88

Post by algobro »

The 1990s called. They want their threads vs events debates back.

Todays processors are fast enough to serve many useful workloads with a single core. The benefit of the async abstraction outweighs the performance benefit in the majority of the cases.

And debugging multithreaded code is way harder than async code, mainly if its the kind of program that needs stepping into.

Re: Asynchronous IO: the next billion-dollar mistake?

#89

Post by spoiler »

ffsm8 wrote: post #82
Veserv wrote: post #54

Earlier quoted context omitted.

No, the hardware is frequently inherently asynchronous. You write some memory and then the hardware consumes the prepared data asynchronously, in parallel, until it informs you in some manner that the operation is complete (usually either a asynchronous interrupt, or asynchronous write to a location you are polling). You can do whatever you want after preparing the data without waiting for completion. That is a inher…

While that is technically true, it's also missing the point entirely. That's why I said you can have either blocking sync or non-blocking async. The article explicitly talks about the API provided by the OS. As a matter of fact, they're even more specific talking about spawning os threads vs async non-blocking file access. At this level, the async is an implementation detail. I guess your comment confirms that you di…

I guess anything is an implementation detail depending on what level we're talking about, but each layer is constrained by the "previous" layer in some ways, and we're working within those constraints.

Sometimes we have the luxury of not caring about this, and we can let the language runtime pick what it wants to do, and abstract it away for us. But sometimes you need to care, because you're constrained by the existing environment/conditions the code runs in, and need specific control over eg like timings.

I think in an ideal world where we can start from scratch and aren't constrained by existing tech stacks/layers/hardware, maybe the OS and compilers (and runtimes) could integrate more tightly when it comes to threading, stack, and memory management. It would be a radically different architecture, though (and I dunno which layers we'd need to invalidate).

Re: Asynchronous IO: the next billion-dollar mistake?

#90

Post by pizza »

dudeinjapan wrote: post #81
pizza wrote: post #43

Correct me if I'm wrong, but Microsoft's DirectStorage seems to me something like what the author is writing about. It lets you do eg massively parallel NVME file io ops from the GPU itself of lots of small files. This avoids the delay of the path through the CPU, any extra threads/saturation of the CPU, and even lets you do eg decompression of game assets on the GPU itself thereby saving even more CPU. This demo ben…

I interpreted it as mainly network I/O, but the core point in the article is less about the I/O itself and more about thread-based async I/O handling.

I see, I had this notion that DirectStorage worked by somehow allowing the GPU cores to host multiple parallel threads each doing full requests on their own, but on further research I was mistaken - requests are still CPU-submitted
Post reply on HN