20 years ago we had the world of threads and it was very easy to get into a mess.
Asynchronous IO: the next billion-dollar mistake?
31–40 of 165 posts
Re: Asynchronous IO: the next billion-dollar mistake?
#32Async/await is a language semantics thing. It's not really relevant whether there's a "real" OS thread under the hood, some language level green thread system, or just the current process blocking on something - the syntax exists because sometimes you don't want to block on things that take a long time semantically - I.e. you want the next line of code to run immediately. You could absolutely write a language where t…
That's something like what Go does. Goroutines are "green threads" - they can be preempted. There's a CPU scheduler in user space. Go tries to provide "async" performance, and goroutines have minimal state. This seems to work well for the web server case.
Pure "Async" means your application is now in the CPU dispatching business. This works well only if your application is totally I/O bound and has no substantial compute sections. Outside of that use case, it may be a huge mismatch to the problem. Worst case tends to be programs where almost all the time, something is fast, but sometimes it takes a lot of compute. Then all those quick async tasks get stuck behind the compute-bound operation. Web browsers struggle with that class of problems.
Async I/O tends to be over-used today because Javascript works that way. Many programmers came up from Javascript land. That's the only way they can conceive concurrency. It's a simple, clean model - no need for locks.
Threading is hard. Especially in languages that don't provide much help with locking. Even then, you have lock order problems, deadlocks, starvation, futex congestion...
Re: Asynchronous IO: the next billion-dollar mistake?
#33Earlier quoted context omitted.
> Asynchronous programming with async/await is about revealing the time dimension of execution as a first class concept People are more likely to assume their code is fast enough and not worry about the execution time of synchronous data processing, then spend weeks investigating why the p99 latency is 5 seconds with clusters of spikes. Async IO is almost entirely about efficiency. It's telling the OS that you can ma…
Are you sure that the developer is the best at determining these context switches? I mean, for a low-level language like rust, sure. But for higher level programming, e.g. some CRUD backend, should the developer really care about all that added complexity, when the runtime knows just as much, if not more. Like, it’s a DB call? Then just use the async primitive of the OS in the background and schedule another job in i…
It is. Until it isn't anymore.
Same as we used to do asm, but then the generated code is becoming good enough, or even better than hand written asm.
I predict the async trend will fade, as hardware and software will improve. And synchronous programming is higher level than using async. And higer level always prevail given enough time, as management always wants to hire the cheapest devs for the task.
Re: Asynchronous IO: the next billion-dollar mistake?
#34Asynchronous IO isn't about efficiency. The approach the author takes with their language is just threads, but scheduled in userland. This model allows a decoupling of the performance characteristics of runtime threads from OS threads - which can sometimes be beneficial - but essentially, the programming model is fundamentally still synchronous. Asynchronous programming with async/await is about revealing the time di…
One of the other aspects of this is that implementing a synchronous model on top of asynchronous primitives is absolutely trivial. You just wait until the asynchronous operation completes. Any program designed for asynchronous execution can be trivially retrofitted for synchronous execution. In contrast, implementing a asynchronous model on top of synchronous primitives is extremely challenging requiring the current…
> Furthermore, retrofitting a program using synchronous execution to use asynchronous execution is a tremendous amount of work as you need to do it bottom up to maintain compatibility during the transition.
Can attest to that. I was part of the team that rewrote Firefox to be fully async. Took us years and we could not maintain compatibility with XUL add-ons (async was not the only reason for this, but that's where the writing on the wall started to become visible).
Re: Asynchronous IO: the next billion-dollar mistake?
#35Some answer was already posted here:
https://utcc.utoronto.ca/~cks/space/blog/tech/OSThreadsAlway...
https://news.ycombinator.com/item?id=41472027
To me the article reads as if the programming language author wants to push a difficult problem out of his language without deeper analysis. As if it would be easier if it was somebody else's problem.
Re: Asynchronous IO: the next billion-dollar mistake?
#36> Now imagine a parallel universe where instead of focusing on making asynchronous IO work, we focused on improving the performance of OS threads such that one can easily use hundreds of thousands of OS threads without negatively impacting performance I actually can't imagine how that would ever be accomplished at the OS level. The fact that each thread needs its own stack is an inherent limiter for efficiency, as sw…
> But at the programming language level the compiler does have insight into the dependencies of your continuation
This is really the key point - coupled with the fact that certain I/O operations are just inherently asynchronous.
The TX/RX queues in NICs are an async, message passing interface - regardless of whether you're polling descriptors or receiving completion interrupts.
So really, async I/O is the natural abstraction for networking.
Re: Asynchronous IO: the next billion-dollar mistake?
#37There are fundamental reasons for OS threads being slow, mostly to do with processor design. Changing the silicon would be hundreds of times more expensive than solving the problem in software in user mode. This is a billion-dollar solution to a hundred-billion dollar problem.
There is a history of trying to support context switching in CPU hardware. That was a hot idea decades ago. Intel had call gates and some other stuff. Some of the RISC machines had hardware to spill all the registers to memory in background. Some early machines, from the days where CPUs were slower than memory, just had a context pointer in hardware. Change that and you're running a different thread.
None of this helped all that much. Vanilla hardware won out. This is to some extent a consequence of C winning. C likes a single flat address space.
Re: Asynchronous IO: the next billion-dollar mistake?
#38Asynchronous IO is just simply how the world works. Instead, the idea that changes happen only during CPU computation is the mistake. Your disk drive/network card exists in parallel to your CPU and can process stuff concurrently. Your CPU very likely has a DMA engine that works in parallel without consuming a hardware thread.
Re: Asynchronous IO: the next billion-dollar mistake?
#39Asynchronous IO isn't about efficiency. The approach the author takes with their language is just threads, but scheduled in userland. This model allows a decoupling of the performance characteristics of runtime threads from OS threads - which can sometimes be beneficial - but essentially, the programming model is fundamentally still synchronous. Asynchronous programming with async/await is about revealing the time di…
I'd argue that few usages of async are motivated this way. In Rust land, it's efficiency and in JS land it's the browser scripting language legacy. > cancelling tasks under the synchronous programming model requires passing a context object through every part of your code that might call down into an IO operation. This is true for some but not all implementations. See eg Erlang or Unix processes (and maybe cancellati…
To be more precise, in JS land, we introduced async not directly because of scripting but because of backwards compatibility - prior to async/Promise, the JS + DOM semantics were specified with a single thread of execution in mind, with complex dependencies in both directions (e.g. some DOM operations can cause sync reflow while handling an event, which is... bad) and run-to-completion.
Promise made it easier to:
- cut monolithic chunks of code-that-needs-to-be-executed-to-completion-before-updating-the-screen into something that didn't cause jank;
- introduce background I/O.
async/await made Promise more readable.
(yes, that's for Promise and async/await on the browser, async callbacks have a different history on Node)
Re: Asynchronous IO: the next billion-dollar mistake?
#40Async/await is a language semantics thing. It's not really relevant whether there's a "real" OS thread under the hood, some language level green thread system, or just the current process blocking on something - the syntax exists because sometimes you don't want to block on things that take a long time semantically - I.e. you want the next line of code to run immediately. You could absolutely write a language where t…
That's something like what Go does. Goroutines are "green threads" - they can be preempted. There's a CPU scheduler in user space. Go tries to provide "async" performance, and goroutines have minimal state. This seems to work well for the web server case. Pure "Async" means your application is now in the CPU dispatching business. This works well only if your application is totally I/O bound and has no substantial com…
> It's a simple, clean model - no need for locks.
Nit: You can very easily have race conditions in async JS. There are all sorts of Mutex-style structures for async.