Async might be a fad

I don't write web applications, and I don't use much JavaScript, so it's very possible that I don't properly understand the motivation for this blog post. However, as others have said there is a difference between some inconvenient syntax in JS, and the fundamental model of non-blocking I/O.

What I do write is lots of C/C++ client/server applications for HPC/HFT/DC workloads where speed both in req/sec (throughput) and speed in min/avg/max(secs/req) (latency) matters. In these environments I almost exclusively use non-blocking I/O. There are several reasons:

1) Threads are not free. Even if you use a thread-pool to avoid spin up costs, context switching overhead matters. Every time you call blocking I/O, you make sure that the kernel will wake up, schedule another thread, and do anything else that it decides to do. Waste time that you could have used to do useful work. Non-blocking I/O puts you in charge of your own "thread scheduler". Your "threads" are functions, they are "cooperatively scheduled" and you can make full use of every cycle that you get.

2) Programming with threads is hard. Trust me. If you think it's easy, or I'm soft, you haven't done it enough. At some point you will need shared state across those threads. And then you'll need locking and unlocking. (also Mutexs are slooooowww) And then you'll need to handle error cases, and you'll need to make sure that all the unlocking is done right in all of the right places. And then you'll need signaling between your threads. And you'll need semaphores or similar. And 3 months down the line, you're thinking to yourself, when a foo exception causes a bar signal, will a baz handler deadlock? Will it make progress? Humans just aren't designed to reason about this sort of thing.

With a single threaded, non-blocking design, it's really easy to reason about exactly what is happening with all of your state. Debugging is obvious and straightforward. This is necessary if you're like me and don't write perfect code first time. There's only ever one function accessing shared state at one time. The "scheduler" is working for you, not against you. If you write your code simply, cleanly and efficiently, you'd be amazed how much work a modern CPU can really do. Honestly, once you've saturated a 10G NIC what more do you want to do?

3) If you buy into the non-blocking design, then, as long as you only use 1 process/thread per core, almost anything a thread can do, a process can do better. Threads have no memory protection, anything you touch probably belongs to some other thread and you're inviting subtle bugs. Processes have memory protection by default if you want to share things you can do it explicitly via safe mechanisms (shared memory rings, pipes, IPC etc). Shared memory rings are (can be) so fast that data is more or less local so if you want to use shared state from a TCP connection or whatever, you can always "dispatch" work to another process to do it for you. You get the benefits of many cores working for you as well as a clean and obvious programming model.

Ultimately, if the question is one of syntax, then I'd happily believe that JS has some ugly syntax for doing these things, but if the question is one of design, then you should think really really hard before deciding that a threaded model is the correct one for you.

I don't think you've seen truly great async support, it's virtually indistinguishable from sync code.

Sync:

  let file = File.Open("foo.txt")
  let data = file.Read(8192)
  // Do some compute stuff with data

ASync:

  let! file = File.OpenAsync("foo.txt")
  let! data = file.ReadAsync(8192)  
  // Do some compute stuff with data

Hmm. I don't think I made myself very clear. I failed to mention that I'm advocating 1 process per core, not hundreds (thousands) of processes. Over subscribing processes to cores has the same effect of oversubscribing threads to cores, which is suboptimal scheduling. Furthermore, this model allows you to easily pin work to cores (using process affinity) and to get all the juicy benefits of using cooperative scheduling inside your processes.

I think the view that processes are "heavy weight" is a dated one. From an OS point of view, processes are pretty much the same amount of "work" as thread. Each has a context, each needs to be scheduled. Processes do have some extra state (notably the TLB context) but modern machines are very good switching these. Spinning up processes is somewhat more expensive than spinning up threads, but you really shouldn't be doing either on the critical path.

I agree that everything is context specific, although my main application area is really low latency scenarios (handfuls of microseconds) and techniques that work well there tend to port well to slower situations pretty easily (at least in my experience).

I disagree that encapsulation and non-overlapping objects will save you from threading nightmares. Every design starts out with clean boundaries and beautiful abstractions. Every design ends up in spaghetti soup. It's just a matter of how long it takes to get there.

I apologize for the sensational and generalizing title. What I'm talking about is focused on web applications, and from empirical data, it seems that most web servers handle very few concurrent requests, therefore it would be silly to go all async to avoid threads.

I'm very surprised that many people here argue that async code is much better to understand than sync code. Ok, so that part is subjective, and let's file it under personal preference. For people who love synchronous coding but fear the cost of threads, I'm trying to make an argument that the fear is probably not justified.

Thanks for the very considered response. I'm pretty interested in this because some of my recent work has been about designing I/O APIs/abstractions.

My reaction is due to my experience which is that threaded programming is something that's very hard to get right and especially to maintain. Async programming cleans up the threading and makes it kind of implicitly cooperative.

I was involved in a big move of some core infrastructure from a multi-threaded design over to a pipeline of async style apps. The result was a huge boost in productivity and debugability which worked out really well for the company.