41–50 of 62 posts
The moral is that threads and asynchronous programming are not mutually exclusive. No matter how much you can do with a single thread, in a highly parallel environment (such as a web server), on a cpu with multiple cores (the norm), you can (almost always) do more with multiple threads.
Sure for threads up to cores / 2. Past that it is likely a throughput and latency loss.
The moral is that threads and asynchronous programming are not mutually exclusive. No matter how much you can do with a single thread, in a highly parallel environment (such as a web server), on a cpu with multiple cores (the norm), you can (almost always) do more with multiple threads.
Sure for threads up to cores / 2. Past that it is likely a throughput and latency loss.
If these are "worker threads", new workers are scheduled when all other workers are blocked and work is available. This guarantees that idle CPU will always be scheduled with work. When workers unblock, they continue execution on their work and start picking up new work. The worker scheduler will stop scheduling new workers and the extra workers will go back to the pool of inactive workers. This is the idea behind a worker or thread pool, they get scheduled when its possible to do more work.
A poor implementation of this idea will just hand work off to threads and then expect them to all execute in parallel, which will lead to negative performance impact.
The problem with (some implementations of) async workers is that just scheduling more will result in the latter problem, work split across too many workers. This is especially true for workers implemented as entire processes. The problem this post brings up is that 1 to 1 mapping of workers to cores suffers from idle CPU due to unanticipated blocking. The general solution is to smartly distribute work over a pool of workers, and not blindly schedule a fixed number workers.
Earlier quoted context omitted.
Sure for threads up to cores / 2. Past that it is likely a throughput and latency loss.
Does anyone have numbers to back this claim? Seems like for IO focused software you'll have a lot of dead CPU time. If you have more threads than cores, you can take advantage of that.
There is no performance benefit to adding more threads, they would increase CPU state/cache thrashing which will slow things down. The main benefit of using a lot of threads is that more programmers know how to design software that way because it was taught for so many years; it isn't good for performance on modern hardware.
Earlier quoted context omitted.
Trick is that nginx uses thread pools for non-blocking reading on the "fast" devices - you can't use coroutines/fibers, because on Linux is not reliably possible to read the regular file in a non-blocking way, as far as I'm aware, so that means that `read` in a single fiber would block entire thread until the data is fetched from the disk. In the storage engine I'm developing/maintaining, I've been using fibers, for…
Why files can't be read without blocking with Linux? There's certainly an API for this.
Earlier quoted context omitted.
Sure for threads up to cores / 2. Past that it is likely a throughput and latency loss.
Does anyone have numbers to back this claim? Seems like for IO focused software you'll have a lot of dead CPU time. If you have more threads than cores, you can take advantage of that.
Yes, see here for more discussion on this:
https://github.com/ronomon/crypto-async#adjust-threadpool-si...
The moral is that threads and asynchronous programming are not mutually exclusive. No matter how much you can do with a single thread, in a highly parallel environment (such as a web server), on a cpu with multiple cores (the norm), you can (almost always) do more with multiple threads.
Sure for threads up to cores / 2. Past that it is likely a throughput and latency loss.
Looks similar to apache worker MPM (release ~2005)
Worker does use process pools with their own thread pools, but they're still not evented/async.
Apache 2.4 (2012) saw the release of 'event', an async variant of worker, which is like the new nginx model.