Looks similar to apache worker MPM (release ~2005)
Thread Pools in Nginx Boost Performance 9x (2015)
11–20 of 62 posts
Re: Thread Pools in Nginx Boost Performance 9x (2015)
#12It would be interesting to compare this with coroutines/fibers, and it would also be interesting to know how latency is affected. In my experience, thread pools introduce latency.
Trick is that nginx uses thread pools for non-blocking reading on the "fast" devices - you can't use coroutines/fibers, because on Linux is not reliably possible to read the regular file in a non-blocking way, as far as I'm aware, so that means that `read` in a single fiber would block entire thread until the data is fetched from the disk. In the storage engine I'm developing/maintaining, I've been using fibers, for…
Why files can't be read without blocking with Linux? There's certainly an API for this.
Re: Thread Pools in Nginx Boost Performance 9x (2015)
#13As long as you're not using Apache in pre-fork mode then the rest boils down to a few percent in terms of http server overhead. Unless you're trying to serve tons of little static object with either no resource or big scale then there are better places to spend your time optimizing.
Re: Thread Pools in Nginx Boost Performance 9x (2015)
#14Earlier quoted context omitted.
Trick is that nginx uses thread pools for non-blocking reading on the "fast" devices - you can't use coroutines/fibers, because on Linux is not reliably possible to read the regular file in a non-blocking way, as far as I'm aware, so that means that `read` in a single fiber would block entire thread until the data is fetched from the disk. In the storage engine I'm developing/maintaining, I've been using fibers, for…
Why files can't be read without blocking with Linux? There's certainly an API for this.
> Why files can't be read without blocking with Linux? There's certainly an API for this.
i guess you are referring to aio(7) ? posix aio on linux is implemented in glibc, and doesn't scale in presence of multiple threads. which is probably what is alluded to...
Re: Thread Pools in Nginx Boost Performance 9x (2015)
#15Earlier quoted context omitted.
Trick is that nginx uses thread pools for non-blocking reading on the "fast" devices - you can't use coroutines/fibers, because on Linux is not reliably possible to read the regular file in a non-blocking way, as far as I'm aware, so that means that `read` in a single fiber would block entire thread until the data is fetched from the disk. In the storage engine I'm developing/maintaining, I've been using fibers, for…
Why files can't be read without blocking with Linux? There's certainly an API for this.
Which API are you referring to? If you're referring to `aio_` that's just glibc's user-space thread-pool emulation of the POSIX's asynchronous IO + many operations needed for the truly async IO are not supported (such as `stat` and `open`). There's `io_` syscalls family, but that's not in the ready-to-use state yet, judging by the manpages and various patches being submitted every month.
Re: Thread Pools in Nginx Boost Performance 9x (2015)
#16It would be interesting to compare this with coroutines/fibers, and it would also be interesting to know how latency is affected. In my experience, thread pools introduce latency.
Fibers and thread pools are not exclusive - you can schedule fibers onto threads. Nginx already uses event-based state-machine approach so context switch is even cheaper than with fibers (at the price of ugly code). What they talk about in the article is that they offload potentially blocking operations to thread pools. That's not about networking, which is already done asynchronously, but about e.g. reading from a f…
Why can Linux do async network IO but not async file IO? Seems like they would be similar. What's so special about file IO?
Re: Thread Pools in Nginx Boost Performance 9x (2015)
#17Earlier quoted context omitted.
Fibers and thread pools are not exclusive - you can schedule fibers onto threads. Nginx already uses event-based state-machine approach so context switch is even cheaper than with fibers (at the price of ugly code). What they talk about in the article is that they offload potentially blocking operations to thread pools. That's not about networking, which is already done asynchronously, but about e.g. reading from a f…
Why can Linux do async network IO but not async file IO? Seems like they would be similar. What's so special about file IO?
When non-blocking socket I/O was implemented the disk was considered to be too fast to warrant the same kind of treatment. At the API level the different is that sockets, pipes and ttys are have an additional "ready for use" state while files are always ready, because the kernel can do something to access them even if it means blocking the thread while the kernel waits for the disks to respond.
Re: Thread Pools in Nginx Boost Performance 9x (2015)
#18Earlier quoted context omitted.
Why can Linux do async network IO but not async file IO? Seems like they would be similar. What's so special about file IO?
When non-blocking socket I/O was implemented the disk was considered to be too fast to warrant the same kind of treatment. At the API level the different is that sockets, pipes and ttys are have an additional "ready for use" state while files are always ready, because the kernel can do something to access them even if it means blocking the thread while the kernel waits for the disks to respond.
Yes, the trick is that disk device is considered "fast", so it's not selectable - it's always ready to read/write, which is a lie. I _feel_ (and think, I don't have any data to prove this) the main problem here is the file system layer, which may or may not need to block _after_ the operation is performed, and this complexity doesn't occur with sockets/pipes.
Having a always-ready state on regular files is a problem since Linux's non-blocking is going around readiness. On Windows (and I believe on Solaris/FreeBSD), completeness model is in place, so you schedule an operation, kernel does _everything_ and then you're resumed upon completion of IO.
Re: Thread Pools in Nginx Boost Performance 9x (2015)
#19Earlier quoted context omitted.
When non-blocking socket I/O was implemented the disk was considered to be too fast to warrant the same kind of treatment. At the API level the different is that sockets, pipes and ttys are have an additional "ready for use" state while files are always ready, because the kernel can do something to access them even if it means blocking the thread while the kernel waits for the disks to respond.
Yes, the trick is that disk device is considered "fast", so it's not selectable - it's always ready to read/write, which is a lie. I _feel_ (and think, I don't have any data to prove this) the main problem here is the file system layer, which may or may not need to block _after_ the operation is performed, and this complexity doesn't occur with sockets/pipes. Having a always-ready state on regular files is a problem…
The most general layer in Linux (VFS) doesn't really support asynchronicity, so changing this would require a bunch of changes to every tree and out-of-tree FS - not viable. However, the usual suspects (ext, xfs) actually use a bunch of other APIs as well, where the FS itself is often not involved in simple stuff like a read(2). This theoretically would allow async IO on files in many cases; I believe there is ongoing work in this direction, though. For now, only raw / O_DIRECT is async.
Re: Thread Pools in Nginx Boost Performance 9x (2015)
#20As long as you're not using Apache in pre-fork mode then the rest boils down to a few percent in terms of http server overhead. Unless you're trying to serve tons of little static object with either no resource or big scale then there are better places to spend your time optimizing.
and even then it only matters if you don't have something in front of it handling the connection-pooling/spoon-feeding/static-stuff like a load balancer or a cdn or varnish or pretty much anything.