Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Boosting Nginx Performance with Thread Pools

nginx.com

Post reply on HN
Print view View original

11–20 of 48 posts

Re: Boosting Nginx Performance with Thread Pools

#11

Post by nitrogen »

istvan__ wrote: post #6

I don't think that a load of 172 is a good idea. I know this is a benchmark that is measuring how fast you can go ideally but in production the question is how fast you can go with with keeping the latency within the SLA. As a general rule you want to run your boxes around 1 normalized load ( load / # of CPU cores). The rest of the article is pretty nice.

Having load average exceed core count isn't necessarily a sign of an imminent cascade failure. IIRC it's just a count of the number of processes that could run, but are waiting for a timeslice. A particular server and application might be perfectly fine with a load average that is 10 times the number of cores, as long as the average remains stable and the server is meeting response requirements.

Re: Boosting Nginx Performance with Thread Pools

#13

Post by daurnimator »

dschiptsov wrote: post #9

Linux has POSIX aio syscalls which seems to work. At least Informix and Oracle rely on them.

> Linux has POSIX aio syscalls which seems to work. At least Informix and Oracle rely on them.

That only works for read, write and fsync.

There are plenty of other blocking syscalls that people need to use.

e.g. An important missing one is getdents.

Re: Boosting Nginx Performance with Thread Pools

#14

Post by dschiptsov »

SethMurphy wrote: post #2

Great explanation of an event driven web server. Helped me understand ome of the benefits of the mongrel2 architecture that separates the tasks needed to be done completely by using ØMQ as the mechanism to decouple the connection handling from the message handling of the request.

While asynchronous messaging is generally a good idea, using a messaging middleware seem to be an overkill. One should use smallest hammer for the job.

Thread is a wrong idea in the first place - by broking isolation of processes (share nothing principle) they brought in the whole new class of problems with locking and synchronization. Only threads that share nothing is a reasonable choice, but without sharing the whole concept makes no sense anymore. So there are kernel lightweight processes which seems to be good choice for offloading the blocking operations from the main loop.

BTW, Erlang does it right from the very beginning.)

Re: Boosting Nginx Performance with Thread Pools

#15

Post by caf »

istvan__ wrote: post #6

I don't think that a load of 172 is a good idea. I know this is a benchmark that is measuring how fast you can go ideally but in production the question is how fast you can go with with keeping the latency within the SLA. As a general rule you want to run your boxes around 1 normalized load ( load / # of CPU cores). The rest of the article is pretty nice.

Remember that processes in (D)isk Wait state count towards the load average even though they are not running or runnable.

Re: Boosting Nginx Performance with Thread Pools

#16

Post by rgbrenner »

misframer wrote: post #8
smegel wrote: post #5

I think this would have been much better titled "Boosting NGINX Performance 9x with Asynchronous wrappers around blocking system calls". Most people when hearing about "thread pools" in the context of a web-server think about using multiple threads for handling separate requests, which is NOT what this is about. It is using threads to enable some blocking syscalls (read and sendfile) to run asynchronously from the ma…

Similarly, there's libuv [0], which is used for Node.js and others. [0] https://github.com/libuv/libuv

libuv is for C projects that miss the joy of javascript callback hell.

Re: Boosting Nginx Performance with Thread Pools

#17

Post by caf »

A slight note on the terminology - reads and writes of ordinary disk files technically do not "block"; they "Disk Wait" instead. The difference is visible for example in that ordinary files are always considered by select()/poll() as readable and writeable.

Re: Boosting Nginx Performance with Thread Pools

#18

Post by anacrolix »

rgbrenner wrote: post #16
misframer wrote: post #8

Earlier quoted context omitted.

Similarly, there's libuv [0], which is used for Node.js and others. [0] https://github.com/libuv/libuv

libuv is for C projects that miss the joy of javascript callback hell.

Or don't want to have 10000 threads. I have a Go server that regularly bumps 140k goroutines. Try that shit with native threads.

Re: Boosting Nginx Performance with Thread Pools

#19

Post by phs2501 »

dschiptsov wrote: post #9

Linux has POSIX aio syscalls which seems to work. At least Informix and Oracle rely on them.

Linux kernel aio will often still block when dealing with the page cache even if you request nonblocking. The workaround for this is to use O_DIRECT, which is okay for databases that do their own cache management but not for something like nginx (which is depending on the OS cache).

Glibc's posix aio (aio_*(3)), on the other hand, does not use Linux's kernel aio AFAIK. It probably uses thread pools. It also uses signals to signal completion. It is not generally considered performant.

Re: Boosting Nginx Performance with Thread Pools

#20

Post by rgbrenner »

anacrolix wrote: post #18
rgbrenner wrote: post #16

Earlier quoted context omitted.

libuv is for C projects that miss the joy of javascript callback hell.

Or don't want to have 10000 threads. I have a Go server that regularly bumps 140k goroutines. Try that shit with native threads.

libuv isn't unique.. It's equivalent to libev + libeio.. in fact, that's what nodejs used before writing libuv. Whether or not it's faster than those is really case-by-case.. but what you'll definitely get with libuv is callbacks everywhere.
Post reply on HN