One thread per CPU, and non-blocking I/O, that's sounds like the usual way to approach the problem. I'm surprised it uses state machines to handle the non-blocking I/O, because modern software engineering provides much more pleasant approaches such as using coroutines.
How We Designed for Performance and Scale
21–30 of 63 posts
Re: How We Designed for Performance and Scale
#22"NGINX’s binary upgrade process achieves the holy grail of high-availability; you can upgrade the software on the fly, without any dropped connections, downtime or interruption in service." Is this really true? I remember seeing an article[1] recently on using an iptables hack to prevent dropping connections when reloading haproxy. Does nginx actually provide zero-downtime configuration reloads? [1] https://medium.co…
Yes, it's true. NGINX does zero-downtime configuration reloads and binary upgrades since 2004 without any dirty hacks.
Re: How We Designed for Performance and Scale
#23"NGINX’s binary upgrade process achieves the holy grail of high-availability; you can upgrade the software on the fly, without any dropped connections, downtime or interruption in service." Is this really true? I remember seeing an article[1] recently on using an iptables hack to prevent dropping connections when reloading haproxy. Does nginx actually provide zero-downtime configuration reloads? [1] https://medium.co…
The irc client irssi has something like this in /upgrade where it spawns a new binary but passing along active socket connections and their associated irc room states, I believe
WeeChat definitely does this. I don't know whether irssi supports it as well. Too bad WeeChat can't keep TLS connections open on /upgrade though
Re: How We Designed for Performance and Scale
#24One thread per CPU, and non-blocking I/O, that's sounds like the usual way to approach the problem. I'm surprised it uses state machines to handle the non-blocking I/O, because modern software engineering provides much more pleasant approaches such as using coroutines.
Coroutines have the distinct disadvantage of needing a stack, much like threads. So-called 'stackless' coroutines aren't really so different to computed gotos in a state machine
Re: How We Designed for Performance and Scale
#25> You can reload configuration multiple times per second (and many NGINX users do exactly that) I thought this was an interesting remark. Can anyone clue me in to what these "many users" might be doing, that requires them to reload configuration so frequently?
I front my docker fleet with Nginx and I need to reload config for new hosts
Re: How We Designed for Performance and Scale
#26This is a lovely article, but: > The fundamental basis of any Unix application is the thread or process. (From the Linux OS perspective, threads and processes are mostly identical; the major difference is the degree to which they share memory.) It's better to be specific in performance discussions, rather than use 'thread' and 'process' interchangeably. As well as the article mentioned about memory sharing, threads (…
In Linux, they are both just entries in the process table. They are created by the `clone` syscall, and the "normal" way of creating them share different amounts of resources by _default_.
You're right to say that treating them differently can be beneficial in some situations, but it really depends.
Re: How We Designed for Performance and Scale
#27Earlier quoted context omitted.
If it's the holy grail, then erlang has had it for a couple decades... and whether it actually works in nginx I don't know, but it does work in erlang. The way it's handled is that the process where the previous socket was connected remains in place until it terminates but all new sockets connect to the new code.
"The way it's handled is that the process where the previous socket was connected remains in place until it terminates but all new sockets connect to the new code." That would imply that two separate processes are bound to the same port right? I thought that was not possible.
While there are certain socket options to share a port - two processes can share a port easily in another elegant way; just call fork().
One way of doing such upgrades is then for the existing code to fork() then exec() the new version. The existing old code closes the original server socket, which leaves accepting new requests to the new code. When all in-flight requests served by the old code is finished, that process can exit.
Re: How We Designed for Performance and Scale
#28Re: How We Designed for Performance and Scale
#29I did as they said at the bottom and gave them my e-mail and other personal details so I could download the eBook that they were giving free preview copies of - "Building Microservices". Unfortunately, they sent link to PDF only so it's not usable to me. Just a heads up to others so you save yourself the time of discovering that. (I'll just wait for when the book is finished and then I'll buy it so I get ePub. I like O'Reilly and have bought many books there before.)
Re: How We Designed for Performance and Scale
#30One thread per CPU, and non-blocking I/O, that's sounds like the usual way to approach the problem. I'm surprised it uses state machines to handle the non-blocking I/O, because modern software engineering provides much more pleasant approaches such as using coroutines.
Coroutines have the distinct disadvantage of needing a stack, much like threads. So-called 'stackless' coroutines aren't really so different to computed gotos in a state machine
For parsing html, you wouldn't need that large a stack. Further, you could make the stack grow dynamically at a very small cost. Also, I bet that even if stackless coroutines are close to FSMs, they are much easier to program.