Live data from Hacker News

How We Designed for Performance and Scale

nginx.com

41–50 of 63 posts

Re: How We Designed for Performance and Scale

#41
post #35
post #24

Earlier quoted context omitted.

Coroutines have the distinct disadvantage of needing a stack, much like threads. So-called 'stackless' coroutines aren't really so different to computed gotos in a state machine

What are stackless coroutines? What do they look like?

Read Adam Dunkels' paper on Protothreads.

Re: How We Designed for Performance and Scale

#42
post #37

"NGINX’s binary upgrade process achieves the holy grail of high-availability; you can upgrade the software on the fly, without any dropped connections, downtime or interruption in service." Is this really true? I remember seeing an article[1] recently on using an iptables hack to prevent dropping connections when reloading haproxy. Does nginx actually provide zero-downtime configuration reloads? [1] https://medium.co…

The holy grail of high-availability isn't upgrading stateless software. It's upgrading stateful ones. Like upgrading when a data structure changes between versions. The HTTP protocol nginx serves is stateless and by comparison far simpler. Same goes for Erlang. It offers nothing more than simple function replacement, and that's not enough to handle data structure changes either.

In Erlang, if you need a new data structure for your state, you can check if your state is old, and upgrade it and continue on. In a gen_server, you might have something like

  handle_call(Request, From, State) when is_record(State, state) -> handle_call(Request, From, upgrade_state(State);
  handle_call(Request, From, State) when is_record(State, state2) -> ...
(you'll want to do something similar on handle_cast and handle_info if you use those). You have to do a little work, but I don't see how you avoid that?

Re: How We Designed for Performance and Scale

#43
post #40
post #21

One thread per CPU, and non-blocking I/O, that's sounds like the usual way to approach the problem. I'm surprised it uses state machines to handle the non-blocking I/O, because modern software engineering provides much more pleasant approaches such as using coroutines.

nginx is written in C though - I know it's possible to do coroutine-type stuff with setjmp/longjmp, but isn't that considered risky?

setjmp()/longjmp() will work, but they're sort of inefficient as, at least under POSIX, they will save and restore the signal mask, which makes for two round-trips to the kernel just for a coroutine context switch.

My web server uses coroutines, and for x86 and x86-64 it uses open-coded assembly routines to yield/resume, with fallbacks to setjmp()/longjmp() on other architectures.

It works fairly well, performance-wise. In fact, it's one of the top-performing servers/frameworks in the TechEmpower's Web Framework benchmarks[1].

I wrote a similar article explaining how everything is put together here[2].

[1] https://www.techempower.com/benchmarks/ [2] http://tia.mat.br/blog/html/2014/10/06/life_of_a_http_reques...

Re: How We Designed for Performance and Scale

#44
post #29

I did as they said at the bottom and gave them my e-mail and other personal details so I could download the eBook that they were giving free preview copies of - "Building Microservices". Unfortunately, they sent link to PDF only so it's not usable to me. Just a heads up to others so you save yourself the time of discovering that. (I'll just wait for when the book is finished and then I'll buy it so I get ePub. I like…

If you have a kindle you can send a pdf to your kindle email with the subject 'convert' -

http://www.amazon.com/gp/sendtokindle/email

Re: How We Designed for Performance and Scale

#45
post #41
post #35

Earlier quoted context omitted.

What are stackless coroutines? What do they look like?

Read Adam Dunkels' paper on Protothreads.

Thanks for your comment.

I don't know how to feel about Protothreads. It's just syntatic sugar for using a state-machine to provide continuations. You're also not allowed to carry state over between continuations (although I can see how you can extend it to carry-over some struct of data or something). This greatly diminishes the usefulness of Protothreads.. so it feels more like a fun proof-of-concept.

Are there any popular/real-world use cases of Protothreads?

Re: How We Designed for Performance and Scale

#46
post #9

"NGINX’s binary upgrade process achieves the holy grail of high-availability; you can upgrade the software on the fly, without any dropped connections, downtime or interruption in service." Is this really true? I remember seeing an article[1] recently on using an iptables hack to prevent dropping connections when reloading haproxy. Does nginx actually provide zero-downtime configuration reloads? [1] https://medium.co…

Yes, nginx allows you to upgrade the binary and reload the configuration without any downtime: http://wiki.nginx.org/CommandLine#Upgrading_To_a_New_Binary_...

Does this work when you update with package manager?

Re: How We Designed for Performance and Scale

#47

> You can reload configuration multiple times per second (and many NGINX users do exactly that) I thought this was an interesting remark. Can anyone clue me in to what these "many users" might be doing, that requires them to reload configuration so frequently?

Service Discovery. Dynamically generating routing rules so microservices can have pretty URLs. For example with us when you deploy a new version of a microservice it starts on a random port, registers with Consul on startup and then dynamically regenerates and reloads Nginx.

Re: How We Designed for Performance and Scale

#48
post #37

"NGINX’s binary upgrade process achieves the holy grail of high-availability; you can upgrade the software on the fly, without any dropped connections, downtime or interruption in service." Is this really true? I remember seeing an article[1] recently on using an iptables hack to prevent dropping connections when reloading haproxy. Does nginx actually provide zero-downtime configuration reloads? [1] https://medium.co…

The holy grail of high-availability isn't upgrading stateless software. It's upgrading stateful ones. Like upgrading when a data structure changes between versions. The HTTP protocol nginx serves is stateless and by comparison far simpler. Same goes for Erlang. It offers nothing more than simple function replacement, and that's not enough to handle data structure changes either.

What about running websockets?

Re: How We Designed for Performance and Scale

#49
post #20

This is a lovely article, but: > The fundamental basis of any Unix application is the thread or process. (From the Linux OS perspective, threads and processes are mostly identical; the major difference is the degree to which they share memory.) It's better to be specific in performance discussions, rather than use 'thread' and 'process' interchangeably. As well as the article mentioned about memory sharing, threads (…

Aside: you really nailed the "I think they're saying something significantly incorrect but don't want to be a jerk about it" tone. Kudos.

Re: How We Designed for Performance and Scale

#50
post #43
post #40

Earlier quoted context omitted.

nginx is written in C though - I know it's possible to do coroutine-type stuff with setjmp/longjmp, but isn't that considered risky?

setjmp()/longjmp() will work, but they're sort of inefficient as, at least under POSIX, they will save and restore the signal mask, which makes for two round-trips to the kernel just for a coroutine context switch. My web server uses coroutines, and for x86 and x86-64 it uses open-coded assembly routines to yield/resume, with fallbacks to setjmp()/longjmp() on other architectures. It works fairly well, performance-wi…

Given the various efforts like libtask, lthread, boost coroutines, etc, it seems like the low-level assembly trickery ought to be isolated and standardized. Maybe some new methods (similar to the setcontext family) should be proposed to the glibc project. Something not at risk of deprecation.
Post reply on HN