GitHub's Unicorn Setup
github.com
GitHub's Unicorn Setup
1–10 of 64 posts
Re: GitHub's Unicorn Setup
#2How does passenger handle restarts? Does it also allow a zero downtime restart ?
Re: GitHub's Unicorn Setup
#3When the Unicorn master starts, it loads our app into memory. As soon as it’s ready to serve requests it forks 16 workers. Those workers then select() on the socket, only serving requests they’re capable of handling. In this way the kernel handles the load balancing for us.
Wasn't there a heated debate here just the other day about the prefork model?
Guess it's at least back en vogue @github.
Re: GitHub's Unicorn Setup
#4When the Unicorn master starts, it loads our app into memory. As soon as it’s ready to serve requests it forks 16 workers. Those workers then select() on the socket, only serving requests they’re capable of handling. In this way the kernel handles the load balancing for us. Wasn't there a heated debate here just the other day about the prefork model? Guess it's at least back en vogue @github.
It's connection pooling, not a fork-per-accept web server. Each worker is select/epoll/kqueue'ing for individual requests.
Re: GitHub's Unicorn Setup
#5Its great to see companies sharing what they've learned from experience about system architecture. So many times this sort of stuff is very difficult to plan out and the only real way to get it right is through experimentation. Having first hand descriptions like this is a great resource if you are setting something up the first time.
I like that they posted their unicorn config file too!
Re: GitHub's Unicorn Setup
#6When the Unicorn master starts, it loads our app into memory. As soon as it’s ready to serve requests it forks 16 workers. Those workers then select() on the socket, only serving requests they’re capable of handling. In this way the kernel handles the load balancing for us. Wasn't there a heated debate here just the other day about the prefork model? Guess it's at least back en vogue @github.
It's connection pooling, not a fork-per-accept web server. Each worker is select/epoll/kqueue'ing for individual requests.
Right, but it is being handled by processes and not threads, which the argument was about the other day.
Re: GitHub's Unicorn Setup
#7When the Unicorn master starts, it loads our app into memory. As soon as it’s ready to serve requests it forks 16 workers. Those workers then select() on the socket, only serving requests they’re capable of handling. In this way the kernel handles the load balancing for us. Wasn't there a heated debate here just the other day about the prefork model? Guess it's at least back en vogue @github.
It's connection pooling, not a fork-per-accept web server. Each worker is select/epoll/kqueue'ing for individual requests.
Has any mainstream web server ever been pure demand-forked? Apache has been connection pooled since the '90s; the second edition of Unix Network Programming used it as a case study.
Re: GitHub's Unicorn Setup
#8How does passenger handle restarts? Does it also allow a zero downtime restart ?
You run "touch #{RAILS_ROOT}/tmp/restart.txt".
That will restart the rails processes. No connections are dropped, and I've not seen any downtime.
Re: GitHub's Unicorn Setup
#9Earlier quoted context omitted.
It's connection pooling, not a fork-per-accept web server. Each worker is select/epoll/kqueue'ing for individual requests.
Right, but it is being handled by processes and not threads, which the argument was about the other day.
The argument encompassed two aspects of the model:
1) "Threads are out -- processes are better than threads."
2) Process-per-connection architectures.
The first is demonstrably false -- for instance, look at Erlang, which maps lightweight erlang processes to operating system threads, providing SMP scalability at a low cost without running into Github's issues with mongrel "thread-killing". More broadly used, look at Servlets and the Servlet 3.0 support for async comet-style event-based request handling. Each request is handled on a thread as necessary. Inter-thread communication (where necessary) is cheap, and this scales just fine.
If you use the fork() model and a long-running request blocks the entire process, comet is basically a non-starter. This is why people are interested (and implement) lightweight threads, coroutines, and restartable request implementations.
For a conceptual challenge, consider how you would implement a live web chat system that can scale up to a considerable number of clients, with extremely low resource usage, and instant message distribution (no polling). Locally, we implemented this with async servlet support and M:N scheduled scala actors -- a blocking HTTP request doesn't hold a thread or process hostage in the web server or the application, and we can scale up to enormous number of live clients on one machine.
The second was merely a lack of understanding of the model. In implementations where fork() is used as an alternative to threads and multiple connections are not handled per sub-process, you quickly run into scaling issues with subprocess memory utilization. In cases where subprocesses handle multiple connections via an event mechanism, you're just using fork() instead of threads.
I'm wondering how many servers github is using to keep up with load -- If each 8 core 16 gigabyte (!!!) server can actually only handle 16 concurrent requests via a pool of 16 workers, that's an incredibly poor (and expensive) scaling model.
Re: GitHub's Unicorn Setup
#10How does passenger handle restarts? Does it also allow a zero downtime restart ?
You run "touch #{RAILS_ROOT}/tmp/restart.txt". That will restart the rails processes. No connections are dropped, and I've not seen any downtime.
To clarify, my understanding is that while rails is restarting, Passenger will queue all the requests that come in and begin processing them as soon as rails is ready. But yeah, zero downtime, it's pretty awesome.