Live data from Hacker News

GitHub's Unicorn Setup

github.com

51–60 of 64 posts

Re: GitHub's Unicorn Setup

#51
post #50

Earlier quoted context omitted.

The argument encompassed two aspects of the model: 1) "Threads are out -- processes are better than threads." 2) Process-per-connection architectures. The first is demonstrably false -- for instance, look at Erlang, which maps lightweight erlang processes to operating system threads, providing SMP scalability at a low cost without running into Github's issues with mongrel "thread-killing". More broadly used, look at…

Erlang is not really an argument for threads. The Erlang model is to utilize poll/select/whatever together with a language that contains its own scheduler. If you execute arbitrary python/ruby/tcl/php code, there's a chance it will block. If you execute arbitrary Erlang code, the chance of that is very low. This means that you can handle all kinds of long running calculations in the same OS process, while you continu…

Erlang is not really an argument for threads.

Solaris 2-8 implemented a general purpose M:N thread scheduler mapping user-space threads to kernel threads. This is the exact same solution Erlang implements for the less general purpose use-case of actor messaging.

Is Solaris' M:N thread implementation somehow not 'threaded'? If it is threaded (and it is), then how is Erlang's implementation not also an argument for threads?

To elucidate rather than rely on Erlang as an example, the argument for threads over processes:

1) Extremely low-cost alternative to IPC. Threads allow (but do not require the use of) shared mutable state. This is a much, much cheaper way to communicate between concurrent entities. You can achieve the same effect with processes and shared memory, but it's significantly more complex to implement and subject to the disadvantages listed below.

2) Extremely low memory foot-print. A thread costs a stack plus minor OS book keeping. A fork(2) can leverage COW pages, but almost invariably the number of non-shared pages will be significantly higher than with a thread. If an operation blocks a thread, it's cheap to create more. If an operation blocks a process, you'll hit resource constraints far more quickly trying to fork() more.

Of course, leveraging shared memory and other operating system tools, you can turn a fork(2) implementation into a thread alternative -- but then, you'd have ... threads. This is what Linux's clone(2) syscall was intended for -- a thread-implementation friendly fork(2) alternative -- and the pthread library was built on top of it.

... for a while, you could call setuid() in a Linux "thread" and the new uid would be a thread-local change, because the thread was actually a 'process'.

This is effectively not an argument against fork(2) (although it's an expensive route to the same solution offered by threads), but rather against scaling models that will block an entire OS process for the sake of a single request.

Re: GitHub's Unicorn Setup

#52
post #36

Earlier quoted context omitted.

Yeah. We load balance across 5 Apache/Passengers and I do rolling deploys (all in Capistrano) by removing a Passenger from load balancing, updating the app, restarting Apache, and adding it back into load balancing with a 10 second delay between each. We tried the Passenger touch restart.txt and that didn't go well at all when we were under load.

Can you share your setup please?

He has been kind enough to share about his setup quite a lot about lately. In fact, looks like it's on the HN homepage right now: http://news.ycombinator.com/item?id=872301

Re: GitHub's Unicorn Setup

#53
post #50

Earlier quoted context omitted.

Erlang is not really an argument for threads. The Erlang model is to utilize poll/select/whatever together with a language that contains its own scheduler. If you execute arbitrary python/ruby/tcl/php code, there's a chance it will block. If you execute arbitrary Erlang code, the chance of that is very low. This means that you can handle all kinds of long running calculations in the same OS process, while you continu…

Erlang is not really an argument for threads. Solaris 2-8 implemented a general purpose M:N thread scheduler mapping user-space threads to kernel threads. This is the exact same solution Erlang implements for the less general purpose use-case of actor messaging. Is Solaris' M:N thread implementation somehow not 'threaded'? If it is threaded (and it is), then how is Erlang's implementation not also an argument for thr…

Threads are a 'nice extra' for Erlang. They're definitely a plus, no arguments there, but in your M:N mapping, N is fixed at the number of CPU's in the system. So if you have one CPU, Erlang doesn't really use threads at all, and yet it is still very scalable. The scalability on our one-CPU system comes from being able to use a nice, compact, select/poll based system and keep everything in one OS process, without spawning additional threads or OS processes.

So, threads are really just an optimized case of launching N Erlang vm's, where N is the number of processes, and splitting the work between them. They were a late addition to the language too - this only happened several years ago, if I recall correctly.

In other words, threads are a nice boost to Erlang, but not really the secret to its success.

Re: GitHub's Unicorn Setup

#54
post #53

Earlier quoted context omitted.

Erlang is not really an argument for threads. Solaris 2-8 implemented a general purpose M:N thread scheduler mapping user-space threads to kernel threads. This is the exact same solution Erlang implements for the less general purpose use-case of actor messaging. Is Solaris' M:N thread implementation somehow not 'threaded'? If it is threaded (and it is), then how is Erlang's implementation not also an argument for thr…

Threads are a 'nice extra' for Erlang. They're definitely a plus, no arguments there, but in your M:N mapping, N is fixed at the number of CPU's in the system. So if you have one CPU, Erlang doesn't really use threads at all, and yet it is still very scalable. The scalability on our one-CPU system comes from being able to use a nice, compact, select/poll based system and keep everything in one OS process, without spa…

In other words, threads are a nice boost to Erlang, but not really the secret to its success.

I think you missed my meaning. On FreeBSD 4.x and Linux 2.0.x, threads were implemented entirely in user-space. They didn't allow for true concurrent execution of multiple threads on multiple CPUs, but they were still very much threads.

Likewise, erlang's lightweight processes are exactly the same -- threads. The fact that they can be modeled to arbitrary numbers of operating system threads is irrelevant to the nature of the model -- one in which execution of a thread can proceed independently of others, implemented via context switching of execution state across those threads, able to access shared state.

However, in the fork model as described in the original article, processes are the only form of concurrency. A blocked request will, in turn, block a process, and unlike threads, far fewer processes may be run concurrently due to their significantly increased resource constraints. Furthermore, those processes are much more limited in their ability to implement low-cost inter-thread communication via shared mutable state.

If the processes actually relied on their own internal M:N scheduled green threads, then at least that part of the concurrency problem would be (mostly) solved, and fewer processes would be required. IPC is still an issue, and of course, there's the multi-decade demonstration of the high difficulty in implementing a 'performant' general-purpose M:N scheduled thread system.

Re: GitHub's Unicorn Setup

#55
post #53

Earlier quoted context omitted.

Threads are a 'nice extra' for Erlang. They're definitely a plus, no arguments there, but in your M:N mapping, N is fixed at the number of CPU's in the system. So if you have one CPU, Erlang doesn't really use threads at all, and yet it is still very scalable. The scalability on our one-CPU system comes from being able to use a nice, compact, select/poll based system and keep everything in one OS process, without spa…

In other words, threads are a nice boost to Erlang, but not really the secret to its success. I think you missed my meaning. On FreeBSD 4.x and Linux 2.0.x, threads were implemented entirely in user-space. They didn't allow for true concurrent execution of multiple threads on multiple CPUs, but they were still very much threads. Likewise, erlang's lightweight processes are exactly the same -- threads. The fact that t…

> Likewise, erlang's lightweight processes are exactly the same -- threads. The fact that they can be modeled to arbitrary numbers of operating system threads is irrelevant to the nature of the model -- one in which execution of a thread can proceed independently of others, implemented via context switching of execution state across those threads, able to access shared state.

From Wikipedia:

> The Erlang virtual machine has what might be called 'green processes' - they are like operating system processes (they do not share state like threads do) but are implemented within the Erlang Run Time System (erts). These are sometimes (erroneously) cited as 'green threads'.

http://en.wikipedia.org/wiki/Green_threads

Going back to Ruby, sure, threads might be theoretically better than processes, but in practical terms, other things are the real bottleneck, so it ends up not really mattering that much.

Re: GitHub's Unicorn Setup

#56
post #55

Earlier quoted context omitted.

In other words, threads are a nice boost to Erlang, but not really the secret to its success. I think you missed my meaning. On FreeBSD 4.x and Linux 2.0.x, threads were implemented entirely in user-space. They didn't allow for true concurrent execution of multiple threads on multiple CPUs, but they were still very much threads. Likewise, erlang's lightweight processes are exactly the same -- threads. The fact that t…

> Likewise, erlang's lightweight processes are exactly the same -- threads. The fact that they can be modeled to arbitrary numbers of operating system threads is irrelevant to the nature of the model -- one in which execution of a thread can proceed independently of others, implemented via context switching of execution state across those threads, able to access shared state. From Wikipedia: > The Erlang virtual mach…

> The Erlang virtual machine has what might be called 'green processes' - they are like operating system processes (they do not share state like threads do) but are implemented within the Erlang Run Time System (erts). These are sometimes (erroneously) cited as 'green threads'.

Erlang processes' implementations share internal mutable state, despite the fact that the inter-process messaging API available via the runtime does not expose this. They have to -- internal cooperative context switching can't be implemented without shared mutable state.

They're threads.

Going back to Ruby, sure, threads might be theoretically better than processes, but in practical terms, other things are the real bottleneck, so it ends up not really mattering that much.

I'm not sure I follow. Is this something like "ruby is really, really slow, so no need to worry about concurrency" ?

Re: GitHub's Unicorn Setup

#57
post #7
post #4

Earlier quoted context omitted.

It's connection pooling, not a fork-per-accept web server. Each worker is select/epoll/kqueue'ing for individual requests.

Has any mainstream web server ever been pure demand-forked? Apache has been connection pooled since the '90s; the second edition of Unix Network Programming used it as a case study.

Ho ho ho. Back in the day, people used to run httpd off of inetd. I was real smug back then when I replaced that at our site with NCSA httpd, which did indeed fork on demand. Think this was '93 or thenabouts.

Re: GitHub's Unicorn Setup

#58
post #23
post #7

Earlier quoted context omitted.

Has any mainstream web server ever been pure demand-forked? Apache has been connection pooled since the '90s; the second edition of Unix Network Programming used it as a case study.

Exactly. Hence the "pre" in prefork. A "forking" web server (one fork per request) would be incredibly inefficient, whereas prefork is only... slightly inefficient. :-)

How's this for incredibly inefficient http://github.com/tlrobinson/wwwoosh/blob/master/wwwoosh.sh ;)

Re: GitHub's Unicorn Setup

#59
post #50

Earlier quoted context omitted.

Erlang is not really an argument for threads. The Erlang model is to utilize poll/select/whatever together with a language that contains its own scheduler. If you execute arbitrary python/ruby/tcl/php code, there's a chance it will block. If you execute arbitrary Erlang code, the chance of that is very low. This means that you can handle all kinds of long running calculations in the same OS process, while you continu…

Erlang is not really an argument for threads. Solaris 2-8 implemented a general purpose M:N thread scheduler mapping user-space threads to kernel threads. This is the exact same solution Erlang implements for the less general purpose use-case of actor messaging. Is Solaris' M:N thread implementation somehow not 'threaded'? If it is threaded (and it is), then how is Erlang's implementation not also an argument for thr…

leveraging shared memory and other operating system tools, you can turn a fork(2) implementation into a thread alternative -- but then, you'd have ... threads

There's an important difference: in the threading model, all memory is shared by default, whereas in the processes + shm model, you need to explicitly allocate and access shared memory regions. Because sharing mutable state is such an important design consideration, being explicit about what is being shared improves reliability by reducing the chance of unintentional sharing. A related benefit is that processes are protected from one another by the virtual memory mechanism: it is much more feasible for a service to recover from a crashed process than to recover from a crashed thread (which typically takes down the entire process).

Re: GitHub's Unicorn Setup

#60
post #36

Earlier quoted context omitted.

Yeah. We load balance across 5 Apache/Passengers and I do rolling deploys (all in Capistrano) by removing a Passenger from load balancing, updating the app, restarting Apache, and adding it back into load balancing with a 10 second delay between each. We tried the Passenger touch restart.txt and that didn't go well at all when we were under load.

Can you share your setup please?

I wrote a short post about our setup in March and it hasn't really changed since then: http://codemonkey.ravelry.com/2009/03/10/quick-update-ravelr...

If you have any questions or are looking for details or something, ask away - just stick a comment on the blog post.

Post reply on HN