Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GitHub's Unicorn Setup

github.com

Post reply on HN
Print view View original

21–30 of 64 posts

Re: GitHub's Unicorn Setup

#21

Post by mrshoe »

mojombo wrote: post #18
mrshoe wrote: post #16

Earlier quoted context omitted.

Where were all you people in this thread the other day when I seemed to be the only one who understood the other side of this debate?!? :-P I'm also very curious about the 16 workers per box model. Frankly, that github would ditch haproxy in favor of a pre-fork Ruby web server is shocking. I'm just glad we have CherryPy in the Python world.

Why is that shocking? Can you elaborate?

Sure, now that HN has decided that enough time has elapsed to allow me to reply. :-)

Actually, antonovka did an excellent job explaining it above. The most important aspect is that threads are "lighter weight" than processes. They use less memory and are quicker to context switch (usually). The result of this lighter weight is that you can spawn more threads than you could processes, on the same hardware. And when you're using one thread/process per connection, that means more concurrent connections on the same hardware. So, if Unicorn used its exact same architecture, but replaced the worker processes with worker threads, you could scale much better.

Secondly, haproxy is written in C, which generally means it's going to perform much better and use a lot less memory than a Ruby webserver. This translates, once again, to more output from the same amount of hardware.

That's why I was pretty surprised to see that github would choose both Ruby and pre-fork over C and threads (or in the case of haproxy, async, which scales even better).

Re: GitHub's Unicorn Setup

#22

Post by joevandyk »

antonovka wrote: post #9
aaronblohowiak wrote: post #6

Earlier quoted context omitted.

Right, but it is being handled by processes and not threads, which the argument was about the other day.

The argument encompassed two aspects of the model: 1) "Threads are out -- processes are better than threads." 2) Process-per-connection architectures. The first is demonstrably false -- for instance, look at Erlang, which maps lightweight erlang processes to operating system threads, providing SMP scalability at a low cost without running into Github's issues with mongrel "thread-killing". More broadly used, look at…

The 16 worker processes only handle dynamic requests, not static. With proper http caching, ESI, etc, it's quite likely that amount amount of concurrency is more than enough for their needs.

Say that when all the workers are being utilized, the average dynamic request takes 50ms (which is on the high end). That means that each box can handle 320 dynamic requests per second. Which is a decent amount, and I'd be surprised if they see that much traffic.

If they are using http caching and ESI properly, the average request time would be significantly lower, and the requests per second would probably be > 500/s on that box.

Re: GitHub's Unicorn Setup

#23

Post by mrshoe »

tptacek wrote: post #7
seiji wrote: post #4

Earlier quoted context omitted.

It's connection pooling, not a fork-per-accept web server. Each worker is select/epoll/kqueue'ing for individual requests.

Has any mainstream web server ever been pure demand-forked? Apache has been connection pooled since the '90s; the second edition of Unix Network Programming used it as a case study.

Exactly. Hence the "pre" in prefork. A "forking" web server (one fork per request) would be incredibly inefficient, whereas prefork is only... slightly inefficient. :-)

Re: GitHub's Unicorn Setup

#24

Post by [deleted] »

joevandyk wrote: post #22
antonovka wrote: post #9

Earlier quoted context omitted.

The argument encompassed two aspects of the model: 1) "Threads are out -- processes are better than threads." 2) Process-per-connection architectures. The first is demonstrably false -- for instance, look at Erlang, which maps lightweight erlang processes to operating system threads, providing SMP scalability at a low cost without running into Github's issues with mongrel "thread-killing". More broadly used, look at…

The 16 worker processes only handle dynamic requests, not static. With proper http caching, ESI, etc, it's quite likely that amount amount of concurrency is more than enough for their needs. Say that when all the workers are being utilized, the average dynamic request takes 50ms (which is on the high end). That means that each box can handle 320 dynamic requests per second. Which is a decent amount, and I'd be surpri…

[deleted]

Re: GitHub's Unicorn Setup

#25

Post by joevandyk »

antonovka wrote: post #20
mojombo wrote: post #17

Earlier quoted context omitted.

Each box can handle plenty more than 16 workers, but we currently don't need that many, so it's pointless to run them. Our frontend machines generally sit about 60% idle. We wanted to have plenty of headroom on our new setup. Also, our Ruby application code spends very little time blocking on external resources (compared to time spent in Ruby execution), so having async execution on these wouldn't give us that much m…

Each box can handle plenty more than 16 workers, but we currently don't need that many, so it's pointless to run them. Our frontend machines generally sit about 60% idle. We wanted to have plenty of headroom on our new setup. At those numbers, it sounds like you'd max out at roughly 32 workers (so 32 concurrent requests), or am I missing something important? It's true that running Ruby code is generally more expensiv…

The github developers are great ruby developers, and I'd be very surprised if they would be as productive in the short term in a different language. I do not see a business case for switching platforms for them.

You seem hung up on the number of concurrent requests. An 8 core machine can only do 8 things at once, no matter if you are using threads or processes.

Re: GitHub's Unicorn Setup

#26

Post by joevandyk »

mrshoe wrote: post #21
mojombo wrote: post #18

Earlier quoted context omitted.

Why is that shocking? Can you elaborate?

Sure, now that HN has decided that enough time has elapsed to allow me to reply. :-) Actually, antonovka did an excellent job explaining it above. The most important aspect is that threads are "lighter weight" than processes. They use less memory and are quicker to context switch (usually). The result of this lighter weight is that you can spawn more threads than you could processes, on the same hardware. And when yo…

The limiting factor of concurrency in a Rails application is the slowness of Ruby, not processes vs threads.

If the bottleneck of Ruby were fixed, then the next bottleneck would be the database.

Re: GitHub's Unicorn Setup

#27

Post by antonovka »

joevandyk wrote: post #25
antonovka wrote: post #20

Earlier quoted context omitted.

Each box can handle plenty more than 16 workers, but we currently don't need that many, so it's pointless to run them. Our frontend machines generally sit about 60% idle. We wanted to have plenty of headroom on our new setup. At those numbers, it sounds like you'd max out at roughly 32 workers (so 32 concurrent requests), or am I missing something important? It's true that running Ruby code is generally more expensiv…

The github developers are great ruby developers, and I'd be very surprised if they would be as productive in the short term in a different language. I do not see a business case for switching platforms for them. You seem hung up on the number of concurrent requests. An 8 core machine can only do 8 things at once, no matter if you are using threads or processes.

You seem hung up on the number of concurrent requests. An 8 core machine can only do 8 things at once, no matter if you are using threads or processes.

You can't assume that the threads are 100% CPU bound. If they were 100% CPU bound, you would have a point -- 8 cores, 8 CPU-bound processes, no processing time left over -- that is where event based and hybrid thread/event-based architectures excel.

In reality, webapp threads will sleep -- waiting on the network, waiting on disk, waiting on the database. When they sleep, the the processor has nothing to do. If the OS can schedule another thread while one sleeps, work can move forward.

If you can run 500 threads to completion in 10ms, then you can serve 500 requests within 10ms.

If you can run 16 (or 32) processes to completion in 10ms, then you can only serve 16 (or 32) requests within 10ms.

Re: GitHub's Unicorn Setup

#29

Post by mbrubeck »

antonovka wrote: post #9
aaronblohowiak wrote: post #6

Earlier quoted context omitted.

Right, but it is being handled by processes and not threads, which the argument was about the other day.

The argument encompassed two aspects of the model: 1) "Threads are out -- processes are better than threads." 2) Process-per-connection architectures. The first is demonstrably false -- for instance, look at Erlang, which maps lightweight erlang processes to operating system threads, providing SMP scalability at a low cost without running into Github's issues with mongrel "thread-killing". More broadly used, look at…

"Look at Erlang, which maps lightweight erlang processes to operating system threads, providing SMP scalability at a low cost..."

Erlang can use native threads, but they do not map to its lightweight processes. Erlang's lightweight processes are basically green threads. Native threads are used for SMP scheduling only. So on a four-core system, you might have thousands of Erlang lightweight processes running on just four OS threads in one OS process.

Post reply on HN