I never ever ever never use SO's search feature: I land on the result through Google. So I guess they should give Google some credit for the claimed performance?
StackOverflow Update: 560M Pageviews a Month, 25 Servers
151–160 of 278 posts
Re: StackOverflow Update: 560M Pageviews a Month, 25 Servers
#152Earlier quoted context omitted.
Yes, the idea is to scale up as much as economically makes sense and only then scale out, but due to Moore's law we're still scaling up. Not everyone has google-like problems that are betters solved by a battery of cheap boxes.
Actually most of the websites today are better solved by a battery of cheap boxes. You simply put a load balancer (Nginx/ELB/HAProxy etc) in front of a fleet of smaller web/application servers that dynamically scale depending on traffic. That way it is cost effective, far more reliable, easier to scale and you can tolerate DC outages better.
As with most systems, the data store is the hard thing to scale out. That requires intelligence at the application level, and that might include CAP issues.
So for us, the SQL data store is the real “up” part of the equation. We have a fair amount of headroom there, so if we can keep sharding (and other data strategies) out of the codebase, so much the better.
(Load-balancing HTTP requests “out” is not a big deal and we are doing that.)
Re: StackOverflow Update: 560M Pageviews a Month, 25 Servers
#153Earlier quoted context omitted.
Programmer time is a factor that comes into play when you build something and re-training your people to use a stack they are not familiar with + the associated mistakes can get expensive in a hurry. If you happen to be familiar with the MS stack and are able to factor the license fees into your business plan there is no strong reason against and many for using that particular stack. MS programmers tend to be a bit m…
> the MS stack is more performant than the Linux one on the same hardware This is provably not true. http://w3techs.com/technologies/overview/web_server/all http://commons.wikimedia.org/wiki/File:Operating_systems_use... http://w3techs.com/technologies/overview/operating_system/al... There is a good reason majority of the World-Wide-Web is run on 'Nix stacks -- and there is a good reason majority of servers in genera…
It will be very hard to get into a flamewar with me supporting the Microsoft side of things. That said, none of the links you posted prove anything regarding performance, they do prove something about total cost of ownership, which once you factor everything in leans towards Linux for most installations, however, just looking at the situation for SO seems to me to suggest that they were more comfortable doing the initial development on the stack they were most familiar with and when the license costs started to count against them they used Linux machines to scale out.
Which is a pretty smart decision. Whether an MS stack at this scale is unusual does not say anything at all about whether or not it performs well.
Re: StackOverflow Update: 560M Pageviews a Month, 25 Servers
#154Earlier quoted context omitted.
> the MS stack is more performant than the Linux one on the same hardware This is provably not true. http://w3techs.com/technologies/overview/web_server/all http://commons.wikimedia.org/wiki/File:Operating_systems_use... http://w3techs.com/technologies/overview/operating_system/al... There is a good reason majority of the World-Wide-Web is run on 'Nix stacks -- and there is a good reason majority of servers in genera…
It will be very hard to get into a flamewar with me supporting the Microsoft side of things. That said, none of the links you posted prove anything regarding performance, they do prove something about total cost of ownership, which once you factor everything in leans towards Linux for most installations, however, just looking at the situation for SO seems to me to suggest that they were more comfortable doing the ini…
I think we agree for the SE team, given the founders were MS stack familiar, it makes sense.
However, I have to disagree on your assertion the 'Nix's are less performant than the MS stack. The top tier web companies are not running 'Nix because TCO is lower; for most of these companies licensing costs are negligible and if it helped to scale better, it may even save them money going with MS stack... but they don't go with a MS stack...
Sometimes a RHEL license can even cost more than a MS license. Couple that with an Oracle DB back-end, and you easily have a much more costly setup than the MS equivalent. It's not about the money.
These companies are choosing the 'Nix stack because it is performing in an entirely different level than the MS stack. Everything from tiny embedded systems with 64k ram, up to monster systems with TB's of ram.
> "I Contribute to the Windows Kernel. We Are Slower Than Other Operating Systems. Here Is Why." http://blog.zorinaq.com/?e=74
Re: StackOverflow Update: 560M Pageviews a Month, 25 Servers
#155So this is what, a 2000 request/sec peak? Over 11 servers, that's like 200 requests/sec peak per frontend? The problem with scale-up is if you actually have to get a few times larger, it becomes super expensive. But fortunately hardware is increasing so much that you can probably just get away with it now. There's probably a crossover point we're rapidly approaching where even global-scale sites can just do all their…
Peak is more like 2600-3000 requests/sec on most weekdays. Remember that programming, being a profession, means our weekdays are significantly busier than weekends (as you can see here: https://www.quantcast.com/p-c1rF4kxgLUzNc).
It's almost all over 9 servers, because 10 and 11 are only for meta.stackexchange.com, meta.stackoverflow.com, and the development tier. Those servers also run around 10-20% CPU which means we have quite a bit of headroom available. Here's a screenshot of our dashboard taken just now: http://i.stack.imgur.com/HPdtl.png We can currently handle the full load of all sites (including Stack Overflow) on 2 servers...not 1 though, that ends badly with thread exhaustion.
We could add web servers pretty cheaply; these servers are approaching 4 years old and weren't even close to top-of-the-line back them. Even current generation replacements would be several times more powerful, if we needed to go that route.
Honestly the only scale-up problem we have is SSD space on the SQL boxes due to the growth pattern of reliability vs. space in the non-consumer space. By that I mean drives that have capacitors for power loss and such. I actually just wrote a lengthy email about what we're planning for storage on one of our SQL clusters...perhaps I should echo it verbatim as a blog post? I'm not sure how many people care about that sort of stuff outside our teams.
Nick Craver - Stack Exchange Sysadmin & Developer
Re: StackOverflow Update: 560M Pageviews a Month, 25 Servers
#156So this is what, a 2000 request/sec peak? Over 11 servers, that's like 200 requests/sec peak per frontend? The problem with scale-up is if you actually have to get a few times larger, it becomes super expensive. But fortunately hardware is increasing so much that you can probably just get away with it now. There's probably a crossover point we're rapidly approaching where even global-scale sites can just do all their…
Peak is more like 2600-3000 requests/sec on most weekdays. Remember that programming, being a profession, means our weekdays are significantly busier than weekends (as you can see here: https://www.quantcast.com/p-c1rF4kxgLUzNc ). It's almost all over 9 servers, because 10 and 11 are only for meta.stackexchange.com, meta.stackoverflow.com, and the development tier. Those servers also run around 10-20% CPU which means…
yes please
Re: StackOverflow Update: 560M Pageviews a Month, 25 Servers
#157Earlier quoted context omitted.
There's a couple of user-contrib projects that do similar things for inserts & updates. Dapper Extensions is the one I'm using. https://github.com/tmsmith/Dapper-Extensions
I wrote a Dapper extension for working with SQL Server's geospatial queries and types a couple of years ago - have they added anything like that yet? Otherwise I'd be happy to add it.
Marc Gravell added in the ability to put in pretty much any custom type without adding any dependency weight about a month ago. You can see the commit here: https://github.com/StackExchange/dapper-dot-net/commit/e26ee...
Look towards the end at the tests for example of how to hook up a custom type (it's pretty simple).
Re: StackOverflow Update: 560M Pageviews a Month, 25 Servers
#158So this is what, a 2000 request/sec peak? Over 11 servers, that's like 200 requests/sec peak per frontend? The problem with scale-up is if you actually have to get a few times larger, it becomes super expensive. But fortunately hardware is increasing so much that you can probably just get away with it now. There's probably a crossover point we're rapidly approaching where even global-scale sites can just do all their…
Peak is more like 2600-3000 requests/sec on most weekdays. Remember that programming, being a profession, means our weekdays are significantly busier than weekends (as you can see here: https://www.quantcast.com/p-c1rF4kxgLUzNc ). It's almost all over 9 servers, because 10 and 11 are only for meta.stackexchange.com, meta.stackoverflow.com, and the development tier. Those servers also run around 10-20% CPU which means…
>I actually just wrote a lengthy email about what we're planning for storage on one of our SQL clusters...perhaps I should echo it verbatim as a blog post? I'm not sure how many people care about that sort of stuff outside our teams.
I'm sure some DBAs and devs here would find it interesting.
Re: StackOverflow Update: 560M Pageviews a Month, 25 Servers
#159So this is what, a 2000 request/sec peak? Over 11 servers, that's like 200 requests/sec peak per frontend? The problem with scale-up is if you actually have to get a few times larger, it becomes super expensive. But fortunately hardware is increasing so much that you can probably just get away with it now. There's probably a crossover point we're rapidly approaching where even global-scale sites can just do all their…
Peak is more like 2600-3000 requests/sec on most weekdays. Remember that programming, being a profession, means our weekdays are significantly busier than weekends (as you can see here: https://www.quantcast.com/p-c1rF4kxgLUzNc ). It's almost all over 9 servers, because 10 and 11 are only for meta.stackexchange.com, meta.stackoverflow.com, and the development tier. Those servers also run around 10-20% CPU which means…
My team would definitely be interested to read that. Please do!