Live data from Hacker News

StackOverflow Update: 560M Pageviews a Month, 25 Servers

highscalability.com

31–40 of 278 posts

Re: StackOverflow Update: 560M Pageviews a Month, 25 Servers

#31

Can anyone provide context on "Heavy usage of static classes and methods, for simplicity and better performance." Is performance _really_ an issue here?

I could be mistaken, but I think this is in regards to that post Sam Saffron wrote a few years back about getting SO to do Gen.2 garbage collection less often, and using static classes and structs were one of the ways that they fixed it.

Re: StackOverflow Update: 560M Pageviews a Month, 25 Servers

#32
post #3

Makes me wonder why smaller website teams need dozens of engineers to keep their infinitesimally smaller app running.

Guess they don't have Jeff Atwood and Joel Spolsky onboard.

As of about 2 years ago, Stack Exchange doesn't have Jeff onboard either.

http://blog.codinghorror.com/farewell-stack-exchange/

Re: StackOverflow Update: 560M Pageviews a Month, 25 Servers

#33
> The cost of inefficient code can be higher than you think. Efficient code stretches hardware further, reduces power usage, makes code easier for programmers to understand.

I'm curious what the reasoning is for "Efficient code ... makes code easier for programmers to understand". To my mind, efficient code (in this case, I assume coding to the hardware, as they mention elsewhere), has many benefits, but making it easier to understand is not one of them. A useful comment by the code may help, but that's not a result of efficient coding, that's a result of good commenting practice.

Re: StackOverflow Update: 560M Pageviews a Month, 25 Servers

#34
post #30

"One problem is not many tests. Tests aren’t needed because there’s a great community... If users find any problems with it they report the bugs that they’ve found." I'm often surprised at the paucity of test-coverage in relatively large companies. http://nerds.airbnb.com/testing-at-airbnb/

That bullet point surprised me. It comes right after "110K lines of code. A small number given what it does"; to me this reads like they didn't write tests because that would add complexity. Can any SO devs give us more details on "not many tests"? Or how many bug reports get filed vs rate of change of software?

The things that obviously should have tests have tests. That means most of the things that touch money on our Careers product, and easily unit-testable features on the Core end (things with known inputs, e.g. flagging, our new top bar, etc), for most other things we just do a functionality test by hand and push it to our incubating site (formerly meta.stackoverflow, now meta.stackexchange).

You can look at reported bugs here: http://meta.stackexchange.com/questions/tagged/bug -- it's pretty frequent, at least one every couple of hours or so. The last commit in our "Tests" project was 14 days ago, where as the last commit in the code base was 4 minutes ago.

Re: StackOverflow Update: 560M Pageviews a Month, 25 Servers

#35
post #22

> AWS would cost a fortune. Isn't that comparing apples to oranges? Taking any application built for large servers and putting the same application onto a cloud-based architecture will be more expensive. Cloud-based architecture requires ground up differences in how the application is built. Now whether or not it is better to use a cloud-based approach or traditional bare metal is highly subjective and isn't my point…

>'Isn't that comparing apples to oranges?'

On another site, in another context, it probably would be, but here it's really presented as a contrast of scale-up versus scale-out - something the regular audience of highscalability will certainly grok.

In context...

'Stack Overflow still uses a scale-up strategy. No clouds in site. With their SQL Servers loaded with 384 GB of RAM and 2TB of SSD, AWS would cost a fortune. The cloud would also slow them down, making it harder to optimize and troubleshoot system issues. Plus, SO doesn’t need a horizontal scaling strategy. Large peak loads, where scaling out makes sense, hasn’t been a problem because they’ve been quite successful at sizing their system correctly.'

It's an acknowledgement of their relatively unique strategy and the short list of caveats that make it possible.

Re: StackOverflow Update: 560M Pageviews a Month, 25 Servers

#36
post #3

Makes me wonder why smaller website teams need dozens of engineers to keep their infinitesimally smaller app running.

I personally think it's due to overly complex applications and systems with many moving (and breaking) parts. Also StackOverflow often practices "scale-up" instead of "scale-out".

More specifically, I think it's due to long feature lists and multiple problem domains. I'm also thinking about in-house line-of-business software.

Re: StackOverflow Update: 560M Pageviews a Month, 25 Servers

#37
post #18

Anyone else not quite expecting StackExchange to be using a Microsoft stack?

Joel worked at MS on office, Excel if I recall correctly. Jeff's blog is named for a feature from a book published by MS. I would be more surprised if it was MS free.

Re: StackOverflow Update: 560M Pageviews a Month, 25 Servers

#38
post #12
post #5

> With their SQL Servers loaded with 384 GB of RAM and 2TB of SSD, AWS would cost a fortune. I have next to zero experience with server administration, but 384GB seems like a lot to me. Is that common for production servers for popular web services? Do you need a customized OS to address that much memory? Seems like you'd really need to beef up the cache hierarchy make 0.38TB of RAM fast.

Their SQL Server is Microsoft SQL Server running on (shock horror) Windows. Says in the article: Windows 2012 is used in New York but are upgrading to 2012 R2 (Oregon is already on it).

To be clear, most servers are moving to 2012 R2 (most already have) but the machines our SQL clusters run on, specifically, will not.

This is because you have to effectively (or literally) rebuild the Windows cluster from scratch and we just don't get that level of benefit from the 2012 to 2012 R2 upgrade. There are quite a few improvements we care about: native NVMe, better dynamic quorum, better DSC support, better SMB, and such...but not enough to make the upgrade worth it.

Nick Craver - Stack Exchange Sysadmin & Developer

Re: StackOverflow Update: 560M Pageviews a Month, 25 Servers

#40
post #18

Anyone else not quite expecting StackExchange to be using a Microsoft stack?

It's a Microsoft/Linux hybrid. SQL Server on the backend, Linux for monitoring, Redis, HAProxy, etc.

They discussed it quite a bit on the podcasts. I found those a really great resource since Jeff and Joel were discussing the website as they were building it and "aired" the reasons for doing specific things.
Post reply on HN