I would love to know more about the Databases:
- Are they used for different things on the sites?
- Is data partitioned across tables?
- Are they all SQL Server instances?
The architecture of Stack Overflow [video]
11–20 of 49 posts
Re: The architecture of Stack Overflow [video]
#12Some points that I find interesting: [1] StackOverflow has VERY FEW tests. He says that StackOverflow doesn't use many unit tests because of their active community and heavy usage of static code. [2] Most StackOverflow employees work remotely. This is very different than a lot of companies that are now trying to force employees back into an office. [3] Heavy usage of Static classes and methods. His main argument is t…
[7]. Millions of page views and just 25 servers for whole infrastructure, which includes everything including load balancing, cache, dbs' etc. etc.
Seems very very optimized and cost effective. Its brilliant.
Re: The architecture of Stack Overflow [video]
#13Some points that I find interesting: [1] StackOverflow has VERY FEW tests. He says that StackOverflow doesn't use many unit tests because of their active community and heavy usage of static code. [2] Most StackOverflow employees work remotely. This is very different than a lot of companies that are now trying to force employees back into an office. [3] Heavy usage of Static classes and methods. His main argument is t…
> StackOverflow employees work from home.
Many do, but they have a fairly large office in NYC and a smaller one in London.
Re: The architecture of Stack Overflow [video]
#14Earlier quoted context omitted.
tldw; He says he doesn't advocate it but they get away with it by having the community test it out for them in their meta site. Then the community writes up the bugs.
He actually says " I'm not advocating that you shouldn't put in tests. [ The reason we can get away with this ] is that we have a great community. " I take this to mean that he feels that StackOverflow doesn't need tests. Not that tests are useless.
User community as testers presents some interesting pros and cons.
Pros:
* Tests are self-updating. Add a new feature: tests come in for free. Change a feature: tests automatically update. Fail to document a change: tests fail.
* Tests are unusually thorough
* Eventually consistent testing. If nobody ever complains, it probably wasn't a bug worth fixing.
Cons:
* Tests cannot be run offline. Feature must be committed and deployed before tests can be run.
* Potentially large quantity of false positives (bad bug reports)
* Potentially large quantity of false negatives (nobody notices particular bug, release considered good)
* Does not work for non-user-visible features
So basically you trade the reliability of your tests for a substantial build/release speedup. Some users experience each bug, but they are the users who are actively using the meta-community and have signed up to experience more bugs. Still, lack of pre-release unit testing must radically increase the importance of VERY careful code reviews.
Not the decision I would have made, but definitely has the sorts of advantages that a small team of engineers drool would drool over.
Re: The architecture of Stack Overflow [video]
#15Some points that I find interesting: [1] StackOverflow has VERY FEW tests. He says that StackOverflow doesn't use many unit tests because of their active community and heavy usage of static code. [2] Most StackOverflow employees work remotely. This is very different than a lot of companies that are now trying to force employees back into an office. [3] Heavy usage of Static classes and methods. His main argument is t…
> StackOverflow employees work from home. Many do, but they have a fairly large office in NYC and a smaller one in London.
You are correct. I edited my post and actually found a good blog post on the subject.
http://blog.stackoverflow.com/2013/02/why-we-still-believe-i...
Re: The architecture of Stack Overflow [video]
#16Some points that I find interesting: [1] StackOverflow has VERY FEW tests. He says that StackOverflow doesn't use many unit tests because of their active community and heavy usage of static code. [2] Most StackOverflow employees work remotely. This is very different than a lot of companies that are now trying to force employees back into an office. [3] Heavy usage of Static classes and methods. His main argument is t…
> StackOverflow employees work from home. Many do, but they have a fairly large office in NYC and a smaller one in London.
The Stack Overflow Q&A dev team has 2 people in New York, out of a team of 10 team. The Careers dev team is more New York heavy, 3 remote and 5 in New York. The sysadmin team is also quite remote, though I don't know the breakdown offhand.
I believe at this point most new technical hires are remote.
Our offices are mostly sales, Denver and London exclusively so.
Re: The architecture of Stack Overflow [video]
#17The most important thing, technically, is having great developers who ship. For piths sake, I want to say "Everything else is noise" but that isn't true. Everything else can help or hurt, depending on the application and how doctrinaire the application of a given approach/methodology is, the organizational knock on effects (e.g. "Mr Tough Guy Testalot" holds up the release train or nukes your architecture to make it…
Why not combine both?
Every feature/user story has to go through a workflow of selected for development -> UX design(If required) -> Development -> Unit Tests(Or the otherway around) -> Staging -> Load Test -> Acceptance Test -> Production -> Analytics(To see if people actually use it) -> Learn from analytics -> back to start if required.
The goal is to get as many issues through the workflow as fast and rigoursly(no shortcuts) as possible at a sustainable pace. Have a continuous flow of features rolling out through this process. Ideally with continuous delivery to automate the majority of it.
Re: The architecture of Stack Overflow [video]
#18Earlier quoted context omitted.
He actually says " I'm not advocating that you shouldn't put in tests. [ The reason we can get away with this ] is that we have a great community. " I take this to mean that he feels that StackOverflow doesn't need tests. Not that tests are useless.
User community as testers presents some interesting pros and cons. Pros: * Tests are self-updating. Add a new feature: tests come in for free. Change a feature: tests automatically update. Fail to document a change: tests fail. * Tests are unusually thorough * Eventually consistent testing. If nobody ever complains, it probably wasn't a bug worth fixing. Cons: * Tests cannot be run offline. Feature must be committed…
Remember that our community writes bug reports but also vets bug reports. We rarely have to deal with bad reports. Interestingly, large quantities of false negatives are a non-issue.
Re: The architecture of Stack Overflow [video]
#19Some points that I find interesting: [1] StackOverflow has VERY FEW tests. He says that StackOverflow doesn't use many unit tests because of their active community and heavy usage of static code. [2] Most StackOverflow employees work remotely. This is very different than a lot of companies that are now trying to force employees back into an office. [3] Heavy usage of Static classes and methods. His main argument is t…
I see more and more static methods and classes last 2 years maybe. It's probably more about the stateless design and less side effects, but it definitely helps garbage collection if you avoid classes at session scope or smaller. In OOP there is another pattern that helps - object pools, but it's a lot of work to get it to work correctly and it's not as efficient.
Re: The architecture of Stack Overflow [video]
#20I would love to know more about the Databases: - Are they used for different things on the sites? - Is data partitioned across tables? - Are they all SQL Server instances?
I would like to know more about this as well.
It sounds like they are all SQL Server instances. However, he made it seem like they are reproducing the schema once per site? I.e., a separate database per site rather than sharding the shared data to multiple hosts per site. Did I hear this right in the question/answer portion?