Live data from Hacker News

StackOverflow Update: 560M Pageviews a Month, 25 Servers

highscalability.com

241–250 of 278 posts

Re: StackOverflow Update: 560M Pageviews a Month, 25 Servers

#241
post #162

Earlier quoted context omitted.

You can negotiate with AWS.

I don't believe that startups in their garage days have anywhere near the necessary leverage to negotiate anything with Amazon...

they negotiate, still. I know several startups which went to amazon, said, "hey, your competitor is cheaper, what is your value prop" and suddenly had 50% discounts, etc. It's amazing how much you get simply asking!

Re: StackOverflow Update: 560M Pageviews a Month, 25 Servers

#242

StackOverflow shows just how powerful using a fast language can be. Compare the TechEmpower benchmarks for, say, Java vs Rails on a very simple JSON serialization benchmark (we can assume .Net would achieve comparable performance to the JVM): http://www.techempower.com/benchmarks/ The Java servers get around 900K requests/s on beefy hardware while Rails squeezes out 6K. That's a 150x difference! Any real application…

> StackOverflow shows just how powerful using a fast language can be. Compare the TechEmpower benchmarks [...] (we can assume .Net would achieve comparable performance to the JVM)

Except those TechEmpower benchmarks show .NET is not nearly as fast as Java. I think StackExchange prove that the platform is NOT the most important: it's much more important to make performance a priority in all engineering decisions, to benchmark everything beforehand and develop new technology where there's no good standard solution available. (think Dapper, protobuf-net, StackExchange.Redis)

Re: StackOverflow Update: 560M Pageviews a Month, 25 Servers

#243
post #52

Earlier quoted context omitted.

There is a deliberate reason the sites are split. Spolsky talks about in depth here: http://vimeo.com/37309773 Since it's largely sociological, it's also the reason my money is on SE as opposed to Quora in this space.

Thanks - that's a great watch. Essentially they're trying to keep SE from becoming a "chat" community, and keep it focused on being a Q&A community. With Yahoo Answers, Usenets, even Subreddits, you have people having conversations within posts, which generates noise for people who were looking for answers to questions.

SE has chat forums now, integrated with the main Q&A forums.

Re: StackOverflow Update: 560M Pageviews a Month, 25 Servers

#244
post #2

I've recently started using their micro-ORM Dapper, and I like it a lot. I get the performance of hand-coded SQL, but without the tedious mapping from SqlDataReader to my entity.

Another library worth mentioning in this space is Massive, which uses .NET dynamics a lot (focusing slightly less on performance than ease of use). https://github.com/robconery/massive

Re: StackOverflow Update: 560M Pageviews a Month, 25 Servers

#245
post #229
post #209

Earlier quoted context omitted.

Conversely, you might be able to iterate faster with a 'web' language and create a product people really use in a shorter time frame and worry about scaling later!

Strange, to me Java and ASP.NET (as used by SE) are web languages.

No, they are general purpose languages that can be used for the web.

Languages like ColdFusion and PHP are web languages that don't make much sense outside of the web.

Re: StackOverflow Update: 560M Pageviews a Month, 25 Servers

#246

StackOverflow shows just how powerful using a fast language can be. Compare the TechEmpower benchmarks for, say, Java vs Rails on a very simple JSON serialization benchmark (we can assume .Net would achieve comparable performance to the JVM): http://www.techempower.com/benchmarks/ The Java servers get around 900K requests/s on beefy hardware while Rails squeezes out 6K. That's a 150x difference! Any real application…

> StackOverflow shows just how powerful using a fast language can be. Compare the TechEmpower benchmarks [...] (we can assume .Net would achieve comparable performance to the JVM) Except those TechEmpower benchmarks show .NET is not nearly as fast as Java. I think StackExchange prove that the platform is NOT the most important: it's much more important to make performance a priority in all engineering decisions, to b…

I agree with your general point, but I believe you need to choose components that have a chance of being fast if you ever want the overall system to be fast.

I didn't notice .Net in the benchmarks -- no .Net languages are listed but .Net is there as a framework. It's still x20 faster than Ruby.

Re: StackOverflow Update: 560M Pageviews a Month, 25 Servers

#247

Earlier quoted context omitted.

But you don't need to create new accounts with different passwords, and login/logout separately whenever you navigate between subreddits.

You don't need to create accounts with different passwords on SE sites either. In fact, SE has pushed for single sign-on like probably no other large site did.

But you still have to go to a different site to have any hope of getting the chance to ask your question.

  It'd be like going to Home Depot and needing information that is related to building supplies but the employees tell you you must go to a different store down the street to even ask your question.

Re: StackOverflow Update: 560M Pageviews a Month, 25 Servers

#248
post #85

Earlier quoted context omitted.

One must wonder, how much better this setup would of scaled had it been a more appropriate webserver stack such as Linux/BSD + Apache/Nginx or similar. Perhaps less boxes would be needed, or with the same number of boxes, more concurrent users. > Microsoft infrastructure works and is cheap enough not as cheap as just paying your team to maintain the boxes. I wonder how many times in the past few years SE has needed t…

> One must wonder, how much better this setup would of scaled had it been a more appropriate webserver stack such as Linux/BSD + Apache/Nginx or similar I guess you're gonna have to define "more appropriate" for me, I have a feeling we'll have a fundamental disagreement there. We could (and have) run the entirety of our peak load with 1 SQL server and 2 web servers (and not pegging them). Don't forget we run with a c…

Where I come from, it is considered bad engineering to build a product that can only run on a single platform, can only run on one particular OS, and the product is at the mercy of future decisions by a 3rd party. We go out of our way to ensure we are not dependent on any single thing.

Even if it was the best choice at the time to go the Microsoft route, it may not be in the future... however -- SE has zero choice now otherwise they'd have to re-write their entire product... that sucks as a business because you have little choice over your own product now.

Re: StackOverflow Update: 560M Pageviews a Month, 25 Servers

#249
post #222

Earlier quoted context omitted.

It would definitely be several orders of magnitude slower. That's the price you pay when you are using a language like ruby: it's very productive and fast to iterate in, but way less performant. At the end of the day, what you use depends on your needs.

Arh... I actually meant Jeff, not Joel. Hardware is cheap, Programmers are expensive! But i am sure there is a line where this crosses over though.

The point of this article is that it costs more programmers to maintain a cloud than to maintain a single master database

Re: StackOverflow Update: 560M Pageviews a Month, 25 Servers

#250
post #247

Earlier quoted context omitted.

You don't need to create accounts with different passwords on SE sites either. In fact, SE has pushed for single sign-on like probably no other large site did.

But you still have to go to a different site to have any hope of getting the chance to ask your question. It'd be like going to Home Depot and needing information that is related to building supplies but the employees tell you you must go to a different store down the street to even ask your question.

On Reddit you have to post separately on different subreddits, there's no way to simply "post to Reddit" either.

I don't agree with your analogy; I think it's more akin to going to Home Depot to ask for vegetables. I think it makes perfect sense to have a separation between Cooking.SE, Photography.SE, Christianity.SE, etc.

There are a few edge cases, primarily in the tech sites, but I don't think those disprove the model.

Post reply on HN