Live data from Hacker News

StackOverflow Update: 560M Pageviews a Month, 25 Servers

highscalability.com

121–130 of 278 posts

Re: StackOverflow Update: 560M Pageviews a Month, 25 Servers

#122
post #66
post #50

Earlier quoted context omitted.

+1 for Dapper - I'm a big fan. It is very fast and we use it to serve our live (mostly read-only) traffic, while using something like Entity Framework for the backend Administrative UI, which has a lot of inserts and updates.

There's a couple of user-contrib projects that do similar things for inserts & updates. Dapper Extensions is the one I'm using. https://github.com/tmsmith/Dapper-Extensions

I wrote a Dapper extension for working with SQL Server's geospatial queries and types a couple of years ago - have they added anything like that yet? Otherwise I'd be happy to add it.

Re: StackOverflow Update: 560M Pageviews a Month, 25 Servers

#123
post #18

Anyone else not quite expecting StackExchange to be using a Microsoft stack?

Joel worked at MS on office, Excel if I recall correctly. Jeff's blog is named for a feature from a book published by MS. I would be more surprised if it was MS free.

You mean to tell me Joel is partially responsible for the abomination known as Excel 2007 where I can't pull-drag the damn window from screen to screen (but all of the other Office 2007 products do!)

Re: StackOverflow Update: 560M Pageviews a Month, 25 Servers

#124

So this is what, a 2000 request/sec peak? Over 11 servers, that's like 200 requests/sec peak per frontend? The problem with scale-up is if you actually have to get a few times larger, it becomes super expensive. But fortunately hardware is increasing so much that you can probably just get away with it now. There's probably a crossover point we're rapidly approaching where even global-scale sites can just do all their…

Not quite sure how you derive a peak figure from a monthly average; but on average it's more like 560000000/(30 * 24 * 3600) = 216 req/s

I took 4 hours at 20 days to get the peak, figuring it probably won't be worse than that. The average over the month isn't a number you'll see in practice. Most perf engineering work seems to go to keep things working nicely while at peak. (And making sure you have good 95th percentile times.)

Re: StackOverflow Update: 560M Pageviews a Month, 25 Servers

#125
post #3

Makes me wonder why smaller website teams need dozens of engineers to keep their infinitesimally smaller app running.

Rigid adherence to certain technologies could be a culprit. "We're a Ruby shop!" is fine for problems where Ruby is great, but the limitations of the technology are bound to creep up somewhere. Not using a polyglot approach would be bad.

It can also be caused by too much embrace of polyglot tools. "We use the best tools for the job" can easily mean an architecture with Redis, Memcached, Mongo, MySQL, Angular, JQuery, Node, and several dozen third-party libraries, all glued together with Thrift or JSON-RPC. If all you're doing is a messaging app, you might be much better off with a single in-process Java app.

Usually you want your solution to be just right for the problem; solve the essential complexity but don't introduce any accidental complexity.

Re: StackOverflow Update: 560M Pageviews a Month, 25 Servers

#126
post #53

Earlier quoted context omitted.

You get +100 rep bonus for linking your accounts, so you shouldn't have a problem being allowed to comment.

You do need to have more than 200 rep on one site though.

Which one? I'm a full member on all the StackExchange sites I care about for now with the +100 cross-membership bonus.

Re: StackOverflow Update: 560M Pageviews a Month, 25 Servers

#127
post #18

Anyone else not quite expecting StackExchange to be using a Microsoft stack?

Jeff Atwood hacked prototype and next several versions, and since he's a .net guy (and ruby guy now), that's why .net was picked. Lot about tech stack planning can be heard in first few SO podcasts, don't even know if there're still available online somewhere

Re: StackOverflow Update: 560M Pageviews a Month, 25 Servers

#128
post #87
post #27

Earlier quoted context omitted.

Anyone who knows Joel Spolsky would expect a MS stack :)

It's difficult to take pragmatic advice from someone who took the least pragmatic (and most expensive) route for a startup web company.

Why does everyone in Silicon Valley run with expensive Apple hardware when Linux can run on any CPU?

Re: StackOverflow Update: 560M Pageviews a Month, 25 Servers

#129
post #88

Earlier quoted context omitted.

Sometimes "efficient" and "easy to understand and be sure is correct" don't have to be mutually exclusive; see this example [1] of Java and Go. Note that the code is autogenerated, so it should be equally efficient. The Go version also happens to be very simple and no different than most humans would write by hand (without trying very hard to optimize). [1] https://gist.github.com/shurcooL/9f94bbd021b4693cf584

It's not clear to me what argument the example is supporting. The Go code is very concise and the Java code is very verbose. The Go code does look more efficient but what is ImpressionData? Where is it checking the string is valid utf8?

Why would you send a string which is not valid utf8 over the wire in your system? That's the kind of validation that should probably be done prior to that. Or if it hasn't, you can do the validation manually on the receiver. There's no reason to incur the cost on each transmission.

Re: StackOverflow Update: 560M Pageviews a Month, 25 Servers

#130
post #87
post #27

Earlier quoted context omitted.

Anyone who knows Joel Spolsky would expect a MS stack :)

It's difficult to take pragmatic advice from someone who took the least pragmatic (and most expensive) route for a startup web company.

If you have extensive Microsoft experience and take advantage of something like BizSpark* (although one suspects with their connections they wouldn't have needed that) it's hard to see how it wouldn't be plenty pragmatic and inexpensive.

* Currently three years of free licenses, all software you download during that time is free forever, and discounted MSDN subs thereafter.

Post reply on HN