Interesting that they've moved to Lucene.net for search - I remember hearing them say that full text search is one of the reasons they were so happy with SQL Server.
Stack Exchange’s Architecture in Bullet Points
11–20 of 34 posts
Re: Stack Exchange’s Architecture in Bullet Points
#12I'd be interested in finding out why they chose to go with Windows-based web servers. Administering 12 production nodes would not be fun and doing so over remote desktop seems like a pain to me. I recently discovered puppet and I am just way too excited about what I can do with it.
This is at a VERY high level, but here you go:
In theory I think that different stacks (LAMP, Java, and Microsoft were the big contenders at the time) are similar enough in capabilities. There are a million pros and cons to each stack. But I strongly believe that how WELL you know your stack is MUCH more important than WHICH stack you use.
In other words, an experienced Java team will FAR outperform if they can use Java, and an experienced Windows team will FAR outperform if they can use Windows, and the skills of the team are much more significant than the variations between otherwise very very similar platforms. (See also http://www.joelonsoftware.com/items/2006/09/01.html).
So, when we started, Jeff was really good at Microsoft technology, so he was able to produce better code faster using Microsoft technology than if he had to learn Ruby or Python or whatever. And that was BY FAR the dominant decision point for us.
Also, the cost of Windows licenses is virtually insignificant. It's just a non-issue. Not just because of BizSpark (which we took advantage of), but because compiled C# code on Windows servers is so damn efficient you don't need very many servers. Our stack of ten web servers is SHOCKINGLY overprovisioned. They run at insanely low loads now.
Re: Stack Exchange’s Architecture in Bullet Points
#13Interesting that they've moved to Lucene.net for search - I remember hearing them say that full text search is one of the reasons they were so happy with SQL Server.
ugh, no, I can't imagine ever saying that. Full Text Search in SQL Server is very badly integrated, buggy, deeply incompetent, and I hate it. Hate it hate it hate it hate it hate it.
Re: Stack Exchange’s Architecture in Bullet Points
#14Interesting that they've moved to Lucene.net for search - I remember hearing them say that full text search is one of the reasons they were so happy with SQL Server.
ugh, no, I can't imagine ever saying that. Full Text Search in SQL Server is very badly integrated, buggy, deeply incompetent, and I hate it. Hate it hate it hate it hate it hate it.
I'm not sure where I got that impression then. Thanks for the correction!
Re: Stack Exchange’s Architecture in Bullet Points
#15Here's an earlier explanation about their caching set-up using Redis: http://meta.stackoverflow.com/questions/69164/does-stackover...
I'd be interested to know what platform they're running Redis on. Is it Windows or Linux?
Edit: Oh, the article link mentions it's running on CentOS. I was looking for that in the parent's link.
Re: Stack Exchange’s Architecture in Bullet Points
#16Congrats to them. 12 page servers and only 2 dbs is super lean for ~100m pageviews. They seem a bit bloated on eng staff though.
Off topic, but more proof how inaccurate Alexa is. I have a service that gets 1/10th of their p/v and is almost 3k points higher.
Re: Stack Exchange’s Architecture in Bullet Points
#17Interesting that they've moved to Lucene.net for search - I remember hearing them say that full text search is one of the reasons they were so happy with SQL Server.
I remember this too. Jeff Atwood made the comment in the stackoverflow blog back in 2008:
"We rely heavily on full-text search on stackoverflow.com, which worked amazingly well for us under SQL Server 2005. Looks like that’s no longer the case for SQL Server 2008, unfortunately."
Here's the entry: http://blog.stackoverflow.com/2008/11/sql-2008-full-text-sea...
Re: Stack Exchange’s Architecture in Bullet Points
#18I'd be interested in finding out why they chose to go with Windows-based web servers. Administering 12 production nodes would not be fun and doing so over remote desktop seems like a pain to me. I recently discovered puppet and I am just way too excited about what I can do with it.
StackOverflow is done in ASP.NET MVC. They were one of the earliest adopters of ASP.NET MVC. It would make sense to go with Windows servers as Mono w/ Linux may not be up-to-date with latest C#/ASP.NET code. BTW, as they are part of BizSpark program, they would have got the server licences for cheap. And I am genuinely curious as to why you think 12 Windows servers are pain compared to 12 Linux boxes?
He seems to think you have to log on via remote desktop to administer.
Re: Stack Exchange’s Architecture in Bullet Points
#19I'd be interested in finding out why they chose to go with Windows-based web servers. Administering 12 production nodes would not be fun and doing so over remote desktop seems like a pain to me. I recently discovered puppet and I am just way too excited about what I can do with it.
@Igor: For management I do feel it is a little bit more tricky since everything isn't just a text file. However, we use some Powershell, Group Policies, and WSUS as management tools (I talk about this in this post: http://blog.serverfault.com/post/1097492931/). Windows also has other management solutions I haven't gotten into yet. So although I am a believer in what Raymond refers to as Textuality in the Art of Unix Programming, it doesn't mean other options don't exist.
Re: Stack Exchange’s Architecture in Bullet Points
#20Congrats to them. 12 page servers and only 2 dbs is super lean for ~100m pageviews. They seem a bit bloated on eng staff though. Off topic, but more proof how inaccurate Alexa is. I have a service that gets 1/10th of their p/v and is almost 3k points higher.
To be fair, Alexa doesn't purport to rank by page views.