Earlier quoted context omitted.
For someone who took $ x million "off the table," during those funding rounds the hypocrisy is impressive.
Since when did "experience" == "hypocrisy"?
Why does digg need so many servers?
61–65 of 65 posts
Re: Why does digg need so many servers?
#62The difference is that digg has network information (IE: Followers). Stack overflow doesn't. Think through the ramifications of each of these page flows: 1. Give me this question and everybody who commented on it. 2. Give me every post that this user is following has made over the last n days. That's why.
That would increase the load on the database. Why does that increase the load on the webservers?
There are a couple of ways you can go about this, the first is the database: Join the network of people against the land of content and bring it back. This doesn't work (as an aside this is what people mean when they say web scale, it has nothing to do with web traffic, it's social graphs) your database will cry. All though not at first, in development it works fine, and you feel fine, and for a while you're ok, but you start growing....
Another way you can go about it is by denormalizing. In this world you store a pointer to each content item for each user. So anytime I do something all the people [following|watch|connected|friended] to me get a record indicating I did this. This works, but now you have lots of data (lots and lots of data!) spread all crazy around. You need some kind of system to push that data out to everybody. It's those last two that drive up your hardware usage, it's not necessarily web boxes, but it's boxes in the background broadcasting the events out to the world, and the datastores to hold it all. Depending on how your web code works you could also have a lot of overhead on the webservers putting all that stuff together.
My experience here comes from building the social features into toolbox.com. A good example is this page http://it.toolbox.com/people/george_krautzel/posts-connectio... That's all the posts from users connected to our CEO (all 750k of them). Getting that to return in near real time is super fun (and you can probably tell that I went down the DB join path before it all fell apart).
Re: Why does digg need so many servers?
#63Earlier quoted context omitted.
Absolutely. Every time someone trots out "And Reddit only has 7 guys", I think to myself that they're innovating as if they had 3. Say what you will about Digg, but at least they rolled out new software.
Say what you will about Digg, but at least they rolled out new software. Digg is a discussion site, not an operating system -- what is the value in "rolling out new software"? Digg's value is in the quality of the discussion and community; "innovation" that doesn't improve that is pointless. Look at HN: it remains nearly as minimalist as always, and mostly receives occasional tweaks and performance improvements.
Personally, I'm in general agreement with you. And you can rebase the whole argument: take MetaFilter, with 4 people on the payroll (and not all technical). What is everyone at Reddit doing?
Re: Why does digg need so many servers?
#64We use PHP and Apache, so the gloating about ASP being way more efficient than PHP seems to be unfounded.
Re: Why does digg need so many servers?
#65Earlier quoted context omitted.
Not to mention a basically non-existent cachability profile, and an order of magnitude more functionality.
I'm assuming you mean Facebook has more functionality and no cachebility. That might be true but I wonder how expensive getting a Digg discussion(with 100-200 posts) page compared to say a 5 answer SO question or even just a simple Facebook profile.