Why does digg need so many servers? • Hacker News Board

Re: Why does digg need so many servers?

#31

Post by ergo98 » Wed, Oct 13, 2010, 4:42 PM UTC

tibbon wrote: post #11

noodle wrote: post #9

Earlier quoted context omitted.

reddit uses something like 250-300 EC2 nodes.

Unless they are using the really big instances though, I find that EC2 nodes are generally lower in power compared to what most people are using for their servers. I remember not too long ago before they moved to EC2 everything fit in just a few racks. Of course, their traffic has expanded significantly recently.

EC2 nodes are terribly underpowered, particularly in the I/O realm (the biggest weakness of virtually every site) where a standard desktop absolutely annihilates the I/O performance of an S3 connection (where you have to perform software RAID to get anywhere near the performance of a single magnetic low-end SATA drive, which is absolutely terrible).

A lot of the "web scale" noise has come from dealing with these gerbil sized instances.

Re: Why does digg need so many servers?

#32

Post by jasonlotito » Wed, Oct 13, 2010, 4:42 PM UTC

Goladus wrote: post #21

Where does the "500 servers" number come from? Are they compute nodes? Are they webservers(5)? Webservers and appservers(10)? Web, app, database(12)? Web, app, database, back-end processing(15)? How much redundancy is built in(30)? Staging environments(60)? Development environments(120)? Standard IT infrastructure like DNS and email(125)? "We run over 500 servers" sounds a lot like "we have 500 servers in our datacen…

Pretty sure those numbers discount serving up Digg information out to every blog that includes a Digg button as well. Pretty sure if SO adding something like it, they'd be running on more than 5 servers.

Re: Why does digg need so many servers?

#33

Post by alnayyir » Wed, Oct 13, 2010, 4:42 PM UTC

jswinghammer wrote: post #4

My last job served around 30-40 million pages a month on 4 servers that used ASP.Net and SQL Server 2000. The only server whose CPU utilization was over 20% was the database server. I'm not sure the CPU utilization on the web servers ever went above 10%. A friend of mine's company has a tough time keeping up with 500,000 page views a month using more equipment. They're using PHP and MySQL.

This is meaningless without the necessary information to evaluate the situations.

Apples and orangoutangs.

Re: Why does digg need so many servers?

#34

Post by ojbyrne » Wed, Oct 13, 2010, 4:43 PM UTC

rythie wrote: post #28

ojbyrne wrote: post #25

Earlier quoted context omitted.

Where is Kevin advising people to take less money? If you mean the $80 million comment, that was almost certainly before the Series C, so digg had less than half the funding.

http://www.zdnet.com/blog/weblife/kevin-rose-10-tips-for-ent... point 4

For someone who took $ x million "off the table," during those funding rounds the hypocrisy is impressive.

Re: Why does digg need so many servers?

#35

Post by jswinghammer » Wed, Oct 13, 2010, 4:45 PM UTC

rythie wrote: post #17

jswinghammer wrote: post #4

My last job served around 30-40 million pages a month on 4 servers that used ASP.Net and SQL Server 2000. The only server whose CPU utilization was over 20% was the database server. I'm not sure the CPU utilization on the web servers ever went above 10%. A friend of mine's company has a tough time keeping up with 500,000 page views a month using more equipment. They're using PHP and MySQL.

Apple and Oranges comparison really. - What do the sites do? - How optimised are they? - How much static/cachable content is there? Really there is too little data for this comparison.

Both are simple sites by any definition. The database queries for almost all the pages views are clustered index seeks (on SQL server) and in MySQL they are mostly looking up on the primary keys for the tables used (these are MyISAM tables so no clustered indexes here).

Both are equally unoptimized outside of the database queries which I believe are fairly well optimized.

No real cacheble content on either outside of the usual suspects (JS, CSS, some images)

Re: Why does digg need so many servers?

#36

Post by jswinghammer » Wed, Oct 13, 2010, 4:47 PM UTC

thirsteh wrote: post #10

jswinghammer wrote: post #4

My last job served around 30-40 million pages a month on 4 servers that used ASP.Net and SQL Server 2000. The only server whose CPU utilization was over 20% was the database server. I'm not sure the CPU utilization on the web servers ever went above 10%. A friend of mine's company has a tough time keeping up with 500,000 page views a month using more equipment. They're using PHP and MySQL.

So, do you get free coffee in the cantina at Microsoft?

I'm not sure what you are suggesting?

Free coffee would be enough to get me to say something nice about Microsoft products?

I work at Microsoft?

Re: Why does digg need so many servers?

#37

Post by sfrench » Wed, Oct 13, 2010, 4:47 PM UTC

Goladus wrote: post #21

Where does the "500 servers" number come from? Are they compute nodes? Are they webservers(5)? Webservers and appservers(10)? Web, app, database(12)? Web, app, database, back-end processing(15)? How much redundancy is built in(30)? Staging environments(60)? Development environments(120)? Standard IT infrastructure like DNS and email(125)? "We run over 500 servers" sounds a lot like "we have 500 servers in our datacen…

I used to work at digg as a engineer. I won't say your distribution numbers are correct, but you've got the gist. There are multiple environments, reserved capacity, research and testing machines, etc.

Re: Why does digg need so many servers?

#38

Post by nkassis » Wed, Oct 13, 2010, 4:48 PM UTC

dasil003 wrote: post #19

ceejayoz wrote: post #14

Earlier quoted context omitted.

Facebook has a lot more than 100 times digg's registered users, though.

Not to mention a basically non-existent cachability profile, and an order of magnitude more functionality.

I'm assuming you mean Facebook has more functionality and no cachebility. That might be true but I wonder how expensive getting a Digg discussion(with 100-200 posts) page compared to say a 5 answer SO question or even just a simple Facebook profile.

Re: Why does digg need so many servers?

#39

Post by sfrench » Wed, Oct 13, 2010, 4:48 PM UTC

ojbyrne wrote: post #24

Where does the "500" come from?

Kevin mentioned it somewhere a few weeks ago. I forget where though.

Re: Why does digg need so many servers?

#40

Post by jshen » Wed, Oct 13, 2010, 4:53 PM UTC

thecoffman wrote: post #20

tibbon wrote: post #3

Just as perplexing, why do they need ~68 employees ( http://about.digg.com/team ). My counting may be off by 1-2, but last I checked Reddit is coming up on them traffic-wise ( http://siteanalytics.compete.com/reddit.com+digg.com/ ), and only has 7 employees? And I'm pretty sure Reddit doesn't have 500 servers either, although I could be wrong. Both sites offer very similar functionality. While past its peak, Slashdot…

Yes, it blows my mind that digg needs so many employees. Reddit actually is larger than digg ( http://www.readwriteweb.com/archives/reddit_to_mainstream_me... ), and until recently had even less than 7 employees. If I'm not mistaken they have 3 programmers and a systems admin. Kind of makes you wonder what all the digg employees do all day.

"Kind of makes you wonder what all the digg employees do all day."

Communication. I'm always surprised that so few people who do software (I'm talking management at companies) are familiar with the mythical man month. Adding employees to a software project always yields diminishing returns due to the increase in communication required.