Why are Facebook, Digg and Twitter So Hard To Scale?
highscalability.com
Why are Facebook, Digg and Twitter So Hard To Scale?
1–10 of 52 posts
Re: Why are Facebook, Digg and Twitter So Hard To Scale?
#2I spent a few hours mocking up a solution to this type of status updates in code. It seemed fairly obvious so I assumed it was not a real problem. However, if there is any interest I can turn it to some sort of blog post next weekend.
Re: Why are Facebook, Digg and Twitter So Hard To Scale?
#3I spent a few hours mocking up a solution to this type of status updates in code. It seemed fairly obvious so I assumed it was not a real problem. However, if there is any interest I can turn it to some sort of blog post next weekend.
Sure it's obvious until you have hundreds of millions of nodes that each link to somewhere between 100 and 2,000,000 other nodes that need to be updated when any given node updates (assume around 5,000 nodes are updating every second with a power law kind of distribution).
If you think that's obvious, I think you're significantly beyond all of the people with all of the "hello world" twitter clones out there.
Re: Why are Facebook, Digg and Twitter So Hard To Scale?
#4I've wondered for a while about some hybrid push/pull distribution scheme. I have 100 followers, just push. Oprah has 1M, 1% of which are currently active, pull. Sounds complicated, but feasible.
Re: Why are Facebook, Digg and Twitter So Hard To Scale?
#5scaling is hard. this is why, for the non-technical audience... also the post ends with "brave new world." yuck.
Re: Why are Facebook, Digg and Twitter So Hard To Scale?
#6I think the trade off between push/pull is influenced by the ratio of the 'velocity' of data (how frequently it changes) to its access frequency.
If accessed frequently, but changes less frequently, then push makes sense, if changes frequently, but accessed infrequently, then pull makes sense.
It also seems likely that the same piece of data may have different ratios from different perspectives.
Re: Why are Facebook, Digg and Twitter So Hard To Scale?
#7I spent a few hours mocking up a solution to this type of status updates in code. It seemed fairly obvious so I assumed it was not a real problem. However, if there is any interest I can turn it to some sort of blog post next weekend.
Sure it's obvious until you have hundreds of millions of nodes that each link to somewhere between 100 and 2,000,000 other nodes that need to be updated when any given node updates (assume around 5,000 nodes are updating every second with a power law kind of distribution). If you think that's obvious, I think you're significantly beyond all of the people with all of the "hello world" twitter clones out there.
Interestingly, the problem isn't so much in building a scalable store on a single machine that goes up to a certain point -- a billion edges, for example (about what we've tested our DB up to). Distributing the data becomes a really hard problem, however, because the issues of data locality is tricky. Typical map-reduce patterns are only of limited utility since each traversal may imply additional traversals, so you end up having to have "smart" (computing) nodes which may themselves trigger a second tier of map-reduces.
This is quite different from the problems of even large scale web apps where there's essentially a set of data that's pulled from a caching layer and assembled.
Re: Why are Facebook, Digg and Twitter So Hard To Scale?
#8I spent a few hours mocking up a solution to this type of status updates in code. It seemed fairly obvious so I assumed it was not a real problem. However, if there is any interest I can turn it to some sort of blog post next weekend.
A blog post would be great. Don't you find it odd though, that the engineers at Digg, Twitter, and Facebook think it is a very difficult problem?
Re: Why are Facebook, Digg and Twitter So Hard To Scale?
#9I think it's laughable to put Twitter and Digg in the same category of scalability as Facebook.
Maybe things have changed since the last time I visited Digg over 2 years ago, but the social networking aspects are not very significant. The vast majority of their hits are practically fully page cacheable.
Twitter at least has an interesting scaling problem, but they don't have any features and they move at a glacial pace.
Facebook on the other hand has a graph that almost as nasty as Twitter's (minus the million followers thing), but they have 100 times the features, and they push new code every week.
Re: Why are Facebook, Digg and Twitter So Hard To Scale?
#10I spent a few hours mocking up a solution to this type of status updates in code. It seemed fairly obvious so I assumed it was not a real problem. However, if there is any interest I can turn it to some sort of blog post next weekend.
Sure it's obvious until you have hundreds of millions of nodes that each link to somewhere between 100 and 2,000,000 other nodes that need to be updated when any given node updates (assume around 5,000 nodes are updating every second with a power law kind of distribution). If you think that's obvious, I think you're significantly beyond all of the people with all of the "hello world" twitter clones out there.
It all comes down how well you can slice the process. There is no cheep over the counter solution to these problems but a little custom code can go a long way.
5k node updates per second might sound like a problem, but one core of one machine can easily keep up with that so you can have several copies and several views of the whole network graph. Public vs. private messages can be handled separately and then joined before presentation to the user. You can separate finding which message to display from the message data. You even get to display dirty reads as long as the data is Ok, describing the solution based on the above insights takes some time and pictures but does it still sound horrible?
PS: Twitter was forced to morph an architecture built to solve a different problem into a working solution. That takes time and can be fairly difficult. But, starting from scratch it's not that bad.