Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why are Facebook, Digg and Twitter So Hard To Scale?

highscalability.com

Post reply on HN
Print view View original

31–40 of 52 posts

Re: Why are Facebook, Digg and Twitter So Hard To Scale?

#31

Post by thomaspaine »

radu_floricica wrote: post #25

I've been wondering for a while why we don't see more often systems with a huge amount of RAM and asynchronous persistent updates, with a more, how should I put it, computer science-ish architecture inside. MVC works ok, but does it really mean we _have_ to use it always? Is it so hard to make system with a TeraByte of RAM? Why even use memcache? Why not make a clear decision that stuff like user statuses will never…

User statuses should never touch hard disk? What happens when your server restarts and everything you were storing in memory disappears?

Re: Why are Facebook, Digg and Twitter So Hard To Scale?

#32

Post by dasil003 »

mikeryan wrote: post #29
dasil003 wrote: post #9

I think it's laughable to put Twitter and Digg in the same category of scalability as Facebook. Maybe things have changed since the last time I visited Digg over 2 years ago, but the social networking aspects are not very significant. The vast majority of their hits are practically fully page cacheable. Twitter at least has an interesting scaling problem, but they don't have any features and they move at a glacial pa…

Just because Twitter and Digg don't operate at the same scale as Facebook doesn't mean they don't face similar issues. I actually think having a bunch of large sites tackling similar issues will present a whole bunch of neat new technologies for addressing these sorts of issues in the near future.

I'm not talking about absolute scale. Facebook is small compared to Google scale, but in my (probably poorly) informed opinion it's not as impressive.

All these sites have scaled to multiple servers, which means they've all had to address some of the hard problems, and no doubt that many of the issues are very similar. However even if Facebook's traffic were half of Twitter's it would still be a much more impressive architecture. They support a full-on integrated development platform for crying out loud. That is orders of magnitude more impressive than Twitter's lightweight API whether they're serving 100 or 100,000 requests per second.

I'm not dissing Twitter either, it's just that what Facebook has going is pretty incredible. Consider that failing to scale is basically what killed Friendster.

Re: Why are Facebook, Digg and Twitter So Hard To Scale?

#33

Post by peterwwillis »

radu_floricica wrote: post #25

I've been wondering for a while why we don't see more often systems with a huge amount of RAM and asynchronous persistent updates, with a more, how should I put it, computer science-ish architecture inside. MVC works ok, but does it really mean we _have_ to use it always? Is it so hard to make system with a TeraByte of RAM? Why even use memcache? Why not make a clear decision that stuff like user statuses will never…

You can't store everything in memory all the time. There simply isn't a computer built which can do that. Horizontal scaling with commodity boxes and layers of cache are the most cheap and effective method of returning a high rate of requests as quickly as possible.

Re: Why are Facebook, Digg and Twitter So Hard To Scale?

#34

Post by spc476 »

thomaspaine wrote: post #31
radu_floricica wrote: post #25

I've been wondering for a while why we don't see more often systems with a huge amount of RAM and asynchronous persistent updates, with a more, how should I put it, computer science-ish architecture inside. MVC works ok, but does it really mean we _have_ to use it always? Is it so hard to make system with a TeraByte of RAM? Why even use memcache? Why not make a clear decision that stuff like user statuses will never…

User statuses should never touch hard disk? What happens when your server restarts and everything you were storing in memory disappears?

The greylist daemon I wrote (http://www.x-grey.com/) keeps everything in memory, with a checkpoint (everything gets written out to disk) every hour (the main process forks, and it's the child that does the writing). A similar approach might be applicable here.

It could very well be a RAM based approach might be viable once memristros (http://en.wikipedia.org/wiki/Memristor#Potential_application...) are used for memory.

Re: Why are Facebook, Digg and Twitter So Hard To Scale?

#35

Post by mikeryan »

dasil003 wrote: post #32
mikeryan wrote: post #29

Earlier quoted context omitted.

Just because Twitter and Digg don't operate at the same scale as Facebook doesn't mean they don't face similar issues. I actually think having a bunch of large sites tackling similar issues will present a whole bunch of neat new technologies for addressing these sorts of issues in the near future.

I'm not talking about absolute scale. Facebook is small compared to Google scale, but in my (probably poorly) informed opinion it's not as impressive. All these sites have scaled to multiple servers, which means they've all had to address some of the hard problems, and no doubt that many of the issues are very similar. However even if Facebook's traffic were half of Twitter's it would still be a much more impressive…

I understand what you're saying Facebook is massively complex compared to digg and twitter - which is likely why it chews up the majority of the article. But I'm not sure why you can't have a top level discussion about scaling (which seems to be the point of the article) that is relevant to all three.

Re: Why are Facebook, Digg and Twitter So Hard To Scale?

#36

Post by dasil003 »

mikeryan wrote: post #35
dasil003 wrote: post #32

Earlier quoted context omitted.

I'm not talking about absolute scale. Facebook is small compared to Google scale, but in my (probably poorly) informed opinion it's not as impressive. All these sites have scaled to multiple servers, which means they've all had to address some of the hard problems, and no doubt that many of the issues are very similar. However even if Facebook's traffic were half of Twitter's it would still be a much more impressive…

I understand what you're saying Facebook is massively complex compared to digg and twitter - which is likely why it chews up the majority of the article. But I'm not sure why you can't have a top level discussion about scaling (which seems to be the point of the article) that is relevant to all three.

okay fair point.

Re: Why are Facebook, Digg and Twitter So Hard To Scale?

#37

Post by pavs »

dasil003 wrote: post #32
mikeryan wrote: post #29

Earlier quoted context omitted.

Just because Twitter and Digg don't operate at the same scale as Facebook doesn't mean they don't face similar issues. I actually think having a bunch of large sites tackling similar issues will present a whole bunch of neat new technologies for addressing these sorts of issues in the near future.

I'm not talking about absolute scale. Facebook is small compared to Google scale, but in my (probably poorly) informed opinion it's not as impressive. All these sites have scaled to multiple servers, which means they've all had to address some of the hard problems, and no doubt that many of the issues are very similar. However even if Facebook's traffic were half of Twitter's it would still be a much more impressive…

"However even if Facebook's traffic were half of Twitter's"

Huh?

Re: Why are Facebook, Digg and Twitter So Hard To Scale?

#38

Post by dasil003 »

pavs wrote: post #37
dasil003 wrote: post #32

Earlier quoted context omitted.

I'm not talking about absolute scale. Facebook is small compared to Google scale, but in my (probably poorly) informed opinion it's not as impressive. All these sites have scaled to multiple servers, which means they've all had to address some of the hard problems, and no doubt that many of the issues are very similar. However even if Facebook's traffic were half of Twitter's it would still be a much more impressive…

"However even if Facebook's traffic were half of Twitter's" Huh?

"only half"

Re: Why are Facebook, Digg and Twitter So Hard To Scale?

#39

Post by radu_floricica »

thomaspaine wrote: post #31
radu_floricica wrote: post #25

I've been wondering for a while why we don't see more often systems with a huge amount of RAM and asynchronous persistent updates, with a more, how should I put it, computer science-ish architecture inside. MVC works ok, but does it really mean we _have_ to use it always? Is it so hard to make system with a TeraByte of RAM? Why even use memcache? Why not make a clear decision that stuff like user statuses will never…

User statuses should never touch hard disk? What happens when your server restarts and everything you were storing in memory disappears?

Exactly how often do servers restart? Your average commodity server - maybe twice a year. For user data it's too much, but for user statuses, I don't think so. And if you save the data in a controlled restart and only consider crashes, with a bit of effort you can probably go down to about once every couple of years or less.

As for the benefit... there was a picture some time back here on HN: http://imgur.com/X1Hi1.gif

Re: Why are Facebook, Digg and Twitter So Hard To Scale?

#40

Post by rythie »

dasil003 wrote: post #9

I think it's laughable to put Twitter and Digg in the same category of scalability as Facebook. Maybe things have changed since the last time I visited Digg over 2 years ago, but the social networking aspects are not very significant. The vast majority of their hits are practically fully page cacheable. Twitter at least has an interesting scaling problem, but they don't have any features and they move at a glacial pa…

Actually I've found that people follow twice as many people on Twitter as on Facebook and receive updates a whole load more often. Facebook does benefit from being a older company with a lot more engineers though.
Post reply on HN