Live data from Hacker News

How Google makes Google(+) fast

plus.google.com

11–20 of 33 posts

Re: How Google makes Google(+) fast

#11
post #6

Are people outside of Google using closure to develop apps? I used the closure compiler with Jammit but I've never tried the library.

clojure-script uses closure as part of compiling clojure to js. I'm not sure if any significant projects are using it though:

https://github.com/clojure/clojurescript

Re: How Google makes Google(+) fast

#12
post #11
post #6

Are people outside of Google using closure to develop apps? I used the closure compiler with Jammit but I've never tried the library.

clojure-script uses closure as part of compiling clojure to js. I'm not sure if any significant projects are using it though: https://github.com/clojure/clojurescript

Google Closure Tools is not the same thing as Clojure.

A little confusing I am sure since Closure Templates is a templating system that dynamically generates HTML in Java and JavaScript... And Clojure is a language that targets the JVM.

I have no experience with Closure Tools but Clojure is awesome definitely.

Re: How Google makes Google(+) fast

#13

Google+ is certainly fast, I just wish it were quicker about showing me valid/updated state. I've never used any website where the data I was seeing was so obviously out of date across the board as it is on Google+. Yes, eventually it coalesces towards correctness, but I don't think this model works well for things like notifications and such. "Oh look I have a new notification... oh wait, no I don't, that's from lik…

That's because most distributed data stores are eventually consistent. This works fabulously in a world where data needs to exist in a handful of places (say, a user's email inbox, which should be replicated so it can be accessed quickly from different geographical areas and for the sake of disaster recovery) but need not necessarily all agree with each other at the same time (If I send you an email, it's OK if you don't get it for 5 or 10 minutes, in fact, it's generally expected).

The problem with social networking is that such silos don't exist anymore. You can't even group users into larger silos with their cliques, because most social circles are overlapping (for some reason people feel compelled to keep making friends throughout their lives). So instead of the trivial email case of caching, you're fighting this battle where you have to constantly trade off consistency (do we synchronously write to every cache in every cluster in every region?) with performance (obviously not, round trip time to sweden is like 200 ms).

And you're thinking "Well, a consistent user experience is the most important thing, so just block the user's notification until the data is available to them." Well how do you know the data is available to them? They might see the notification on their cell phone, which is routed to the east coast for DNS reasons, but the write hasn't made it over to the west coast for whatever reason yet, and they're accessing it on their PC. If you wait for it to appear everywhere, two users sitting beside each other in a dorm room think that Facebook sucks because one guy posted on the other's ten minutes ago and it's not there yet. This isn't a logical consequence to the end user of a backhoe cutting a fiber line somewhere in oregon.

To summarize, it's a super hard problem, which makes it incredibly interesting to work on :) https://www.facebook.com/careers

Re: How Google makes Google(+) fast

#14

Google+ is certainly fast, I just wish it were quicker about showing me valid/updated state. I've never used any website where the data I was seeing was so obviously out of date across the board as it is on Google+. Yes, eventually it coalesces towards correctness, but I don't think this model works well for things like notifications and such. "Oh look I have a new notification... oh wait, no I don't, that's from lik…

Facebook is pretty amazing in that respect. They managed to engineer a scaleable infrastructure without making it too obvious that everything is being cached. There are times when it takes a minute or two for my profile picture to change everywhere after I change it in my profile, but aside from minute-or-two scenarios, Facebook's interface (and the real-time notification system) is just amazing in comparison.

Facebook runs largest memcached cluster in the world and largest sharded MySQL installation in the world (which they use as a key-value store since it's hard to evolve schema otherwise). So, essentially, quick updates are tremendous amount of duct tape and hand-crafted code to make it work more or less correctly.

Re: How Google makes Google(+) fast

#15

Google+ is certainly fast, I just wish it were quicker about showing me valid/updated state. I've never used any website where the data I was seeing was so obviously out of date across the board as it is on Google+. Yes, eventually it coalesces towards correctness, but I don't think this model works well for things like notifications and such. "Oh look I have a new notification... oh wait, no I don't, that's from lik…

That's because most distributed data stores are eventually consistent. This works fabulously in a world where data needs to exist in a handful of places (say, a user's email inbox, which should be replicated so it can be accessed quickly from different geographical areas and for the sake of disaster recovery) but need not necessarily all agree with each other at the same time (If I send you an email, it's OK if you d…

Wow, thanks for the writeup Alex! I never though about the DNS routing issues on mobile that would send the same packets on two completely different paths (all to end up in the same dorm room). All of this on top of the ticker / chat / live connections between 750 million people is pretty insane. I don't think Facebook gets enough technical credit for overcoming all of those challenges.

Makes me excited for my interview on Friday!

Re: How Google makes Google(+) fast

#16

Google+ is certainly fast, I just wish it were quicker about showing me valid/updated state. I've never used any website where the data I was seeing was so obviously out of date across the board as it is on Google+. Yes, eventually it coalesces towards correctness, but I don't think this model works well for things like notifications and such. "Oh look I have a new notification... oh wait, no I don't, that's from lik…

Facebook is pretty amazing in that respect. They managed to engineer a scaleable infrastructure without making it too obvious that everything is being cached. There are times when it takes a minute or two for my profile picture to change everywhere after I change it in my profile, but aside from minute-or-two scenarios, Facebook's interface (and the real-time notification system) is just amazing in comparison.

Relevant: http://gigaom.com/cloud/facebook-trapped-in-mysql-fate-worse...

I'm not so sure that using mysql at this scale is in the end such a good decision...

Re: How Google makes Google(+) fast

#17
post #11

Earlier quoted context omitted.

clojure-script uses closure as part of compiling clojure to js. I'm not sure if any significant projects are using it though: https://github.com/clojure/clojurescript

Google Closure Tools is not the same thing as Clojure. A little confusing I am sure since Closure Templates is a templating system that dynamically generates HTML in Java and JavaScript... And Clojure is a language that targets the JVM. I have no experience with Closure Tools but Clojure is awesome definitely.

ClojureScript uses Closure Tools: https://github.com/clojure/clojurescript/wiki/Google-Closure

Re: How Google makes Google(+) fast

#18

Google+ is certainly fast, I just wish it were quicker about showing me valid/updated state. I've never used any website where the data I was seeing was so obviously out of date across the board as it is on Google+. Yes, eventually it coalesces towards correctness, but I don't think this model works well for things like notifications and such. "Oh look I have a new notification... oh wait, no I don't, that's from lik…

Facebook is pretty amazing in that respect. They managed to engineer a scaleable infrastructure without making it too obvious that everything is being cached. There are times when it takes a minute or two for my profile picture to change everywhere after I change it in my profile, but aside from minute-or-two scenarios, Facebook's interface (and the real-time notification system) is just amazing in comparison.

I've heard (from a coworker that knows some people that work at FB; don't take this as gospel) they use some clever trickery to get this to work. Basically, there's a localized cache of information just for you, storing information entered by you that you might see. Since this is a limited set and is easily sharded by user, you can put everything in memcached, have the frontends hit that record and merge it with the rest of the data, and pretty much guarantee immediate consistency - as long as it's just you looking at the profile.

When other people look at your profile, they have to go through the normal eventual-consistency mechanisms, deal with the normal replication lag and message-passing delays, etc. But they have no idea that the information is out of date, because they're not the ones who inputted it. As long as you don't go ask someone through backchannels "did you see my FB update?", they'll never be the wiser.

FWIW, I've seen data-consistency problems on my FB wall before where the same entry will appear multiple times, usually due to pagination bugs.

Re: How Google makes Google(+) fast

#19
"On a side note, you may have noticed that we load our CSS via a XHR instead of a style tag - that is not for optimization reasons, that’s because we hit Internet Explorer’s max CSS selector limit per stylesheet!"

Can anyone go into more detail about what they're talking about here? I didn't realize IE had a "CSS selector limit." Seems like a funny thing to mention when you're talking about how optimized your site is.

Re: How Google makes Google(+) fast

#20

"On a side note, you may have noticed that we load our CSS via a XHR instead of a style tag - that is not for optimization reasons, that’s because we hit Internet Explorer’s max CSS selector limit per stylesheet!" Can anyone go into more detail about what they're talking about here? I didn't realize IE had a "CSS selector limit." Seems like a funny thing to mention when you're talking about how optimized your site is…

Yes, the magic number is 4095 selectors. And it's still true, even in IE9.
Post reply on HN