Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What does it take to make Google work at scale?

docs.google.com

Post reply on HN
Print view View original

131–134 of 134 posts

Re: What does it take to make Google work at scale?

#131

Post by chetanahuja »

jasonpeacock wrote: post #7
ska wrote: post #6

Earlier quoted context omitted.

I think it is crazy that something as simple as keeping time can have such a complicated solution at that sort of scale. Time keeping is one of the things that is most often underestimated and screwed up at any scale, in my experience. Many a subtle bug turns out to be a poor assumption or misunderstanding about how clocks work. Or calendars.

i.e. A lot of distributed systems problems can be simplified if you can trust your clocks.

But you simply can't. Not even Spanner (Google's globally distributed db that's using that fancy GPS based clock) doesn't pretend that the clock's are actually synchronized.

https://queue.acm.org/detail.cfm?id=2745385

Re: What does it take to make Google work at scale?

#132

Post by tschottdorf »

nickpsecurity wrote: post #130
sargun wrote: post #124

Earlier quoted context omitted.

You don't _NEED_ the GPS tricks and all, and realistically you can replace some of the HLC code in Cockroach with a time network time server implementation that speaks to GPS / Atomic clocks, if you want.

Good to hear :)

Note that Spanner always waits out the clock uncertainty on commit, while Cockroach doesn't need to do that - by design there are fewer situations in which the offset really matters. We can still benefit from a good bound in some situations, in particular you get global linearizability between causally unrelated transactions by waiting out the maximum clock offset (so just what Spanner does). BTW, I'm not sure when you would ever need the above guarantee, but it's cool! (disclaimer: actually working on the project).

Re: What does it take to make Google work at scale?

#133

Post by runamok »

jerf wrote: post #51
runamok wrote: post #50

Earlier quoted context omitted.

> I would not enjoy living on either one of the coasts. Have you left the tiny midwest town and know that for a fact? Or do you base that on just word of mouth, articles, or..? I have lived in a lot of different places: (New Jersey, North Carolina, Texas, Southern California, Northern California (don't laugh re: me splitting those!) and briefly in Illinoism Connecticut and the Dominican Replublic. ) and find value in…

I live in the Midwest, and have traveled frequently to Silicon Valley for work. It's a nice enough area in most ways, but I get a really nice view of it through travel-expensed meals and hotel accomodations where the commute to the office is 10 minutes of pleasent traffic. The experience the locals have involves more traffic and rather fewer trips to modestly nice restaurants. I don't hate the area by any means, but…

It's a mecca for programmers. I can go to talks on various technical topics every day of the week if I wanted to. I have spoken to the creator of Scala, the Symfony PHP framework and Optimizely. It's just really exciting (to me) to have that level of access.

Whereas my friends in Houston get to go to one conference a year (if that). But I totally agree if you want a house with land to raise kids personally I think it sucks to live here.

Re: What does it take to make Google work at scale?

#134

Post by nickpsecurity »

tschottdorf wrote: post #132
nickpsecurity wrote: post #130

Earlier quoted context omitted.

Good to hear :)

Note that Spanner always waits out the clock uncertainty on commit, while Cockroach doesn't need to do that - by design there are fewer situations in which the offset really matters. We can still benefit from a good bound in some situations, in particular you get global linearizability between causally unrelated transactions by waiting out the maximum clock offset (so just what Spanner does). BTW, I'm not sure when y…

Again, appreciate the insight.
Post reply on HN