Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What does it take to make Google work at scale?

docs.google.com

Post reply on HN
Print view View original

91–100 of 134 posts

Re: What does it take to make Google work at scale?

#91

Post by animefan »

threeseed wrote: post #67
segmondy wrote: post #60

Earlier quoted context omitted.

There are plenty of other companies that also have tech that Google doesn't have or use too. Google has all these amazing tech, but I can assure you working at Google doesn't mean you will ever get to use it directly.

Exactly. Facebook and Yahoo have contributed equally as much to the technology landscape as Google has. And unlike Google they actually make it available to the public and then follow it up with supporting the projects.

A lot of these things were originally developed at Google then spread to other companies (when employees moved etc.). So Google is contributing probably more, but not directly.

Re: What does it take to make Google work at scale?

#92

Post by peterwwillis »

Keep in mind this is really "How does Google (and other big companies) do it", not "what does it take", because what it actually takes is much simpler than custom hardware and custom software. This was all pretty accessible 15 years ago using available software; the trick was putting it together right.

In general, the maxim of "cache everything, everywhere" is the cheapest way to gain both speed and availability. Of course you need to have results already to cache them, which is where all that big data processing comes into play, but there's still way more optimization going on here than is necessary to get results like this. You could make a tradeoff and give people newly-processed results more slowly and still give them mostly what they want, and not have to maintain so much custom software+hardware infrastructure.

Sometimes I feel like Batty from Blade Runner when I think about how well old crappy software used to work. "I've seen things you people wouldn't believe. Attack ships on fire off the shoulder of Orion. I watched mod_perl apps serve 50 thousand dynamic requests per second with nothing but Apache and MySQL. All those moments will be lost in time... like tears in rain..."

Re: What does it take to make Google work at scale?

#93

Post by tomcam »

untog wrote: post #19
munificent wrote: post #17

Earlier quoted context omitted.

> Every review I hear about working at Google makes me want to stay away Google was Fortune Magazine's #1 best place to work for six years in a row. The reason you occasionally hear a story about how Google isn't perfect to work at is because those are news . In general, it's a fantastic workplace.

Everyone is different - while I don't doubt that Google has fantastic perks, I can't think of many things that I would want to work on that aren't the moonshot crazy experimental projects (and, let's face it, most people won't be working on them). And working on a giant campus that's isolated from the outside world... no thank you. But like I said, we're all very different.

> But like I said, we're all very different.

Actually, I am indistinguishable from everybody else.

Re: What does it take to make Google work at scale?

#95

Post by hueving »

pauleastlund wrote: post #26
tikhonj wrote: post #16

Earlier quoted context omitted.

It does for me, at least. I'd absolutely hate having my job satisfaction determined so much by a manager and team that I was assigned . (Ideally I wouldn't want it impacting much at all , but truly flat companies are still hard to find, despite the rhetoric.) It works for some people, but it won't work for me.

I spent about 5 years at Google starting in 2006. When I arrived I was assigned to one manager, but I wasn't _super_ excited about the project. A week in, another manager offered me another opportunity and got me permission to transfer. Problem solved. A couple years later, a new hire got assigned to a team I was on. He was a little bummed because he'd really had his eye on another project. So we talked to our manage…

>I have more anecdotes like this, but the long and short of it is that in my experience Google is a lot less capricious and uncaring an organization than you imagine.

I think the problem is that their interviewing process is. That's the first thing people encounter and it's the only thing rejected people encounter. That harms people's views of the company, even if they understand the explanation about false positives being so expensive.

Re: What does it take to make Google work at scale?

#96

Post by hueving »

mahyarm wrote: post #48
drcross wrote: post #20

Earlier quoted context omitted.

After 9 different interview rounds I've all but giving up on trying to get into there. Any advice on trying to get in?

Master Cracking the Coding Interview? Have a positive attitude?

Right, spend all of your free time cramming on algo and data structure questions that you weren't using before and probably wont use at Google.

Re: What does it take to make Google work at scale?

#97

Post by bagels »

chetanahuja wrote: post #65
AnimalMuppet wrote: post #58

Earlier quoted context omitted.

If the two clocks are stationary with respect to each other, that isn't a problem. Most of Google's servers are on the Earth's surface, so... (Edit: Yes, different elevations cause a gravitational time dilation difference. For Earth's gravitational field and the elevation difference between different Google servers, I doubt it's an issue at the time resolution that Google needs to maintain.)

Even if the clocks are stationary with respect to each other (within some tolerance), it's impossible to guarantee completely synchronized clocks among the systems. This follows trivially from the impossibility of instantaneous communication due to the second postulate of special relativity.

If you move the clocks together physically, and synchronize them, this removes a large degree of the concern about instantaneous communication (this probably makes communication faster than your tolerances)

Re: What does it take to make Google work at scale?

#98

Post by acqq »

mad44 wrote: post #38
ska wrote: post #6

Earlier quoted context omitted.

I think it is crazy that something as simple as keeping time can have such a complicated solution at that sort of scale. Time keeping is one of the things that is most often underestimated and screwed up at any scale, in my experience. Many a subtle bug turns out to be a poor assumption or misunderstanding about how clocks work. Or calendars.

Hybrid Logical Clocks offer a simple and feasible alternative to Google's custom hardware based TrueTime. http://muratbuffalo.blogspot.com/2014/07/hybrid-logical-cloc...

It would be nice if the authors would evaluate how the hardware clocks compare to their software method in the case of maintenance regimes. I understand that their method is "good enough" in the "stable" state across the datacenters spread out through the world. The question remains if the existence of the hardware clocks simplifies the "unstable" states (that occur not when everything runs according to your software design, but when you maintain the infrastructure at that scale) and I believe it still does. I don't have proofs, but my belief is that using GPS and atomic clocks at that scale (GPS clocks on different physical locations) isn't an example of doing unnecessary stuff.

Re: What does it take to make Google work at scale?

#99

Post by monksy »

jsolson wrote: post #55
davis_m wrote: post #37

Earlier quoted context omitted.

Reading it back, my comment was harsher than I intended, especially with the reference to Amazon, but most of my aversion to working at a place like Google is just the shear size as well as where most of the offices are located. I grew up in a tiny midwest town and I love it here. I would not enjoy living on either one of the coasts. Many of my classmates in college couldn't wait to get out of the midwest. I have fri…

You could try Google Pittsburgh; it's almost the midwest :) (I work for Google Seattle, but I grew up in Pittsburgh -- Seattle is definitely more my speed, but I do enjoy a trip home now and then)

Google also does some dev work in Chicago.

Re: What does it take to make Google work at scale?

#100

Post by monksy »

bwillard wrote: post #74
selectodude wrote: post #53

Earlier quoted context omitted.

Google Chicago is almost all salespeople. Not much technology going on there.

It's true the sales people do out number us here in Chicago, but we do have O(100) engineers in the office. We are working on lots of cool stuff: Ads, Search, and Privacy to name a couple.

You guys need to help the CJUG get some more talks! It's been a while since Google has had a role in that.
Post reply on HN