Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

History of massive-scale sorting experiments at Google

cloud.google.com

Post reply on HN
Print view View original

61–70 of 82 posts

Re: History of massive-scale sorting experiments at Google

#61

Post by wantreprenr007 »

AI_Overlord wrote: post #47
seiji wrote: post #44

Earlier quoted context omitted.

Just one summer of study and it'll be solved, right? It's always been "one summer away" for the past 60 years. Get back to us when you have an algorithm for love and art and petrichor.

Emotions are not a requirement for strong AI. I frankly would not waste any time with them. I just need an AI that can learn and solve problems at the human level. I'm not naive enough to think that it can be solved in a short time. I do think that it is worth it for a person to spend the rest of their life working on it. There is just nothing more exciting than AI in my opinion. Just imagine the possibilities...

"Narrow" AI is already here: search, banking, insurance, internet ads, crime prediction, Siri, plane autopilot (autotakeoff/landing too), autoparking, driver assist, on and on.

(At Trimble, we had fully autonomous tractor PoC in 2001)

"Deep" AI (self-directed / human-interactive) will take more time and effort, and can have (simulated) emotions if so programmed; the determinate is how to sell such as a viable product or service that doesn't freak people out too much or do something stupid like place untrustworthy systems in charge of live nuclear missiles.

Re: History of massive-scale sorting experiments at Google

#62

Post by [deleted] »

AI_Overlord wrote: post #43
seiji wrote: post #38

Earlier quoted context omitted.

working on strong AI is, in my mind, the ultimate challenge. So is working on antigravity, free energy, and backwards time travel, but people don't do that because they are no reasonable approaches we can try. Glorifying "AI AI AI" is silly because — there are no approaches we can try. Sure, we can identify ten million images per second, but none of that involves the least bit of "thinking."

Evolution has already proven that strong AI is possible. It is up to us to discover how. It is only a matter of time.

[deleted]

Re: History of massive-scale sorting experiments at Google

#63

Post by dang »

AI_Overlord wrote: post #35

To each their own. While impressive I do not find it inspiring. Instead, working on strong AI is, in my mind, the ultimate challenge. Far more amazing by any measure than anything we have acomplished so far.

We detached this subthread from https://news.ycombinator.com/item?id=11130660 and marked it off-topic.

Re: History of massive-scale sorting experiments at Google

#64

Post by gefh »

existencebox wrote: post #14
gefh wrote: post #10

Earlier quoted context omitted.

You may well be asked how a distributed sort would work in principle, and be asked to code a small part of it. Any reasonable answer that shows you can think on your feet would be evidence in favor of hiring. It sounds like you didn't get hired, and I'm sorry about that. Interviewers' preferences are of course diverse, but for at least the last 5 years, probably 10, Google interviews favor practical solutions to prob…

As a datapoint, a while back I was asked (for a G interview within the last 10 years) to code an RB tree from start to finish. I'll admit I was a little miffed at how distanced this seemed from anything I would potentially do; I was not interviewing for a high theory/deep algorithms group. That being said, in my other experience with a G interview, the entire process was much more mundane and as expected. For the rec…

Thanks for the datapoint. I concur with your friends, I wouldn't ask that question. IMHO even if you get a perfect answer, all you can be sure of is that the candidate has a good memory and happened to read the right books last night. I'd be more interested in a much simpler problem but with some modifications that prove there's original thought going on.

There is a (surprising?) latitude given to interviewers to ask their own questions however, and the best advice I could give to interviewers and interviewees is that the question should really just be a seed to a good technical discussion.

Re: History of massive-scale sorting experiments at Google

#65

Post by mayanksinghal »

drx wrote: post #26
asuffield wrote: post #23

Earlier quoted context omitted.

(Tedious disclaimer: not speaking for anybody else, my opinion only, etc. I'm an SRE at Google.) "Within the last 10 years" covers several generations of refinement to the interview process. I'd suggest forgetting about anything more than a couple of years old. The major thing to keep in mind is that these days, your recruiter will ask you up front what subject areas you are strongest in, and you should expect to get…

What's the process like these days?

Faster (at least it's attempting to be) without losing any quality metrics.

Re: History of massive-scale sorting experiments at Google

#66

Post by fritzo »

cmrdporcupine wrote: post #50
late2part wrote: post #15

Earlier quoted context omitted.

Maybe, maybe not. Most people's surface area of problems don't even approach Google's problems. The kind of things most companies aspire to solve, Google automated a decade ago. So, this esoteric CS theory may be more practical inside Google than your external vantage point can consider.

The problems being automated years ago means the vast majority of Google engineers don't ever get close to having to work on those solutions. One of things that disappointed me after joining G was that many of the problems I found fun to solve at previous employers were a) solved already and b) solved by people much smarter and senior than me. It's a great place to work for all sorts of reasons. But working on cuttin…

"with more rules we have solved most of the problems in the world. That just leaves the weird events" -Steve Coast

The remaining problems are indeed weird, and succumb only after application of every available tool: "asking around", CS theory, low-level debugging, visualization, heavy logs analysis, pouring over source code, strolling aimlessly, mining commit history, politicking, brainstorming...

Re: History of massive-scale sorting experiments at Google

#69

Post by amelius »

rockinghigh wrote: post #51
azurezyq wrote: post #8

>> Nobody really wants a huge globally sorted output. We haven’t found a single use case for the problem as stated. Do anyone has a real world use case of global sorting other than top-k?

If you sort records by time you can perform time range queries. You can also create an index to access random keys (like a MapFile in Hadoop).

But isn't sorting by time easy, because records come in the correct order? I would say you don't need an algorithm for that.

Re: History of massive-scale sorting experiments at Google

#70

Post by rajacombinator »

KMag wrote: post #19
seanp2k2 wrote: post #7

I'm sure they're asking about this in interviews now. My experience with Google interviews was that the interviewers were very keen on proving that they knew more theoretical CS than I did vs talking about what the actual work would require or entail.

I worked at Google, and performed quite a few interviews. At least at that time, the interviewer had no idea where in the company the software engineer would go. The people who got hired would go into a pool, and the managers who needed staff would then horse-trade for them. They needed to hire people who could be plugged in to tons of different positions, including some that genuinely had tough problems where if the…

How many work on indexing? I'm guessing like 100 tops? (Genuine guess/curiosity, not sarcasm.)
Post reply on HN