Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google's hybrid approach to research

norvig.com

Post reply on HN
Print view View original

11–19 of 19 posts

Re: Google's hybrid approach to research

#11

Post by fa_il »

alextp wrote: post #10
fa_il wrote: post #9

The thing is that "MapReduce" is a concept that was in practice, e.g. by LISP programmers, long, long before Google rediscovered it. Much like Google's many acquisitions that the public perceives as resulting from "Google R&D", things like map-reduce are also viewed as coming from "unparalleled Google capabilities". Let's get real. Google is a big company that employs thousands upon thousands of overqualified Java an…

Mapreduce as a concept goes beyond lisp implementations. On the surface it might seem like the point of mapreduce is expressing computations in terms of map and reduce functions. It isn't. The point of mapreduce is reducing the problem of high-throughput fault-tolerant distributed systems to a very efficient and reliable distributed sorting algorithm (the shuffle phase, which is implemented by the implementations of…

Sorry, but applying an old concept to a new problem (actually just new buzzwords... it's only the size of the problem that's new) does not make a "novel" solution. Moreover, it's an obvious solution. But I guess that depends on who is doing the programming.

I would love to see how programmers with large clusters at their disposal were approaching large datasets before the moment they realized splitting the task into smaller pieces was what they should do.

Re: Google's hybrid approach to research

#12

Post by jrockway »

fa_il wrote: post #9

The thing is that "MapReduce" is a concept that was in practice, e.g. by LISP programmers, long, long before Google rediscovered it. Much like Google's many acquisitions that the public perceives as resulting from "Google R&D", things like map-reduce are also viewed as coming from "unparalleled Google capabilities". Let's get real. Google is a big company that employs thousands upon thousands of overqualified Java an…

I see you created a brand new account just to write this post.

MapReduce is the name of a piece of software, not simply the concept of map followed by reduce. That's something that every high school freshman invents on his own in Algebra 1 class. The interesting research area is making that concept scale to "run this command on every web page on the Internet" billions of times a day. I don't know about you, but I don't see anything to do that in my apt repositories.

Research at Google isn't about solving problems that are beyond the comprehension or reach of any average practitioner of programming. A good example is Street View. Anyone can understand strapping some cameras and sensors to a car and driving it around to make pictures of places available on the Internet. It's hard to call that "state of the art research" because it's such a simple idea, but before Google did the research, the product didn't exist. Now you can see almost any street address anywhere in the world in your browser. (The hard part is in the details. How do you map images to locations on the Earth in areas where GPS reception isn't good enough to provide enough accuracy? Scaling something to the entire Earth is not easy.)

That many programmers, who have standards that consistently hover around varying levels of mediocrity

How exactly did you come to this conclusion?

Re: Google's hybrid approach to research

#13

Post by alextp »

fa_il wrote: post #11
alextp wrote: post #10

Earlier quoted context omitted.

Mapreduce as a concept goes beyond lisp implementations. On the surface it might seem like the point of mapreduce is expressing computations in terms of map and reduce functions. It isn't. The point of mapreduce is reducing the problem of high-throughput fault-tolerant distributed systems to a very efficient and reliable distributed sorting algorithm (the shuffle phase, which is implemented by the implementations of…

Sorry, but applying an old concept to a new problem (actually just new buzzwords... it's only the size of the problem that's new) does not make a "novel" solution. Moreover, it's an obvious solution. But I guess that depends on who is doing the programming. I would love to see how programmers with large clusters at their disposal were approaching large datasets before the moment they realized splitting the task into…

It's not about splitting the task into smaller pieces. It's about factoring out the parts of the task that need synchronization among all machines into one specific subroutine (groupBy) which makes mapreduce so powerful.

If you speak with people experienced in multithreaded and distributed programming you will see that synchronization with fault-tolerance is _hard_, and mapreduce provides a widely-applicable set of sufficient conditions for an algorithm to be executable with implicit fault-tolerance and implicit synchronization.

Without mapreduce-like abstractions eveyr piece of software has to be responsible for its own (1) checkpointing (to recover from errors), (2) checksumming (to ensure that no errors happened), and (3) distributed communication (to make sure the global state becomes global and the local state becomes local).

Re: Google's hybrid approach to research

#14

Post by fa_il »

alextp wrote: post #13
fa_il wrote: post #11

Earlier quoted context omitted.

Sorry, but applying an old concept to a new problem (actually just new buzzwords... it's only the size of the problem that's new) does not make a "novel" solution. Moreover, it's an obvious solution. But I guess that depends on who is doing the programming. I would love to see how programmers with large clusters at their disposal were approaching large datasets before the moment they realized splitting the task into…

It's not about splitting the task into smaller pieces. It's about factoring out the parts of the task that need synchronization among all machines into one specific subroutine (groupBy) which makes mapreduce so powerful. If you speak with people experienced in multithreaded and distributed programming you will see that synchronization with fault-tolerance is _hard_, and mapreduce provides a widely-applicable set of s…

Because you consider a problem to be "hard" and because someone else has proferred a solution that previously was not obvious to you does not lead me to the conclusion that map-reduce is something new (nor the result of ground-breaking research). It leads me to the conclusion that it is helpful to you. Those are to me two different things.

Re: Google's hybrid approach to research

#15

Post by fa_il »

jrockway wrote: post #12
fa_il wrote: post #9

The thing is that "MapReduce" is a concept that was in practice, e.g. by LISP programmers, long, long before Google rediscovered it. Much like Google's many acquisitions that the public perceives as resulting from "Google R&D", things like map-reduce are also viewed as coming from "unparalleled Google capabilities". Let's get real. Google is a big company that employs thousands upon thousands of overqualified Java an…

I see you created a brand new account just to write this post. MapReduce is the name of a piece of software, not simply the concept of map followed by reduce. That's something that every high school freshman invents on his own in Algebra 1 class. The interesting research area is making that concept scale to "run this command on every web page on the Internet" billions of times a day. I don't know about you, but I don…

Right. I would say it's more about execution. For that Google gets full credit. And I do find the execution impressive.

The mediocrity line is my opinion. Not necessarily fact. Downvote me if you are offended.

Better yet, prove me wrong.

Re: Google's hybrid approach to research

#16

Post by jrockway »

fa_il wrote: post #15
jrockway wrote: post #12

Earlier quoted context omitted.

I see you created a brand new account just to write this post. MapReduce is the name of a piece of software, not simply the concept of map followed by reduce. That's something that every high school freshman invents on his own in Algebra 1 class. The interesting research area is making that concept scale to "run this command on every web page on the Internet" billions of times a day. I don't know about you, but I don…

Right. I would say it's more about execution. For that Google gets full credit. And I do find the execution impressive. The mediocrity line is my opinion. Not necessarily fact. Downvote me if you are offended. Better yet, prove me wrong.

Better yet, prove me wrong.

First you say, all programmers at Google have standards that "hover around mediocrity". Then you say you're amazed at Google's execution on world-changing ideas. Do you see a contradiction?

Argument over, you can close your trolling account! :)

Re: Google's hybrid approach to research

#17

Post by fa_il »

jrockway wrote: post #16
fa_il wrote: post #15

Earlier quoted context omitted.

Right. I would say it's more about execution. For that Google gets full credit. And I do find the execution impressive. The mediocrity line is my opinion. Not necessarily fact. Downvote me if you are offended. Better yet, prove me wrong.

Better yet, prove me wrong. First you say, all programmers at Google have standards that "hover around mediocrity". Then you say you're amazed at Google's execution on world-changing ideas. Do you see a contradiction? Argument over, you can close your trolling account! :)

Can you show me where I said:

"all programmers at Google"

"amazed"

"world-changing ideas"

What I see are your words, not mine. Yet they are attributed to me.

If this is an "argument" as you suggest, then you are doing a poor job at making your case.

There is a difference between 1. "research" and developing "novel" ideas and 2. executing well on large projects. It's possible to accomplish 2. without invoking 1., and vice versa.

There is no contradiction.

When a programmer complains there's nothing in "apt" to solve his problem, I'm never impressed. Nor am I ever surprised. Convenience makes some programmers very lazy.

Re: Google's hybrid approach to research

#18

Post by jrockway »

fa_il wrote: post #17
jrockway wrote: post #16

Earlier quoted context omitted.

Better yet, prove me wrong. First you say, all programmers at Google have standards that "hover around mediocrity". Then you say you're amazed at Google's execution on world-changing ideas. Do you see a contradiction? Argument over, you can close your trolling account! :)

Can you show me where I said: "all programmers at Google" "amazed" "world-changing ideas" What I see are your words, not mine. Yet they are attributed to me. If this is an "argument" as you suggest, then you are doing a poor job at making your case. There is a difference between 1. "research" and developing "novel" ideas and 2. executing well on large projects. It's possible to accomplish 2. without invoking 1., and…

Isn't there a bridge you should be guarding?

Re: Google's hybrid approach to research

#19

Post by mark_l_watson »

Great article. I always try, sometimes with a little success, to apply ideas I read about to my own life. I am a one man consulting shop but I try to integrate experiments/research into my weekly work flow because that was what I was used to as an employee (about a day a week of IR&D at SAIC, lots of experiments were done at Angel Studios for work for Nintendo and Disney, etc.)

Compared to companies like Facebook and Google, where I often go wrong is not more tightly coupling experiments/side projects to what my customers need. Instead, I invest my 1/3 time for experiments more on things that just really interest me and very often I don't really apply what I learn to practical projects.

My Dad is a scientist (at 91 he is still a member of the National Academy of Sciences) and over breakfast today I was describing to him the really fast iterative business/research/deployment cycles at Google, Twitter, Facebook, etc. Very different time scale than what he was used to.

Post reply on HN