Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Inside Google Brain

wired.com

Post reply on HN
Print view View original

71–77 of 77 posts

Re: Inside Google Brain

#71

Post by snowwrestler »

jfields513 wrote: post #29
ckaygusu wrote: post #8

Earlier quoted context omitted.

If you think human brain is so sophisticated that it can perform its cognitive duties with little data, this is simply wrong. While it is definitely not a simple organic construct, it does get stimulated significantly all the time. See [1] what happens when you cut out this factors. Regarding artificial systems, I think more data is the only way to reach super-performing classifiers. The data you supply doesn't have…

Your first statement here is not true. Humans are excellent at learning from very few or even 1 example. Show a toddler a single image of an elephant and the toddler will generalize perfectly on new examples; show a machine a few thousand images of elephants and it might generalize decently if your machine is really clever. There are very few tasks where machine systems achieve anything resembling human level perform…

I can only speak for my situation, but my daughter saw easily hundreds and maybe thousands of examples of elephants, including real live ones at the zoo, before she could ever seem to use the abstract concept to identify new examples.

Re: Inside Google Brain

#72

Post by dsymonds »

programminggeek wrote: post #28
michaelochurch wrote: post #25

Google is not really a search company. It's a machine-learning company. It wants to be seen that way, but until it abolishes closed allocation (and maybe it has, but I haven't heard anything to indicate that it has) it will just be an ads company. It's still a pretty good place to work, by industry standards, but the percentage of people who'll get to work on machine learning is very low. Google definitely wants to h…

Google is an Advertising company and it delivers ads via these platforms like search, mobile apps, etc. It clearly is a tech company, but to pretend it's anything other than an ad delivery machine is sort of ignoring the elephant in the room. Everything goes back to ads. Even Google doesn't bite the hand that feeds it.

That's a common trope, but it doesn't make sense. We don't call the New York Times an advertising company simply because the majority of its revenue comes from ads. Why apply the same standard to Google?

Re: Inside Google Brain

#73

Post by cbsmith »

cliveowen wrote: post #5

"They’ve also found that the models tend to become more accurate the more data they consume. That may be the next big goal for Google: building AI models that are based on billions of data points, not just millions. " I'm not versed in machine learning, but it looks to me that any model whose output quality is dependent on the quantity of data it ingests is deeply flawed. There's no doubt a bigger number of samples w…

Have you ever taught a kid to learn? I don't think there is a lot of evidence suggesting the human brain reliably learns without a number of data points...

Seriously, Norvig has been big on this since forever: the reality is that consuming large amounts of data with relatively subtle features tends to be one of the few areas where computers can easily outclass the human brain.

Re: Inside Google Brain

#74

Post by gipp »

cliveowen wrote: post #23
mendicantB wrote: post #19

Earlier quoted context omitted.

Every model's output quality is dependent on the quantity of data it ingests. Statistics developed as a science because of the need to overcome the weakness of large samples being expensive. Machine learning has taken off as a direct result of the field's ability to take advantage of and get serious performance gains from the massive amounts of data being generated and leveraged recently. Here is the best summation I…

I completely agree with this and, as I said, it's obvious that more data produces better predictions, even with simple models. My point is that it looks backwards to me putting effort into finding more and better data (creating a corpus for a given subject is a challenge in itself) instead of trying to come up with a model that infers more and produces better predictions with less data. Once you have such a model the…

It's not like they aren't trying to improve the model as well, all the time. It's just saying that right now the benefit of getting more data for existing (already very sophisticated) models is greater than the incremental benefits of model improvements given existing data.

Re: Inside Google Brain

#75

Post by dekhn »

paperwork wrote: post #59
gunnario wrote: post #57

"This form of internal code-sharing has already helped another cutting-edge Google technology called MapReduce catch fire." Map-Reduce is not a Google technology.

I believe it is. Obviously map and reduce operations have existed for a long time, but MapReduce was Google's work, before it was popularize by hadoop and friends. Soon after MapReduce became popular, there was a paper published (by non-googlers) which 'described' the algorithm and poked some fun at the terminology: http://lambda-the-ultimate.org/node/1669

Most people miss this important point: the most important part of MapReduce is the shuffle step, which is a global, partitioned disk-to-disk sort. See the FlumeJava for a bit more discussion on why this is relevant. Everything else about MapReduce is just framework to make programmer's lives easier. BTW, not having strong typing in classic MR is a major pain point. Flume goes a long way to addressing this in a practical way.

BTW, this point elucidates the importance of this classic Google Interview question: http://www.glassdoor.com/Interview/Sort-a-million-32-bit-int...

Re: Inside Google Brain

#76

Post by mendicantB »

gipp wrote: post #74
cliveowen wrote: post #23

Earlier quoted context omitted.

I completely agree with this and, as I said, it's obvious that more data produces better predictions, even with simple models. My point is that it looks backwards to me putting effort into finding more and better data (creating a corpus for a given subject is a challenge in itself) instead of trying to come up with a model that infers more and produces better predictions with less data. Once you have such a model the…

It's not like they aren't trying to improve the model as well, all the time. It's just saying that right now the benefit of getting more data for existing (already very sophisticated) models is greater than the incremental benefits of model improvements given existing data.

Bingo. More data beats better algorithms, see

http://anand.typepad.com/datawocky/2008/03/more-data-usual.h...

Re: Inside Google Brain

#77

Post by niangb »

MaysonL wrote: post #68
niangb wrote: post #16

Earlier quoted context omitted.

This is true to the extent that you are not overfitting your dataset. Neural networks and random trees are quite good at fitting anything! And still they can perform poorly on your validation set.

Overfitting will only occur when the dataset is too small for the model.

Not only. Your example is too particular. I would say that overfitting tends to occur when one do not understand the underlying dynamic of a system you are trying to model. Any model with enough degrees of liberties can fit anything and still explain nothing.
Post reply on HN