Inside Google Brain • Hacker News Board

Re: Inside Google Brain

#21

Post by mendicantB » Thu, Jul 17, 2014, 1:02 PM UTC

nl wrote: post #20

cliveowen wrote: post #5

"They’ve also found that the models tend to become more accurate the more data they consume. That may be the next big goal for Google: building AI models that are based on billions of data points, not just millions. " I'm not versed in machine learning, but it looks to me that any model whose output quality is dependent on the quantity of data it ingests is deeply flawed. There's no doubt a bigger number of samples w…

No. The outcome is the goal. It's rapidly becoming apparent that some algorithms (eg Deep Learning related models) work much better at scale than on small amounts of data. It doesn't make sense to discount these better algorithms because they don't work as well as other models when tested against less data. It is also apparent that these models require significantly more computing power to perform well than other mod…

That's another good point. The required computing power that is now cheap and widely available has changed our ability to even try these methods.

Re: Inside Google Brain

#22

Post by quarterwave » Thu, Jul 17, 2014, 1:06 PM UTC

cliveowen wrote: post #5

"They’ve also found that the models tend to become more accurate the more data they consume. That may be the next big goal for Google: building AI models that are based on billions of data points, not just millions. " I'm not versed in machine learning, but it looks to me that any model whose output quality is dependent on the quantity of data it ingests is deeply flawed. There's no doubt a bigger number of samples w…

How quickly does a model's accuracy and precision improve with increasing sample size? Indeed, there are counter examples (like the periodogram estimator for psd) where variance does not decrease with window length.

We often don't know which model to use. Occam's Razor [1] can be effective in favouring simpler models, but I tend toward the view that a good data scientist is invariably needed to build good models. Hence I view Big Data more as a consulting business than SaaS.

[1] For an excellent Bayesian discussion on why Occam's Razor actually works, see Chapter 28 of David J.C. MacKay's book 'Information Theory, Inference and Learning Algorithms'.

Re: Inside Google Brain

#23

Post by cliveowen » Thu, Jul 17, 2014, 1:12 PM UTC

mendicantB wrote: post #19

cliveowen wrote: post #5

"They’ve also found that the models tend to become more accurate the more data they consume. That may be the next big goal for Google: building AI models that are based on billions of data points, not just millions. " I'm not versed in machine learning, but it looks to me that any model whose output quality is dependent on the quantity of data it ingests is deeply flawed. There's no doubt a bigger number of samples w…

Every model's output quality is dependent on the quantity of data it ingests. Statistics developed as a science because of the need to overcome the weakness of large samples being expensive. Machine learning has taken off as a direct result of the field's ability to take advantage of and get serious performance gains from the massive amounts of data being generated and leveraged recently. Here is the best summation I…

I completely agree with this and, as I said, it's obvious that more data produces better predictions, even with simple models. My point is that it looks backwards to me putting effort into finding more and better data (creating a corpus for a given subject is a challenge in itself) instead of trying to come up with a model that infers more and produces better predictions with less data. Once you have such a model then you can surely collect and feed it a lot of data to improve the output, but until then, why even bother?

Re: Inside Google Brain

#24

Post by daviddumenil » Thu, Jul 17, 2014, 1:37 PM UTC

The original paper being discussed:

http://arxiv.org/abs/1312.6082

Re: Inside Google Brain

#25

Post by michaelochurch » Thu, Jul 17, 2014, 1:53 PM UTC

Google is not really a search company. It's a machine-learning company.

It wants to be seen that way, but until it abolishes closed allocation (and maybe it has, but I haven't heard anything to indicate that it has) it will just be an ads company. It's still a pretty good place to work, by industry standards, but the percentage of people who'll get to work on machine learning is very low.

Google definitely wants to have the image of being the machine learning company because that's a great way to attract talent (even if that talent is mostly wasted under closed allocation). And if you land in the right place, there is interesting work. The reality most people face, though, is that most people (especially outside of Mt. View) aren't going to get real projects and won't be anywhere near the machine learning work.

Google does have a lot of talent and probably would be the undisputed #1 tech company if it implemented open allocation, though.

Re: Inside Google Brain

#26

Post by sp332 » Thu, Jul 17, 2014, 2:27 PM UTC

astazangasta wrote: post #12

>About a year later, Google had reduced Android’s voice recognition error rate by an astounding 25 percent. lol. This is the grand payoff?

It was already as good as a bunch of PhD's could make it after working on it for years. Removing 25% of the remaining errors after that is a grand payoff!

Re: Inside Google Brain

#27

Post by astazangasta » Thu, Jul 17, 2014, 2:27 PM UTC

deong wrote: post #15

astazangasta wrote: post #12

>About a year later, Google had reduced Android’s voice recognition error rate by an astounding 25 percent. lol. This is the grand payoff?

Considering that for the preceding 25 years or so, progress in the state of the art had been annual reductions of far less than 1%, it's a pretty big deal.

From an academic perspective, maybe. In terms of practical use of machine methods, not much. Machine learning is largely hype. I pity the army of PhDs they must have building training sets.

Re: Inside Google Brain

#28

Post by programminggeek » Thu, Jul 17, 2014, 2:31 PM UTC

michaelochurch wrote: post #25

Google is not really a search company. It's a machine-learning company. It wants to be seen that way, but until it abolishes closed allocation (and maybe it has, but I haven't heard anything to indicate that it has) it will just be an ads company. It's still a pretty good place to work, by industry standards, but the percentage of people who'll get to work on machine learning is very low. Google definitely wants to h…

Google is an Advertising company and it delivers ads via these platforms like search, mobile apps, etc.

It clearly is a tech company, but to pretend it's anything other than an ad delivery machine is sort of ignoring the elephant in the room.

Everything goes back to ads. Even Google doesn't bite the hand that feeds it.

Re: Inside Google Brain

#29

Post by jfields513 » Thu, Jul 17, 2014, 2:42 PM UTC

ckaygusu wrote: post #8

cliveowen wrote: post #5

"They’ve also found that the models tend to become more accurate the more data they consume. That may be the next big goal for Google: building AI models that are based on billions of data points, not just millions. " I'm not versed in machine learning, but it looks to me that any model whose output quality is dependent on the quantity of data it ingests is deeply flawed. There's no doubt a bigger number of samples w…

If you think human brain is so sophisticated that it can perform its cognitive duties with little data, this is simply wrong. While it is definitely not a simple organic construct, it does get stimulated significantly all the time. See [1] what happens when you cut out this factors. Regarding artificial systems, I think more data is the only way to reach super-performing classifiers. The data you supply doesn't have…

Your first statement here is not true.

Humans are excellent at learning from very few or even 1 example. Show a toddler a single image of an elephant and the toddler will generalize perfectly on new examples; show a machine a few thousand images of elephants and it might generalize decently if your machine is really clever.

There are very few tasks where machine systems achieve anything resembling human level performance. But on all such tasks, the machine requires far more data and still underperforms.

Re: Inside Google Brain

#30

Post by DanBC » Thu, Jul 17, 2014, 2:54 PM UTC

astazangasta wrote: post #12

>About a year later, Google had reduced Android’s voice recognition error rate by an astounding 25 percent. lol. This is the grand payoff?

There's a bunch of stuff that gets optimised. You start with big gains, and those get smaller and smaller until you're spending a lot of effort to get 0.5% increase in performance.

See, for example, compressing the ENWIK8 in the Hutter Prize.

http://prize.hutter1.net/

If you can compress this approx 100 MB file to less than approx 16 MB (including the decompressor) you win cash.

That website shows small decreases in filesize over a few years.

    Alexander Rhatushnyak	23.May 2009	6.27 | 	1614€	Marcus Hutter
    Alexander Rhatushnyak	14.May 2007	6.07 | 	1732€	Marcus Hutter
    Alexander Rhatushnyak	25.Sep.2006	5.86 | 	3416€	Marcus Hutter
    Matt Mahoney	24.Mar.2006	5.46 | 	pre-prize	 -

The compression ratio goes from 5.46 to 5.86 to 6.07 to 6.27.