Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google’s self-training AI turns coders into machine-learning masters

technologyreview.com

Post reply on HN
Print view View original

11–20 of 32 posts

Re: Google’s self-training AI turns coders into machine-learning masters

#11

Post by radarsat1 »

sanxiyn wrote: post #9
YeGoblynQueenne wrote: post #4

>> “We need to scale AI out to more people,” Fei-Fei Li, chief scientist at Google Cloud, said ahead of the launch today. Problem is, when Google says "AI" they mean deep learning, on ginormous datasets with humongous clusters of GPUs. That don't scale. Accordingly, when Google says "we need to scale AI out to more people" what they really mean is "we need to make more people use our services". Sure- but most develop…

Of course you can train deep learning models on small datasets. Where did you hear otherwise? Try yourself at https://nanonets.com/

He didn't say you couldn't, he said that's what Google means.

Re: Google’s self-training AI turns coders into machine-learning masters

#12

Post by sanxiyn »

radarsat1 wrote: post #11
sanxiyn wrote: post #9

Earlier quoted context omitted.

Of course you can train deep learning models on small datasets. Where did you hear otherwise? Try yourself at https://nanonets.com/

He didn't say you couldn't, he said that's what Google means.

My understanding is that Google very much intends Cloud AutoML Vision usable for small datasets. To me "deep learning ... inherently impossible to train small" sounds entirely confused.

Re: Google’s self-training AI turns coders into machine-learning masters

#13

Post by seeekr »

vidarh wrote: post #10
YeGoblynQueenne wrote: post #4

>> “We need to scale AI out to more people,” Fei-Fei Li, chief scientist at Google Cloud, said ahead of the launch today. Problem is, when Google says "AI" they mean deep learning, on ginormous datasets with humongous clusters of GPUs. That don't scale. Accordingly, when Google says "we need to scale AI out to more people" what they really mean is "we need to make more people use our services". Sure- but most develop…

Frankly I think we'd benefit more, faster even from just better standard libraries of really simple methods in ways that makes it more obvious how to take advantage of them. E.g. there are so many scenarios where even really basic statistical approaches like bayesian models can provide drastic improvements over what people tend to do, but most developers I've worked with don't know how to take advantage of even that…

I would be interested in hearing about some examples. I'm one of those who are not yet taking advantage of these potential drastic improvements you are referring to.

Re: Google’s self-training AI turns coders into machine-learning masters

#14

Post by nolroz »

vidarh wrote: post #10
YeGoblynQueenne wrote: post #4

>> “We need to scale AI out to more people,” Fei-Fei Li, chief scientist at Google Cloud, said ahead of the launch today. Problem is, when Google says "AI" they mean deep learning, on ginormous datasets with humongous clusters of GPUs. That don't scale. Accordingly, when Google says "we need to scale AI out to more people" what they really mean is "we need to make more people use our services". Sure- but most develop…

Frankly I think we'd benefit more, faster even from just better standard libraries of really simple methods in ways that makes it more obvious how to take advantage of them. E.g. there are so many scenarios where even really basic statistical approaches like bayesian models can provide drastic improvements over what people tend to do, but most developers I've worked with don't know how to take advantage of even that…

I'm trying to start simple on my journey towards thinking more like a statistician and have long suspected this would be the case at many companies. Besides Bayesian models, do any other common methods seem to be ignored as often? Any suggestions resources that a newbie on the scene might use to educate themselves?

Re: Google’s self-training AI turns coders into machine-learning masters

#15

Post by conjectures »

sanxiyn wrote: post #9
YeGoblynQueenne wrote: post #4

>> “We need to scale AI out to more people,” Fei-Fei Li, chief scientist at Google Cloud, said ahead of the launch today. Problem is, when Google says "AI" they mean deep learning, on ginormous datasets with humongous clusters of GPUs. That don't scale. Accordingly, when Google says "we need to scale AI out to more people" what they really mean is "we need to make more people use our services". Sure- but most develop…

Of course you can train deep learning models on small datasets. Where did you hear otherwise? Try yourself at https://nanonets.com/

From the site, "large pretrained models + your data."

...it's not really a small dataset then, though.

Re: Google’s self-training AI turns coders into machine-learning masters

#17

Post by make3 »

conjectures wrote: post #15
sanxiyn wrote: post #9

Earlier quoted context omitted.

Of course you can train deep learning models on small datasets. Where did you hear otherwise? Try yourself at https://nanonets.com/

From the site, "large pretrained models + your data." ...it's not really a small dataset then, though.

The question is, is the power of deep nets constrained to tasks where you have a huge amount of data.

there are a large amount of applications where you can use big models trained on generic tasks where the data is free and available to everyone, and then fine tune to your specific more limited data.

let's say I need a person detector for my cctv camera. I use a pretrained ResNet ImageNet model distributed by Google, that was trained on a super large dataset identifying things like animals, furniture and whatnot on a huge free dataset, and then train it just a bit more on my specific task.

this allows to create very performant networks with very little task specific data.

Re: Google’s self-training AI turns coders into machine-learning masters

#19

Post by yorwba »

sanxiyn wrote: post #12
radarsat1 wrote: post #11

Earlier quoted context omitted.

He didn't say you couldn't, he said that's what Google means.

My understanding is that Google very much intends Cloud AutoML Vision usable for small datasets. To me "deep learning ... inherently impossible to train small" sounds entirely confused.

The whole advantage of deep learning comes from the large number of parameters that can fit very complex feature distributions. If you don't have a lot of data and try to train a deep net on it, you're just going to overfit.

Of course there's the opportunity for transfer learning where you try to adjust a pretrained model slightly for a different task, but that only works if the model already mostly solves your problem. (e.g. it picks up on the necessary features, but the class you want is not one of the possible outputs.)

If your data is unlike any popular task with large datasets available, (say because you're trying to predict something from a small in-house database), transfer learning can't work (because there's no knowledge you could transfer) and you are better off with simpler models.

Re: Google’s self-training AI turns coders into machine-learning masters

#20

Post by yorwba »

nolroz wrote: post #14
vidarh wrote: post #10

Earlier quoted context omitted.

Frankly I think we'd benefit more, faster even from just better standard libraries of really simple methods in ways that makes it more obvious how to take advantage of them. E.g. there are so many scenarios where even really basic statistical approaches like bayesian models can provide drastic improvements over what people tend to do, but most developers I've worked with don't know how to take advantage of even that…

I'm trying to start simple on my journey towards thinking more like a statistician and have long suspected this would be the case at many companies. Besides Bayesian models, do any other common methods seem to be ignored as often? Any suggestions resources that a newbie on the scene might use to educate themselves?

One simple method that is often good enough is k-nearest neighbors. Basically computing the k most similar training values to a test value and then averaging their outputs to get a prediction.
Post reply on HN