Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Andrew Ng: Unbiggen AI

spectrum.ieee.org

Post reply on HN
Print view View original

11–20 of 90 posts

Re: Andrew Ng: Unbiggen AI

#11

Post by teruakohatu »

vasco wrote: post #7
itissid wrote: post #6

That is the problem with generalization and cop outs like these. It's no good to people in the field doing actual work where the devil is in the detail. Big data is fairly important to a lot of things, for example I was listening to Tesla's use of Deep net models where they mentioned that there were literally so many variations of Stop Signs that they needed to learn what was really in the "tail" of the distribution…

Interestingly, when you learn how to drive you need to see approximately one example and you're able to identify them all.

That is called transfer learning. You might only need to see one photo of a sign to identify it in real life (although arguably learner drivers take a while to notice signs) but that is only because you have been training on identifying generic objects since you left the womb.

You brain already knows how to select the most important features of a sign. The shape, the size and the color. You have also learned how to understand the text on the sign.

A new born baby does not have that ability.

This is applied in ANN as well. Transfer learning is using a pre-trained neural network, which has already learned identifying objects, and then using it to train on identifying a new, usually smaller, set of objects using, usually, a lot less training data. That is what Andrew is talking about in the article.

Re: Andrew Ng: Unbiggen AI

#12

Post by itissid »

vasco wrote: post #7
itissid wrote: post #6

That is the problem with generalization and cop outs like these. It's no good to people in the field doing actual work where the devil is in the detail. Big data is fairly important to a lot of things, for example I was listening to Tesla's use of Deep net models where they mentioned that there were literally so many variations of Stop Signs that they needed to learn what was really in the "tail" of the distribution…

Interestingly, when you learn how to drive you need to see approximately one example and you're able to identify them all.

It does feel though that a model like the human mind will be very fundamentally different from any of the models of today. No?

Like the NN State of the art models of today are so different from state of the art 12 or so years ago which was SVMs.

Re: Andrew Ng: Unbiggen AI

#13

Post by riku_iki »

Datenstrom wrote: post #4
TOMDM wrote: post #2

Yeah that'd be great. I also want cars that run on salt water. I'm not saying that small data ai is equally impossible, but simply saying "we should make this better thing" isn't enough.

> simply saying "we should make this better thing" isn't enough. Besides the references to his company which has customers and a product that already works on these principles the literature currently shows that this is very much possible if you dig into the correct niches. Besides the SOTA in few-shot and meta-learning it is possible to smartly choose the correct few samples for the network that yield the same resul…

> it is possible to smartly choose the correct few samples for the network that yield the same results.

And then, someone is using pretrained 500B model, and fine-tuning your few examples, and getting new SOTA.

Re: Andrew Ng: Unbiggen AI

#14

Post by overkalix »

whatever1 wrote: post #5

My understanding is that they are trying to automate the data preparation steps that seasoned ML practitioners are doing anyway today. The fact that he tries this in manufacturing makes the case stronger. In most manufacturing companies you do not have access to top ML talent. You have Greg who knows python and recently visualized some production metrics. If we could empower Greg with automated ML libraries that guid…

Greg probably also knows SAS and AMPL, and has a good knowledge of ops research, which is within stone-tossing distance of whatever ML is pretending to be this week.

Re: Andrew Ng: Unbiggen AI

#15

Post by whatever1 »

overkalix wrote: post #14
whatever1 wrote: post #5

My understanding is that they are trying to automate the data preparation steps that seasoned ML practitioners are doing anyway today. The fact that he tries this in manufacturing makes the case stronger. In most manufacturing companies you do not have access to top ML talent. You have Greg who knows python and recently visualized some production metrics. If we could empower Greg with automated ML libraries that guid…

Greg probably also knows SAS and AMPL, and has a good knowledge of ops research, which is within stone-tossing distance of whatever ML is pretending to be this week.

OR and ML have their own space in manufacturing.

OR is perfect when you can describe explicitly what the decision space is and what the restrictions are.

ML is great fit when you want to identify and use patterns. Quality control with machine vision is a good application for ML. NLP for PDF documents is a huge field for manufacturing as well. Companies have so much data in email attachments that they do not currently take advantage of.

Re: Andrew Ng: Unbiggen AI

#16

Post by gary_0 »

teruakohatu wrote: post #11
vasco wrote: post #7

Earlier quoted context omitted.

Interestingly, when you learn how to drive you need to see approximately one example and you're able to identify them all.

That is called transfer learning. You might only need to see one photo of a sign to identify it in real life (although arguably learner drivers take a while to notice signs) but that is only because you have been training on identifying generic objects since you left the womb. You brain already knows how to select the most important features of a sign. The shape, the size and the color. You have also learned how to u…

> The shape, the size and the color.

And the context. For example, self-driving cars need to account for "Pizza Stop" restaurant signage, placards stuck to telephone poles that say things like "Stop Cancer", stop signs retracted into the sides of school buses, signs with additional instructions like "Stop when lights flashing", road workers with handheld stop signs, and the unconventional stop signs you see in parking lots.

You can probably get pretty far by checking the proximity to the road, height, dimensions, orientation, what it's mounted on, and if the sign incorporates any other text. But you can't just scan some pixels for "red octagon with STOP on it".

Re: Andrew Ng: Unbiggen AI

#17

Post by vasco »

corndoge wrote: post #10
vasco wrote: post #7

Earlier quoted context omitted.

Interestingly, when you learn how to drive you need to see approximately one example and you're able to identify them all.

Is there some underlying point to this statement? It comes off as a passive dismissal of something but I'm not sure what. It might be helpful to directly state what you're trying to say so that other people can engage with it.

It doesn't seem like the other replies are having trouble engaging with it. Since we're giving each other advice, you should use your down votes instead of pontificating that other people's comments are "passive dismissals" when you don't like or understand them.

Re: Andrew Ng: Unbiggen AI

#18

Post by xiphias2 »

I can imagine that customizing AI solutions in an automated way is quite important, but writing that as the next wave is probably an overstatement.

Of course few shot learning is important for models, but for example for Pathways it was already part of the evaluation.

Re: Andrew Ng: Unbiggen AI

#19

Post by overkalix »

whatever1 wrote: post #15
overkalix wrote: post #14

Earlier quoted context omitted.

Greg probably also knows SAS and AMPL, and has a good knowledge of ops research, which is within stone-tossing distance of whatever ML is pretending to be this week.

OR and ML have their own space in manufacturing. OR is perfect when you can describe explicitly what the decision space is and what the restrictions are. ML is great fit when you want to identify and use patterns. Quality control with machine vision is a good application for ML. NLP for PDF documents is a huge field for manufacturing as well. Companies have so much data in email attachments that they do not currently…

> OR is perfect when you can describe explicitly what the decision space is and what the restrictions are.

As opposed to having to figure it out later from the outputs of a black box?

> Quality control with machine vision is a good application for ML.

I can't imagine CV could be an actual replacement for actual SPC in many industries. There's a reason we need to take samples and stress test, analyze composition, etc.

> NLP for PDF documents is a huge field for manufacturing as well.

NPL could be big everywhere... if it provides actual value, which is not a given. ML has a lot of tangential applications (you could also say, better forecasting), but how will directly improve manufacturing processes?

I apologize for being abrasive, but I'm so tired of cs people descending upon all industries, plugging shit data into pytorch and doing shitty ML like it will automatically add value. Even more so in industrial engineering, which in my experience is full of people way better at math than computer scientists and requires a deep understanding of the product and the manufacturing process.

Re: Andrew Ng: Unbiggen AI

#20

Post by DeathArrow »

Pretty interesting. Mr. Ng claims that for some applications having a small set of quality data can be as good as using huge set of noisy data.

I wonder if, assuming the data is of highest quality, with minimal noise, having more data will matter for training or not. And if it matters, on what degree?

Post reply on HN