21–30 of 107 posts
Is there something fundamental we are missing in going about building these deep learning stuff ?
Has anyone figured out how to debug and/or analyze deep ANN? That's the problem I always had, you may get them into a trained state, but good luck figuring out any reason 'why' they ended up in that state (or even what that state really is).
> That's the problem I always had, you may get them into a trained state, but good luck figuring out any reason 'why' they ended up in that state (or even what that state really is). Can you give a specific example of what you mean? I ask because I see this sentiment often, but primarily from people who are very new to deep learning. You can definitely debug a neural network. You mostly want to look at metrics from t…
When it doesn't discover that it's a stop sign, how do you debug it? Did it recognize the shape.. who knows?
Has anyone figured out how to debug and/or analyze deep ANN? That's the problem I always had, you may get them into a trained state, but good luck figuring out any reason 'why' they ended up in that state (or even what that state really is).
They tweak the input to maximize the response of specific neurons somewhere in the middle of the network to figure out what those neurons actually "learned".
If I need to show 1 million cat images to train a neural net to see cats, I wonder how a human brain can figure out cats of any kind just by seeing one or two. Is there something fundamental we are missing in going about building these deep learning stuff ?
Even a human brain has to train for ~4-5 months to become interested in shapes (https://en.wikipedia.org/wiki/Infant_visual_development). Once the human brain has been trained for these basic shapes for a while, it is able to quickly break down a new class (i.e. a cat) and recognize similar patterns in other images. This is something that is very similar to the way that training a deep NN works.
Also don't forget that the current NN are being trained mainly for photos, not moving images. A brain may recognize a cat by its tail-wagging or fur movements, which is a dimension that is completely missing from still images.
Also check out this similar post and discussion: https://news.ycombinator.com/item?id=9247851
If I need to show 1 million cat images to train a neural net to see cats, I wonder how a human brain can figure out cats of any kind just by seeing one or two. Is there something fundamental we are missing in going about building these deep learning stuff ?
An untrained neural net has to learn everything from scratch (ha!), from pixel values to "knowing" what a cat is. There is some work on decreasing the amount of data needed to learn, but it's a very tricky subject.
If I need to show 1 million cat images to train a neural net to see cats, I wonder how a human brain can figure out cats of any kind just by seeing one or two. Is there something fundamental we are missing in going about building these deep learning stuff ?
Great question. Even a human brain has to train for ~4-5 months to become interested in shapes ( https://en.wikipedia.org/wiki/Infant_visual_development ). Once the human brain has been trained for these basic shapes for a while, it is able to quickly break down a new class (i.e. a cat) and recognize similar patterns in other images. This is something that is very similar to the way that training a deep NN works. Als…
Everyone who wonder how (on a superficial level) grown up humans are so good at learning new categories really should spend time around babies and toddlers and children for this reason...
You quickly realise how much training and brain development it actually takes before we're capable of doing much.
Also, an interesting read related to this would 'the AI effect'[0]. A lot of the stuff Deep Learning/Machine Learning is able to do today would be looked at as something that only 'true' AI (whatever consensus on what that means is; I think of it as AGI) would be able to do. But as soon as we are able to solve a problem that we think (feel?) only true AI (AGI) would be able to solve, as soon as we know how it was sol…
Also, an interesting read related to this would 'the AI effect'[0]. A lot of the stuff Deep Learning/Machine Learning is able to do today would be looked at as something that only 'true' AI (whatever consensus on what that means is; I think of it as AGI) would be able to do. But as soon as we are able to solve a problem that we think (feel?) only true AI (AGI) would be able to solve, as soon as we know how it was sol…
Earlier quoted context omitted.
> That's the problem I always had, you may get them into a trained state, but good luck figuring out any reason 'why' they ended up in that state (or even what that state really is). Can you give a specific example of what you mean? I ask because I see this sentiment often, but primarily from people who are very new to deep learning. You can definitely debug a neural network. You mostly want to look at metrics from t…
You cannot say: "This image was classified as a stop sign, because this part recognized the shape and this part recognized the color, and this part the text", which you could do with other approaches. When it doesn't discover that it's a stop sign, how do you debug it? Did it recognize the shape.. who knows?
This paper does a good job of showing how CNNs learn a hierarchy of increasingly complex features to classify images: http://arxiv.org/abs/1311.2901