31–40 of 156 posts
I think the main problem is that we're incredibly dependent on advances in computing power to make advances in deep learning. IMHO, we've only made incremental software progress in using deep networks since the advent of convolutional networks.
Yes back-propagation can help classify and predict many things.
But semantic information needs more research. For now the state of the art is Cyc.
Brief description of the issue for non-experts: Supervised learning You can judge the output of your network against ground truth. You say that's a cat? Nope, it's a dog! And then slightly adjust your network so it's less likely to give that wrong answer in the future. How exactly you adjust the network is what backpropagation describes (in combination with something called a learning rate). Unsupervised learning You…
If the output of your networks performs well enough when it confronts the real world, you get to reproduce before you die, and some of the essence of your network gets to live for another generation.
If not, you die, and your network doesn't get to contribute to the next generation.
But here's an example:
I see a stove eye is black when cold, then when I see it turn red, I touch it. Ow, it hurts. That's supervised learning. Don't touch things that are glowing red when they don't normally glow red. Now, I see an iron pole glowing red. It's not a spiral like the stove eye, but it's not normally glowing red and now it is. I'd better not touch it. I can deduce that these two objects are made out of a similar material since they're normally black or gray and now glow red. I can also deduce that something glowing red means it's hot. That's unsupervised learning.
To me, unsupervised learning is looking at a fluffy object with four legs, eyes, a nose, tail, it moves, etc. and knowing that it's some kind of animal. Creating groups of things like k-means. Supervised learning is your mom telling you that this one example is a tiger. Unsupervised learning is understanding the delta between your one labeled example and the rest of the examples in your animal group is largest when the animal is NOT a tiger.
Brief description of the issue for non-experts: Supervised learning You can judge the output of your network against ground truth. You say that's a cat? Nope, it's a dog! And then slightly adjust your network so it's less likely to give that wrong answer in the future. How exactly you adjust the network is what backpropagation describes (in combination with something called a learning rate). Unsupervised learning You…
>>Most of the time for biological intelligence there isn't an oracle describing the truth at every moment of life
This is false for most of learning of human intelligence. Most of schooled human intelligence looks like supervised learning.
Innate intelligence and other animal intelligence looks like pre-trained models and some reinforcement learning.
Earlier quoted context omitted.
You need to learn without someone telling you what the answer is. Most of the time for biological intelligence there isn't an oracle describing the truth at every moment of life to judge actions/decisions against. There is: Death, without reproducing. If an organism dies quickly but left children, they were biologically successful. If you didn't, then your evolutionary branch goes away. I'm not sure this can be simul…
>> Most of the time for biological intelligence there isn't an oracle describing the truth at every moment of life to judge actions/decisions against. [emphasis mine] > There is: Death, without reproducing. The problem is that it's not describing the truth at every moment of life, just the total approximate fitness of that lifeform by the end of its lifetime. It would be time consuming and cost-inefficient to simulat…
Earlier quoted context omitted.
>> Most of the time for biological intelligence there isn't an oracle describing the truth at every moment of life to judge actions/decisions against. [emphasis mine] > There is: Death, without reproducing. The problem is that it's not describing the truth at every moment of life, just the total approximate fitness of that lifeform by the end of its lifetime. It would be time consuming and cost-inefficient to simulat…
Yet it's the only strategy that we're absolutely certain produces intelligent life.
edit: This comment isn't a joke, I mean this seriously. Modeling a system after ourselves may not be the best idea if we're not the best possible system.
I don't know. I think backprop is probably utilized a lot in biological networks. Isn't that why we take tests in school? Obviously backprop doesn't make sense in an unsupervised setting. There's no label to backprop on. But here's an example: I see a stove eye is black when cold, then when I see it turn red, I touch it. Ow, it hurts. That's supervised learning. Don't touch things that are glowing red when they don't…
Earlier quoted context omitted.
Yet it's the only strategy that we're absolutely certain produces intelligent life.
Are you absolutely certain we qualify as intelligent life? edit: This comment isn't a joke, I mean this seriously. Modeling a system after ourselves may not be the best idea if we're not the best possible system.
Brief description of the issue for non-experts: Supervised learning You can judge the output of your network against ground truth. You say that's a cat? Nope, it's a dog! And then slightly adjust your network so it's less likely to give that wrong answer in the future. How exactly you adjust the network is what backpropagation describes (in combination with something called a learning rate). Unsupervised learning You…
Actually: >>Most of the time for biological intelligence there isn't an oracle describing the truth at every moment of life This is false for most of learning of human intelligence. Most of schooled human intelligence looks like supervised learning. Innate intelligence and other animal intelligence looks like pre-trained models and some reinforcement learning.