It's time we begin to incorporate meso to large scale structure into neural nets. The human brain is able to accomplish what it can because of the way it organizes neurons into clusters and sub organs (thalamus, Broca's area etc.)
Artificial intelligence pioneer says we need to start over
71–80 of 156 posts
Re: Artificial intelligence pioneer says we need to start over
#72Earlier quoted context omitted.
Please describe what "unsupervised learning" a child may be doing.
Learning to crawl. Learning to perceive depth. Learning to walk. Learning to pick objects. Learning to hold balance. Learning to hold balance while picking an object. Learning to handle liquids (such as food or drink). Learning to avoid dangerous situations. Learning to ride on a bike. Learning to see. Learning to interact with other agents. And so on and so on. The list is almost endless. In all of those cases, if t…
Even before: learning to swallow in the womb. There's definitely no reinforcement from other humans involved in that, and it's fundamental that they learn it in the womb because it would be too late to learn it with milk.
Re: Artificial intelligence pioneer says we need to start over
#73Earlier quoted context omitted.
I think his quote is out of context. If he is saying that back propagation networks alone are insufficient for AGI, that is neither surprising nor controversial. Hinton has never been working on the AGI problem. It sounds more like the author is making use of the quotes he/she got from Hinton in the most clickbaity way possible.
I get what you're saying, but I don't think he was speaking explicitly about AGI. To me it didn't sound like they were leaving the applied problems he has worked on. Maybe I'm reading too much between the lines, but so far there haven't been any major successes in un-supervised learning models and he's suggesting here that rather than continue with the models that have had success up to this point, it would be more w…
> Maybe I'm reading too much between the lines, but so far there haven't been any major successes in un-supervised learning models
Word2vec, a variant of which is Transform, Google's new machine translation framework which beats the pants off every other approach, to name one. The system it is replacing is also unsupervised.
Re: Artificial intelligence pioneer says we need to start over
#74Earlier quoted context omitted.
I think the point is that just because companies are making money off of AI advances, that doesn't necessarily mean we won't hit another AI winter once we hit a ceiling with current techniques.
Okay but what's the relevance of expert systems to this argument?
This isn't the first round of AI that's been useful for "large-scale profitable applications"
Re: Artificial intelligence pioneer says we need to start over
#75Back in the nineteen-nineties I'd built a neural-net creator using a method other than back-propogation that was able to create nets with about a hundred neurons that played tic-tac-toe well or perfectly (depending on the net created that day.) This was on a 12 mhz 286 using what I called static point multiplication (based on shift instructions, not multiplication instructions.) So I went to Toronto to see Professor Hinton, to share. My not being a big back-propagation fan was one thing that divided us before meeting him, but otherwise I thought we'd be pretty much on the same page.
It didn't go well. I happened to mention that I thought parallel programming would be a big help speeding up neural nets, and that I'd toured such a multi-processor computer in Alberta. He blew up. I never got to explain what I'd done with neural nets. He tossed me out of his office summarily and wouldn't listen to another word I had to say - somehow he'd gotten the idea in his head that I was a salesman for the company whose machine I'd seen, which machine was too weird to be considered anyway, and absolutely nothing I could say would persuade him that I wasn't a salesman. So the nets I'd created and methods other than back-propagation never got mentioned. Weirdest meeting of my life, and the end of my desire to do anything with neural nets, since other academic departments were even more violently rejecting of the idea back then.
Re: Artificial intelligence pioneer says we need to start over
#76Brief description of the issue for non-experts: Supervised learning You can judge the output of your network against ground truth. You say that's a cat? Nope, it's a dog! And then slightly adjust your network so it's less likely to give that wrong answer in the future. How exactly you adjust the network is what backpropagation describes (in combination with something called a learning rate). Unsupervised learning You…
Pain and pleasure are the oracles. Maybe not at every moment, but within a given lag.
Re: Artificial intelligence pioneer says we need to start over
#77Earlier quoted context omitted.
I think the point is that just because companies are making money off of AI advances, that doesn't necessarily mean we won't hit another AI winter once we hit a ceiling with current techniques.
Okay but what's the relevance of expert systems to this argument?
Expert systems are one of the, supposedly not profitable, previous AI booms the preceded an AI winter.
My point is, they were and are profitable, but clearly that didn't keep winter at bay.
Re: Artificial intelligence pioneer says we need to start over
#78Back in the nineteen-nineties I'd built a neural-net creator using a method other than back-propogation that was able to create nets with about a hundred neurons that played tic-tac-toe well or perfectly (depending on the net created that day.) This was on a 12 mhz 286 using what I called static point multiplication (based on shift instructions, not multiplication instructions.) So I went to Toronto to see Professor…
What's the idea? Sounds interesting? Publish a paper on it? Or put up a blog post?
Re: Artificial intelligence pioneer says we need to start over
#79Back in the nineteen-nineties I'd built a neural-net creator using a method other than back-propogation that was able to create nets with about a hundred neurons that played tic-tac-toe well or perfectly (depending on the net created that day.) This was on a 12 mhz 286 using what I called static point multiplication (based on shift instructions, not multiplication instructions.) So I went to Toronto to see Professor…
[deleted]
Re: Artificial intelligence pioneer says we need to start over
#80Brief description of the issue for non-experts: Supervised learning You can judge the output of your network against ground truth. You say that's a cat? Nope, it's a dog! And then slightly adjust your network so it's less likely to give that wrong answer in the future. How exactly you adjust the network is what backpropagation describes (in combination with something called a learning rate). Unsupervised learning You…
This is codifying "instinct" which presumably has already been done at least once (via DNA in some species).