101–110 of 156 posts
Earlier quoted context omitted.
Yet it's the only strategy that we're absolutely certain produces intelligent life.
Are you absolutely certain we qualify as intelligent life? edit: This comment isn't a joke, I mean this seriously. Modeling a system after ourselves may not be the best idea if we're not the best possible system.
Earlier quoted context omitted.
Learning to crawl. Learning to perceive depth. Learning to walk. Learning to pick objects. Learning to hold balance. Learning to hold balance while picking an object. Learning to handle liquids (such as food or drink). Learning to avoid dangerous situations. Learning to ride on a bike. Learning to see. Learning to interact with other agents. And so on and so on. The list is almost endless. In all of those cases, if t…
All those things you described can be explained by having a pre-trained model built from DNA. Babies aren't a blank slate. Why would you even think they are? Do you think a baby learns to breath from blank-slate intelligence?
Clearly there is stuff pre-programmed in the DNA and some animals e.g. are able to hold balance right after birth. But apparently most of this stuff in case of humans is instructions how to build a learning system, rather then encoding the complexity of the world in a strand of DNA. I grant you, the evolution did not have the time to encode ways to handle lego bricks and yet 2yr olds spontaneously acquire ability to manipulate these objects way beyond what any contemporary robot can do.
Earlier quoted context omitted.
Language acquisition. Sure, parents will point at an object and say, "Ball", "Chair", "Food", etc. But they don't do this for the conjunctions and interjections and prepositions. Nor for most of the 10,000 words a child will learn in the first few years of life. Children learn the rules of grammar long before they realize there are any rules at all. Tenses and plurals, etc.
It's amazing how little people on this site know or understand intelligence, artificial and human. You would be surprised how much infants learn from listening around them and how long it takes to learn that language, and how a brain interprets language was formed through so many years of evolution to be highly tuned to learn language fast.
I understood your previous comments as diminishing the sheer amount of unsupervised learning we do as humans.
I don't know. I think backprop is probably utilized a lot in biological networks. Isn't that why we take tests in school? Obviously backprop doesn't make sense in an unsupervised setting. There's no label to backprop on. But here's an example: I see a stove eye is black when cold, then when I see it turn red, I touch it. Ow, it hurts. That's supervised learning. Don't touch things that are glowing red when they don't…
Maybe not. I remember (but don't have on hand) that people who studied a novel concept for 1 hour and then were tested on it for 1 hour, without seeing the result of the testing, bettered recalled the content than those who studied it for 2 hours. Granted, lots of variants of such a study should be performed before drawing strong conclusions, but it may be that back propagation isn't much used.
>I see a stove eye is black when cold, then when I see it turn red, I touch it. Ow, it hurts. That's supervised learning.
Yet most people learn to avoid glowing hot things without having to touch them. I can be told something red that isn't normally red is a danger, and I learn without ever having to touch one and be told my touching it was the wrong course of action. There is also interesting things that happen with sensation vs perception. Ever burn yourself with really hot metal that wasn't glowing? When I did, my fingers perceived extreme cold, not heat. You can also do the reverse by getting a rod made to look like it is glowing hot, chilling it until it is really cold, and hitting someone with it. They'll perceive being burned.
At first glance, the obvious solution seems to be to create intelligence the same way nature did: some sort of evolution. Some sort of algorithm where multiple "networks" mutate and reproduce in some way, in response to some fitness function. Mutations that make a network more fit result in an increased reproductive rate, while other mutations decrease that rate. Evolutionary algorithms have been around for a while b…
But nature also evolved a bunch of other organisms alongside humans that are fit in their own way. How do you evolve for just the (human level) intelligent entity and not a cockroach or rat? Point is that you're going to end up with a massive ecosystem of organisms if you could fully simulate an evolutionary history. It's not even clear that human level intelligence would evolve. It's only happened once in our planet…
One would have to figure out which evolutionary pressures to apply in order to select for the desired traits. For example, at some point in the past, humans and rats had a common ancestor. That ancestor had two offspring. One offspring's descendants faced a series of evolutionary pressures that led to rats. The other's descendants faced a different series of pressures, leading to humans. The designer of a multi-stage evolutionary algorithm could choose which evolutionary pressures to apply, and when to apply them. The designer could thus shape the evolution of the program in the desired direction.
The trick is knowing what pressures to apply in order to get the desired results. How could one do that? I have no idea! But if someone can figure it out, perhaps they could mimic what nature has blindly done.
EDIT: As for the worry of ending up with a menagerie of programs, there are a couple of things that I think could help. First, the designer must be careful not to take the natural-selection analogy too far. The real world has numerous diverse environments, each with myriad niches; and in each niche, different evolutionary pressures shape organisms. This sort of algorithm need not provide all these diverse niches. It could apply a single set of pressures to all the programs in it, pushing them either in the desired direction or to extinction. Second, if the designer could identify evolutionary states that are a "dead end" and unlikely to evolve in the desired direction, they could remove the dead-end programs. Whether such dead-end states can be reliably identified is anyone's guess.
Back in the nineteen-nineties I'd built a neural-net creator using a method other than back-propogation that was able to create nets with about a hundred neurons that played tic-tac-toe well or perfectly (depending on the net created that day.) This was on a 12 mhz 286 using what I called static point multiplication (based on shift instructions, not multiplication instructions.) So I went to Toronto to see Professor…
Earlier quoted context omitted.
Biological plausibility is only silly if you assume that biology has NOT in ~4 billion years of evolution managed to explore the design space and arrive at good general maxima/peak design for intelligent processes and are not merely stuck on the first local peak design encountered. It assumes that outside of this biological design space lies both lower peaks and much higher unexplored peaks. While it is likely that s…
1) One trouble with imitating biology is that stochastically driven processes tend to achieve maxima by exploiting environmental circumstances that are not intended or considered to be part of the design constraint. See for example the genetic-algorithm designed tone differentiator circuit on FPGA, which achieved extreme efficiency but the experimentors could not explain the circuit's behavior based on known theory,…
Point 2, this reads as some advantages compared to our current computers.
Earlier quoted context omitted.
I get what you're saying, but I don't think he was speaking explicitly about AGI. To me it didn't sound like they were leaving the applied problems he has worked on. Maybe I'm reading too much between the lines, but so far there haven't been any major successes in un-supervised learning models and he's suggesting here that rather than continue with the models that have had success up to this point, it would be more w…
> Maybe I'm reading too much between the lines, but so far there haven't been any major successes in un-supervised learning models Word2vec, a variant of which is Transform, Google's new machine translation framework which beats the pants off every other approach, to name one. The system it is replacing is also unsupervised.
Earlier quoted context omitted.
Each was overhyped at the time, and each hit its ceiling. That each was over-hyped I can buy. I'm less convinced of the latter part of that statement. Take ANN's. You would have thought that they had hit their ceiling at one time as well, but eventually the confluence of more compute power, more data, and better algorithms resulted in the recent resurgence. I don't see any reason to think that some of the other exist…
I think that machine learning, particularly deep learning, will increasingly be used in combination with some of the older AI techniques to produce hybrid systems, which will solve previously intractable problems. AlphaGo is a great example; of course it used deep learning, but it also used classical search, as well as reinforcement learning. The point is, progress isn't linear but exponential: each new technique mul…
This is what I believe as well. I've been doing some work lately on some older approaches to automated abductive inference, with an eye towards doing some of this kind of hybridization. No results to report yet, but I do think there's some meat on this bone.
Earlier quoted context omitted.
Even before: learning to swallow in the womb. There's definitely no reinforcement from other humans involved in that, and it's fundamental that they learn it in the womb because it would be too late to learn it with milk.
That's been passed down over 3.5 billion years to form the DNA that creates that part of the brain in an infant. AKA pre-trained.
Starting to suck when they feel something on their lips _is_ a reflex though, it isn't learnt.