Brief description of the issue for non-experts: Supervised learning You can judge the output of your network against ground truth. You say that's a cat? Nope, it's a dog! And then slightly adjust your network so it's less likely to give that wrong answer in the future. How exactly you adjust the network is what backpropagation describes (in combination with something called a learning rate). Unsupervised learning You…
Actually: >>Most of the time for biological intelligence there isn't an oracle describing the truth at every moment of life This is false for most of learning of human intelligence. Most of schooled human intelligence looks like supervised learning. Innate intelligence and other animal intelligence looks like pre-trained models and some reinforcement learning.
Artificial intelligence pioneer says we need to start over
81–90 of 156 posts
Re: Artificial intelligence pioneer says we need to start over
#82Machine learning may be nearing its ceiling. The history of AI goes in cycles. Someone has a good idea which solves some problems, followed by "strong AI Real Soon Now" enthusiasm, followed by that idea hitting its ceiling. AI has been through search, backtracking, the General Problem Solver, hill-climbing, and expert systems. Each was overhyped at the time, and each hit its ceiling. The big difference this time is t…
That each was over-hyped I can buy. I'm less convinced of the latter part of that statement. Take ANN's. You would have thought that they had hit their ceiling at one time as well, but eventually the confluence of more compute power, more data, and better algorithms resulted in the recent resurgence. I don't see any reason to think that some of the other existing AI techniques might not also experience an explosion in utility, if combined with massive increases in compute power, more data, and better algorithms as well.
Re: Artificial intelligence pioneer says we need to start over
#83Earlier quoted context omitted.
That's the one. I'm actually just finishing it up now, so it's fresh enough in my mind that it jumps out at me in application.
I'd be careful reading it: Kuhn is excellent at building narratives, and what is more narrative friendly than revolution? In reality science is far more messy than he acknowledges. Unfortunately while I think many of Kuhn's observations are interesting, I'm confused what you think scientists might learn from them. After all, the work in question is science, not philosophy of science, and frankly it seems a lot of tim…
Funny you should say that. I take Kuhn's whole point to be that science is far messier than the empiricists acknowledge. What is the complication that you think Kuhn overlooks?
Re: Artificial intelligence pioneer says we need to start over
#84Machine learning may be nearing its ceiling. The history of AI goes in cycles. Someone has a good idea which solves some problems, followed by "strong AI Real Soon Now" enthusiasm, followed by that idea hitting its ceiling. AI has been through search, backtracking, the General Problem Solver, hill-climbing, and expert systems. Each was overhyped at the time, and each hit its ceiling. The big difference this time is t…
The mindset that I feel is lacking in the ML/AI community is that we should be creating systems that can be augmented and improved using AI, the product itself should stand alone without using any AI at all. This is similar to how JIT compilation can improve the machine code as the program runs. Part of how our own brains work is that the cerebral cortex doesn't really have any direct control over the body at all, it can only watch and identify patterns and project intentions to the the rest of the brain/body. This is a passive structure initially, it observes both the environment and how the lower structure interact with that environment. Only after it's observed and learned the "commands" can it begin to influence how the body moves around and interacts with the world. You could look at the CLI terminal as an example, on it's own I can interact and use it perfectly fine. But lets say I added some patterning matching and prediction AI that could make really good suggestions about what command I will type next, at some point you could promote these suggestions so that they always execute when the AI predicts you would use it. This is exactly how the brain works.
Most of my ideas of applied AI revolve around improving our HID interfaces to computers themselves. For example, I'd like to reduce our chances of developing RSI related to keyboard/mouse/trackpad usage and I feel that chorded keyboards are more ergonomic. But chorded keyboards are slower to use and harder to learn. The mapping of the chords is where I plan to apply some AI, initially I'm going to randomize how chords are bound to character inputs, these random values will use many fingers to input 1 character. As the user learns and uses the chords to input characters the AI can identify common patterns and promote those patterns into chords that use less and less fingers to input, thus improving the input speed potential of the user. The next place to use AI in this project would be autocorrection and using the user's input context(program, filetype, time, surrounding text) to continue improving these bindings and making completion suggestions.
Re: Artificial intelligence pioneer says we need to start over
#85Earlier quoted context omitted.
I don't think that's correct. There are a lot of people who should know better that think that the current ml technology will lead to agi.
Just ML is never going to lead to an AGI. There are lots of other things that need to be first understood about intelligence, common sense reasoning, cognition and other things which we don't have a clue of (though some think they know it all), to build an AGI.
I think there's a chance it could in principle. Or at least, I think it could yield something close. If human intelligence is largely rooted in a neural network with billions of neurons and trillions of synapses, then it seems reasonable to think that, in principle, a sufficiently deep/wide ANN could show "intelligence".
But here's the rub. Doing that would be silly. We already know how to make a neural network with human like intelligence using billions of neurons and trillions of synapses... it just takes a man, a woman, and about 9 months. :-)
Also consider the power consumption of an ANN of that scale, given the current state of things.
So yeah, to achieve anything truly AGI like, which doesn't need it's own nuclear power plant, I suspect we're going to need other approaches in addition to today's ML/DL stuff.
Re: Artificial intelligence pioneer says we need to start over
#86Evolutionary algorithms have been around for a while but haven't really taken off. Maybe the problem is that you can't get from zero to intelligence with a single fitness function.
Think about it: the "fitness function" for our single-called ancestors was not how well they could thrive in a post-industrial service economy. Nor was it how well they could thrive as hunter-gatherers in the African Savannah. It was a very, very different fitness function.
To get from zero to humans, nature had to evolve single-called organisms, then mitochondria, then basic multi-celled organisms, then animals that lived in the ocean, then amphibious ones, then land-dwelling ones, then mammals, then apes, then early humans, then modern humans.
By the way, I left out the overwhelming majority of the steps.
At each step, the evolutionary pressures on these organisms were wildly different. Each set of pressures was necessary in order for the next layer of complexity to evolve.
I think that if we want to evolve an intelligent entity, we would probably have to do it like this.
Re: Artificial intelligence pioneer says we need to start over
#87Machine learning may be nearing its ceiling. The history of AI goes in cycles. Someone has a good idea which solves some problems, followed by "strong AI Real Soon Now" enthusiasm, followed by that idea hitting its ceiling. AI has been through search, backtracking, the General Problem Solver, hill-climbing, and expert systems. Each was overhyped at the time, and each hit its ceiling. The big difference this time is t…
Each was overhyped at the time, and each hit its ceiling. That each was over-hyped I can buy. I'm less convinced of the latter part of that statement. Take ANN's. You would have thought that they had hit their ceiling at one time as well, but eventually the confluence of more compute power, more data, and better algorithms resulted in the recent resurgence. I don't see any reason to think that some of the other exist…
The point is, progress isn't linear but exponential: each new technique multiplies our capabilities.
Re: Artificial intelligence pioneer says we need to start over
#88Earlier quoted context omitted.
>> Most of the time for biological intelligence there isn't an oracle describing the truth at every moment of life to judge actions/decisions against. [emphasis mine] > There is: Death, without reproducing. The problem is that it's not describing the truth at every moment of life, just the total approximate fitness of that lifeform by the end of its lifetime. It would be time consuming and cost-inefficient to simulat…
Yet it's the only strategy that we're absolutely certain produces intelligent life.
Although I do tend to think that given enough time (some millions of years), other intelligence will evolve on earth, from parrots or elephants or dolphins or octipuses or something. Though there's no evidence of a brain growth spurt that led to us, in other species; and it's not settled why we had it.
Re: Artificial intelligence pioneer says we need to start over
#89Earlier quoted context omitted.
Biological plausibility is just silly. What it mostly does is just put artificial constraints on the problem that doesn't need to be there. It's all mathematics. However you want to interpret or metaphorize the mathematics is up to each math-phobic field, but constraining yourself to what randomly evolved serves little good.
Biological plausibility is only silly if you assume that biology has NOT in ~4 billion years of evolution managed to explore the design space and arrive at good general maxima/peak design for intelligent processes and are not merely stuck on the first local peak design encountered. It assumes that outside of this biological design space lies both lower peaks and much higher unexplored peaks. While it is likely that s…
2) Biology has to satisfy many constraints which are not hard limitations on artificial brains, such as:
* Thermodynamics concerns, surface-area to volume ratio
* A tightly controlled operating temperature of 97-99 F
* Weight (wetware brains must float on CSF because they belong to moving animals)
* Size
Re: Artificial intelligence pioneer says we need to start over
#90At first glance, the obvious solution seems to be to create intelligence the same way nature did: some sort of evolution. Some sort of algorithm where multiple "networks" mutate and reproduce in some way, in response to some fitness function. Mutations that make a network more fit result in an increased reproductive rate, while other mutations decrease that rate. Evolutionary algorithms have been around for a while b…
Point is that you're going to end up with a massive ecosystem of organisms if you could fully simulate an evolutionary history. It's not even clear that human level intelligence would evolve. It's only happened once in our planet's history. We might be flukes.
You might have to run the massive simulation a thousand times and somehow be able to pinpoint the human-level intelligence when it does succeed.