I had success with GP because, as a programmer, I found it straightforward to map the problem domain onto a set of operations. Mapping the problem domain, IMHO, is the hardest part, and in all of my problems, it was hard to map it to differentiable functions. GP doesn't have that problem.
Evolution Is the New Deep Learning
91–100 of 242 posts
Re: Evolution Is the New Deep Learning
#92This is funny. I did my master thesis back in 2003 using "genetic" algorithms. Maybe I should try to get funding for an AI startup
I did a 1st year Uni project using GA to evolve neural network topology in 1996 ... maybe I should dig that out and get me some VC $.
That stuff is hot right now.
Re: Evolution Is the New Deep Learning
#93I submitted 4 bars of a Boston song and what the AI generated was incredible if you're deaf or enjoy bleeding from the ears.
Re: Evolution Is the New Deep Learning
#94Earlier quoted context omitted.
> We are not giving proper credit to how complex it is, and the multi-billion year developmental process that it took. Or we are simply not ready to accept that it's simply a big book of heuristics fine-tuned over biological eons. It's just big. We have too many interwoven, interdependent, synergistic faculties. Input, output, and a lot of mental stuff for making the right connections between the ins and the outs. Th…
Your last paragraph seems to contain the kind of overconfidence that I'm talking about. I don't understand how you can say "consciousness is simply X" or "it's easy to do that [if you handwave away the hard parts]." Clearly it's not that simple or easy, or we would have done it. We can't even create life from non-life. How can we begin to understand all the stuff you're talking about that's been layered on top? We do…
> We can't even create life from non-life.
We can't really define life in the first place.
But this is not necessarily to your favour. I think it's more of an indication of how the world doesn't fit into our... anthropomorphic way of thinking. That is, everything follows the laws of physics, no magic involved. We aren't special.
Re: Evolution Is the New Deep Learning
#95Earlier quoted context omitted.
> We are not giving proper credit to how complex it is, and the multi-billion year developmental process that it took. Or we are simply not ready to accept that it's simply a big book of heuristics fine-tuned over biological eons. It's just big. We have too many interwoven, interdependent, synergistic faculties. Input, output, and a lot of mental stuff for making the right connections between the ins and the outs. Th…
Your last paragraph seems to contain the kind of overconfidence that I'm talking about. I don't understand how you can say "consciousness is simply X" or "it's easy to do that [if you handwave away the hard parts]." Clearly it's not that simple or easy, or we would have done it. We can't even create life from non-life. How can we begin to understand all the stuff you're talking about that's been layered on top? We do…
I'm assuming a simpler model, no need for magic, because so far I don't see what behavior/data this simple model cannot explain.
> Clearly it's not that simple or easy, or we would have done it.
We don't have the computational power yet. Not to mention the vast amount of development required. Think of the climate models, that are huge (millions of lines of code), but they're still nowhere near complete enough, and they only have to model sunlight (Earth's rotation, orbital position, albedo), clouds, flows (winds and currents), some topography (big mountains, big flats), ice (melting, freezing), some chemistry (CO2, salts). And they only have to match a simple graph, not the behavior of a human mind (eg Turing test).
So, it's not easy, even if simple.
> We can't even create life from non-life.
We understand life. Cells, RNA, DNA, proteins, mitochondria, actins, etc. It's big, it's a lot of moving parts, and we understand it, but we can't just pop a big chunk of matter into an atomic assembler and make a cell.
And I think intelligence/sentience is similar. It's big, not magic.
Re: Evolution Is the New Deep Learning
#96Earlier quoted context omitted.
Genetic algorithms 'invent new things' by doing exactly the same thing as other optimization methods - tweaking the values of particular parameters. The only difference in is how they do so.
Isn't this kind of a banal statement? Doesn't everything that learns in any way whatsoever "tweak values of particular parameters"?
It's banal but sensible in response to someone claiming a large difference between two fundamentally similar things. Either both or neither "invents".
Re: Evolution Is the New Deep Learning
#97Earlier quoted context omitted.
Genetic algorithms 'invent new things' by doing exactly the same thing as other optimization methods - tweaking the values of particular parameters. The only difference in is how they do so.
Optimizing genes leading to an intelligent neural network is what biology did basically. If your optimization problem changes the behavior of something able to compute in order to maximize the ability to solve a task, then that's inventing things IMHO.
Which is then no different to DL or any other learning algorithm.
Re: Evolution Is the New Deep Learning
#98Earlier quoted context omitted.
Genetic algorithms 'invent new things' by doing exactly the same thing as other optimization methods - tweaking the values of particular parameters. The only difference in is how they do so.
Optimizing genes leading to an intelligent neural network is what biology did basically. If your optimization problem changes the behavior of something able to compute in order to maximize the ability to solve a task, then that's inventing things IMHO.
Respectfully,
We've had "learning-to-learn" algorithms for a few years now. Including LSTMs that can be learn the gradients for other LSTMs, and deep-RL algorithms that can optimize neural networks.
Its hard to say its inventing new things... there is still a clear goal - a loss - and we are optimizing it; poorly in the case of genetic algorithms.
I would say genetic methods are simply poorly described reinforcement learning problems, which means that there is a 1-to-1 mapping between Deep Learning and Genetic Algorithms.
PS. Making a distinction between "Genetic Algorithms" and "Genetic Programming" is like calling Deep Learning "Differential Programming" -- changing the name of a thing does not change the thing itself.
Re: Evolution Is the New Deep Learning
#99Having studied this extensively back when they were called Genetic Algorithms, I would like to offer a few insights. 1) One of the biggest reasons they fell out of favor for more "mathematical" approaches was that no one could really explain why exactly they worked. It makes sense on the surface that "survival of the fittest" and doing something akin to multiple stochastic gradient descents would work, but no one has…
1) One of the biggest reasons they fell out of favor for more "mathematical" approaches was that no one could really explain why exactly they worked. Kind of like how nobody can really explain how the brain works, or life in general. My gut feeling is that it is hubris to think that we are going to "figure out" intelligence with increasingly sophisticated mathematical models anytime soon. We are not giving proper cre…
While I can’t predict that we can solve intelligence completely any time soon, I would argue that machine intelligence allows us to examine the phenomena of intelligence more thoroughly than anything else in the past.
I would expect this to yield new insights. So at a minimum, I’d think we can learn more now than we have been able to before. Maybe that will lead to a great increase in understanding of intelligence or a slight one, but it will lead to an increase in our understanding of the phenomena.
Re: Evolution Is the New Deep Learning
#100Earlier quoted context omitted.
Genetic algorithms yes but genetic programming is actually able to invent new things. Possibly optimizing the set of weight of a neural network to solve a given problem is more similar to genetic programming, if you see the NN as a computational unit.
Genetic algorithms 'invent new things' by doing exactly the same thing as other optimization methods - tweaking the values of particular parameters. The only difference in is how they do so.
While I agree that the quotes you made sound more like marketing than science, antirez has a point here: They might be using GA to evolve a new activation function represented as a GP tree.
But that's just my guess. And even so, I am not terribly convinced that would lead to astonishingly new activation functions that are better than the known ones.