Genetic Algorithms for Training Deep Neural Networks for Reinforcement Learning
1–10 of 47 posts
Re: Genetic Algorithms for Training Deep Neural Networks for Reinforcement Learning
#2I'm not well-versed in machine learning, so I just want to make sure I understand the general point here: these researchers trained a competitive deep neural network without using gradient descent? Does this mean neural networks might become more useful for discrete optimization problems without gradient information, like program synthesis?
Re: Genetic Algorithms for Training Deep Neural Networks for Reinforcement Learning
#3I'm not well-versed in machine learning, so I just want to make sure I understand the general point here: these researchers trained a competitive deep neural network without using gradient descent? Does this mean neural networks might become more useful for discrete optimization problems without gradient information, like program synthesis?
[deleted]
Re: Genetic Algorithms for Training Deep Neural Networks for Reinforcement Learning
#4Well,this is surprising. Paper doesn't mention image recognotion and voice recognition at all. Is this because they are not interesting for new research or because in theese applications results were worse than that of gradient methods?
Re: Genetic Algorithms for Training Deep Neural Networks for Reinforcement Learning
#5Note that using genetic algorithms to come up with initial weights for neural networks was the state of the art in the late 90s/early 2000s. So this paper is not as novel as it seems, but it's good to have it for reference.
My feeling is that since shallow networks can be made to have equivalent accuracy to deep networks, that the real challenge isn't topology but training. Hobbyists have access to so much processing power with GPUs now that they can explore techniques that weren't practical for experts 20 years ago. So we may see training speed increase by a few orders of magnitude using techniques besides gradient descent (maybe quantum computing someday, who knows).
The big question though is how to combine networks into hierarchies so that the number of behaviors that can be learned is no longer limited (since pattern recognition is largely a solved problem). I think the way GAs fit in is that they make it much easier to understand and build simple NNs, and possibly train hierarchies or discover topologies that aren't immediately obvious.
Re: Genetic Algorithms for Training Deep Neural Networks for Reinforcement Learning
#6I'm not well-versed in machine learning, so I just want to make sure I understand the general point here: these researchers trained a competitive deep neural network without using gradient descent? Does this mean neural networks might become more useful for discrete optimization problems without gradient information, like program synthesis?
Neuroevolution is nothing new. I don't know why people think this is something big/innovative.
Re: Genetic Algorithms for Training Deep Neural Networks for Reinforcement Learning
#7Without reading this specific paper, I think this is the future. If not genetic search, then some other metaheuristic. I think this is only really credible road to AGI in the near term, as it seems to be largely a hardware limitation that will be alleviated in the next few years.
Through the history of deep learning, the frontier has been networks that can be trained to some feedback of whether they're working or not in ~6-8 days. Within a few years there will almost certainly be ASICs that allow us to train hundreds of thousands or millions of networks in several weeks, relative to several weeks for one of the same network in 2017. This will let us start exploring things like evolving networks seriously.
Fundamentally, we know neural networks can instantiate general intelligence, and we know genetic search is capable of finding the right neural networks. There are big differences between the CS and biological versions of each, but it's striking that the big breakthrough in "AI" was deep neural networks and not anything else.
When I think about the difference between AlphaZero and human intelligence, I don't think it's "more intelligence." AlphaZero seems perfectly intelligent to me: I think the difference is more about the selective pressures that produced us. AlphaZero is a reflection of its environment and the process by which it developed in that environment. I would be shocked the future of deep learning continues to be hand-design from human intuition.
Edit: looking at the link, I want to caveat the above by saying this paper may or may not be "it," but directionally I think the idea is underrated and erroneously out of favor now as neural networks were in 2009. Ken Stanley (coauthor on the linked paper) in particular has been hung up on one particular approach, NEAT, since like 2002 that is kind of interesting but definitely not the end-all.
Re: Genetic Algorithms for Training Deep Neural Networks for Reinforcement Learning
#8Note that using genetic algorithms to come up with initial weights for neural networks was the state of the art in the late 90s/early 2000s. So this paper is not as novel as it seems, but it's good to have it for reference. My feeling is that since shallow networks can be made to have equivalent accuracy to deep networks, that the real challenge isn't topology but training. Hobbyists have access to so much processing…
Deep learning wasn't novel in 2012 either - it was the removal of a hardware limitation that made it compelling again. I think the same is true for evolving DNNs, but I don't know if the available compute power is there yet.
> My feeling is that since shallow networks can be made to have equivalent accuracy to deep networks, that the real challenge isn't topology but training.
This is not really true though... even very shallow neural networks can be universal function approximators in a trivial sense because they can be lookup tables, but they are really not expressive enough to generalize well and lack a lot of the expressivity of deep networks.
Re: Genetic Algorithms for Training Deep Neural Networks for Reinforcement Learning
#9I'm not well-versed in machine learning, so I just want to make sure I understand the general point here: these researchers trained a competitive deep neural network without using gradient descent? Does this mean neural networks might become more useful for discrete optimization problems without gradient information, like program synthesis?
Yes and maybe.
Re: Genetic Algorithms for Training Deep Neural Networks for Reinforcement Learning
#10I'm not well-versed in machine learning, so I just want to make sure I understand the general point here: these researchers trained a competitive deep neural network without using gradient descent? Does this mean neural networks might become more useful for discrete optimization problems without gradient information, like program synthesis?
The Blondie24 book was published in 2002.
https://www.amazon.com/product-reviews/1558607838
People need to recognize that this is not a new idea. It may have been groundbreaking in the early 2000s, but Genetic Algorithms + Neural Nets are a relatively old strategy at this point.
Definitely read the book if you want a bit of a throwback. When cutting-edge AI research was about beating players on Yahoo-games Checkers with new techniques.