I'm surprised Uber is funding general AI research given their reported financial situation.
Genetic Algorithms for Training Deep Neural Networks for Reinforcement Learning
11–20 of 47 posts
Re: Genetic Algorithms for Training Deep Neural Networks for Reinforcement Learning
#12i wonder if something like this is the abstraction where we as humans stop understanding it.
I do know that we define the cost function / the goal of it, but from using evolutionary Technic to build the network, there is only x layers left to add to create any net.
Re: Genetic Algorithms for Training Deep Neural Networks for Reinforcement Learning
#13I'm not well-versed in machine learning, so I just want to make sure I understand the general point here: these researchers trained a competitive deep neural network without using gradient descent? Does this mean neural networks might become more useful for discrete optimization problems without gradient information, like program synthesis?
Neuroevolution is nothing new. I don't know why people think this is something big/innovative.
Because as far as I know, genetic algorithms perform poorly compared to basically every other optimization technique. I'm under the impression that they're not actually used for anything interesting these days, now that the initial hype about implementing natural selection on a computer has died down.
Stochastic hill climbing as a baseline method for evaluating genetic algorithms: http://papers.nips.cc/paper/1172-stochastic-hillclimbing-as-...
When will a genetic algorithm outperform hill climbing? http://web.cecs.pdx.edu/~mm/nips93.pdf
A GA being competitive with modern gradient based methods is very surprising to me.
Re: Genetic Algorithms for Training Deep Neural Networks for Reinforcement Learning
#14Note that using genetic algorithms to come up with initial weights for neural networks was the state of the art in the late 90s/early 2000s. So this paper is not as novel as it seems, but it's good to have it for reference. My feeling is that since shallow networks can be made to have equivalent accuracy to deep networks, that the real challenge isn't topology but training. Hobbyists have access to so much processing…
I'm new to the field. What makes you say that shallow and deep networks have equivalent accuracy? Is there a recent publication/experiment I missed? Thanks!
Re: Genetic Algorithms for Training Deep Neural Networks for Reinforcement Learning
#15Note that using genetic algorithms to come up with initial weights for neural networks was the state of the art in the late 90s/early 2000s. So this paper is not as novel as it seems, but it's good to have it for reference. My feeling is that since shallow networks can be made to have equivalent accuracy to deep networks, that the real challenge isn't topology but training. Hobbyists have access to so much processing…
I'm new to the field. What makes you say that shallow and deep networks have equivalent accuracy? Is there a recent publication/experiment I missed? Thanks!
https://arxiv.org/abs/1703.02660
Disclosure: I contributed to the linked work.
To do simple tasks, you don’t even need neural networks, in short. If you want to do complex tasks, training the neural net isn’t the problem, but a NN can indeed represent more than say a linear policy. The issue is getting the NN to do the complex thing correctly: it’s more of an exploration problem than function approximation problem.
Re: Genetic Algorithms for Training Deep Neural Networks for Reinforcement Learning
#16Earlier quoted context omitted.
Neuroevolution is nothing new. I don't know why people think this is something big/innovative.
Because as far as I know, genetic algorithms perform poorly compared to basically every other optimization technique. I'm under the impression that they're not actually used for anything interesting these days, now that the initial hype about implementing natural selection on a computer has died down. Stochastic hill climbing as a baseline method for evaluating genetic algorithms: http://papers.nips.cc/paper/1172-sto…
Try optimizing for the shape of an antenna using hill climbing.
Thing about evolutionary networks is that they can evolve topology, which usually is just a thing decided by people implementing NNs (and not guaranteed to be anywhere near most optimal). I don't see why given enough time GAs (or EAs) could not outperform simple backpropagation based on differentiating simple cost functions.
Re: Genetic Algorithms for Training Deep Neural Networks for Reinforcement Learning
#17Earlier quoted context omitted.
Because as far as I know, genetic algorithms perform poorly compared to basically every other optimization technique. I'm under the impression that they're not actually used for anything interesting these days, now that the initial hype about implementing natural selection on a computer has died down. Stochastic hill climbing as a baseline method for evaluating genetic algorithms: http://papers.nips.cc/paper/1172-sto…
Try optimizing for the shape of an antenna using hill climbing. Thing about evolutionary networks is that they can evolve topology, which usually is just a thing decided by people implementing NNs (and not guaranteed to be anywhere near most optimal). I don't see why given enough time GAs (or EAs) could not outperform simple backpropagation based on differentiating simple cost functions.
> I don't see why given enough time GAs (or EAs) could not outperform simple backpropagation based on differentiating simple cost functions.
It's already happened. You are it.
Re: Genetic Algorithms for Training Deep Neural Networks for Reinforcement Learning
#18Note that using genetic algorithms to come up with initial weights for neural networks was the state of the art in the late 90s/early 2000s. So this paper is not as novel as it seems, but it's good to have it for reference. My feeling is that since shallow networks can be made to have equivalent accuracy to deep networks, that the real challenge isn't topology but training. Hobbyists have access to so much processing…
My feeling is that since shallow networks can be made to have equivalent accuracy to deep networks, that the real challenge isn't topology but training.
In a sense, topology is training. One reason that DNNs outperform shallow networks for so many problems is that presumably their topology captures structure inherent in the world that would otherwise have to be discovered by a simpler network.
Re: Genetic Algorithms for Training Deep Neural Networks for Reinforcement Learning
#19Earlier quoted context omitted.
Try optimizing for the shape of an antenna using hill climbing. Thing about evolutionary networks is that they can evolve topology, which usually is just a thing decided by people implementing NNs (and not guaranteed to be anywhere near most optimal). I don't see why given enough time GAs (or EAs) could not outperform simple backpropagation based on differentiating simple cost functions.
> I don't see why given enough time GAs (or EAs) could not outperform simple backpropagation based on differentiating simple cost functions. It's already happened. You are it.
This was the line of reasoning that led to the initial hype about GAs, but I don't think it's sound. We still don't understand exactly how natural selection works, so there's no reason to believe the GAs we can implement now have the "special sauce" that makes real world natural selection effective. We don't know how to implement genotypes well, we don't know how to implement recombination or mutation well, etc.
Re: Genetic Algorithms for Training Deep Neural Networks for Reinforcement Learning
#20Earlier quoted context omitted.
I'm new to the field. What makes you say that shallow and deep networks have equivalent accuracy? Is there a recent publication/experiment I missed? Thanks!
https://arxiv.org/abs/1703.02660 Disclosure: I contributed to the linked work. To do simple tasks, you don’t even need neural networks, in short. If you want to do complex tasks, training the neural net isn’t the problem, but a NN can indeed represent more than say a linear policy. The issue is getting the NN to do the complex thing correctly: it’s more of an exploration problem than function approximation problem.
> To do simple tasks, you don’t even need neural networks, in short.
Ssh! We've got a gold rush to maintain.