Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Genetic Algorithms for Training Deep Neural Networks for Reinforcement Learning

arxiv.org

Post reply on HN
Print view View original

31–40 of 47 posts

Re: Genetic Algorithms for Training Deep Neural Networks for Reinforcement Learning

#31

Post by cameldrv »

fao_ wrote: post #30
erokar wrote: post #25

In 2005 I took a sub-symbolic AI course where one of the homework assignments was to evolve a neural network using genetic algorithms. So that in itself is nothing new. The novelty of the paper seems to be the scale of the solution. Personally, I hope we'll see a little bit of revival of genetic algorithms/programming -- I think it's promosing in design (cars, aeroplanes, architecture, etc.), since it's good at findi…

Overall the idea is about 20 years old. The earliest reference I can think of is Karl Simms work. It's pretty impressive considering how old it is: http://www.karlsims.com/evolved-virtual-creatures.html

This was the paper that excited me about neural networks in the nineties, from 92... http://liacs.leidenuniv.nl/assets/PDF/boers-kuiper.92.pdf

Re: Genetic Algorithms for Training Deep Neural Networks for Reinforcement Learning

#32

Post by nurettin »

frisco wrote: post #8
zackmorris wrote: post #5

Note that using genetic algorithms to come up with initial weights for neural networks was the state of the art in the late 90s/early 2000s. So this paper is not as novel as it seems, but it's good to have it for reference. My feeling is that since shallow networks can be made to have equivalent accuracy to deep networks, that the real challenge isn't topology but training. Hobbyists have access to so much processing…

Deep learning wasn't novel in 2012 either - it was the removal of a hardware limitation that made it compelling again. I think the same is true for evolving DNNs, but I don't know if the available compute power is there yet. > My feeling is that since shallow networks can be made to have equivalent accuracy to deep networks, that the real challenge isn't topology but training. This is not really true though... even v…

Shallow networks would probably differ in the number of categories they can differentiate with accuracy.

Re: Genetic Algorithms for Training Deep Neural Networks for Reinforcement Learning

#33

Post by sythe2o0 »

m3kw9 wrote: post #28

Maybe backprop is actually a form of evolution, a straight forward one where you iterate towards a better form

Better isn't always straightforward, sometimes unintuitive steps need to be taken to reach a better fitness value. The advantage of evolutionary methods over backprop is that evolutionary methods can take steps "backwards" and avoid local minima.

Re: Genetic Algorithms for Training Deep Neural Networks for Reinforcement Learning

#34

Post by killjoywashere »

viewtransform wrote: post #26
zackmorris wrote: post #5

Note that using genetic algorithms to come up with initial weights for neural networks was the state of the art in the late 90s/early 2000s. So this paper is not as novel as it seems, but it's good to have it for reference. My feeling is that since shallow networks can be made to have equivalent accuracy to deep networks, that the real challenge isn't topology but training. Hobbyists have access to so much processing…

The 'curse of dimensionality' should be required reading https://en.wikipedia.org/wiki/Curse_of_dimensionality for anyone getting into genetic algorithms or deep learning. Also previously on HN http://www.visiondummy.com/2014/04/curse-dimensionality-affe...

The visiondummy article is great, but how many dimensions are too many for a DNN? Let's say I have 100 TB of image data with 50 classes. Satellite data, I can have huge images, or tile them into small images. He's arguing for smaller down to a point? What point? When the convergence gets to a minimum?

Re: Genetic Algorithms for Training Deep Neural Networks for Reinforcement Learning

#35

Post by argonaut »

frisco wrote: post #8
zackmorris wrote: post #5

Note that using genetic algorithms to come up with initial weights for neural networks was the state of the art in the late 90s/early 2000s. So this paper is not as novel as it seems, but it's good to have it for reference. My feeling is that since shallow networks can be made to have equivalent accuracy to deep networks, that the real challenge isn't topology but training. Hobbyists have access to so much processing…

Deep learning wasn't novel in 2012 either - it was the removal of a hardware limitation that made it compelling again. I think the same is true for evolving DNNs, but I don't know if the available compute power is there yet. > My feeling is that since shallow networks can be made to have equivalent accuracy to deep networks, that the real challenge isn't topology but training. This is not really true though... even v…

> shallow neural networks can be universal function approximators in a trivial sense because they can be lookup tables, but they are really not expressive enough to generalize well

You've got it flipped. If anything, shallow neural networks (of an equivalent number of parameters) are more "expressive" than deep networks, BUT that expressivity just makes them overfit. This is the bias vs. variance tradeoff. If anything, deep networks encode our prior belief that there is a hierarchy of features / a compressed representation, which limits the model that is learned, to a model conforming to those priors.

Re: Genetic Algorithms for Training Deep Neural Networks for Reinforcement Learning

#36

Post by chillee »

sythe2o0 wrote: post #33
m3kw9 wrote: post #28

Maybe backprop is actually a form of evolution, a straight forward one where you iterate towards a better form

Better isn't always straightforward, sometimes unintuitive steps need to be taken to reach a better fitness value. The advantage of evolutionary methods over backprop is that evolutionary methods can take steps "backwards" and avoid local minima.

But local minima aren't a problem for modern neural networks that are optimizing in very high dimensional space.

Re: Genetic Algorithms for Training Deep Neural Networks for Reinforcement Learning

#37

Post by irremediable »

frisco wrote: post #7

Without reading this specific paper, I think this is the future. If not genetic search, then some other metaheuristic. I think this is only really credible road to AGI in the near term, as it seems to be largely a hardware limitation that will be alleviated in the next few years. Through the history of deep learning, the frontier has been networks that can be trained to some feedback of whether they're working or not…

> Fundamentally, we know neural networks can instantiate general intelligence

This isn't at all the case! We know neuronal networks, of the human brain variety specifically, can host this vague thing we're calling general intelligence.

It's not at all clear that artificial neural networks of the deep learning variety can do everything a neuronal network can do.

Re: Genetic Algorithms for Training Deep Neural Networks for Reinforcement Learning

#38

Post by ramgorur »

tree_of_item wrote: post #13
Yajirobe wrote: post #6

Earlier quoted context omitted.

Neuroevolution is nothing new. I don't know why people think this is something big/innovative.

Because as far as I know, genetic algorithms perform poorly compared to basically every other optimization technique. I'm under the impression that they're not actually used for anything interesting these days, now that the initial hype about implementing natural selection on a computer has died down. Stochastic hill climbing as a baseline method for evaluating genetic algorithms: http://papers.nips.cc/paper/1172-sto…

What do you do if you don't have any computable "gradient"? Many real world optimization problems don't come with any gradient information.

Re: Genetic Algorithms for Training Deep Neural Networks for Reinforcement Learning

#39

Post by ramgorur »

tree_of_item wrote: post #22
Yajirobe wrote: post #16

Earlier quoted context omitted.

Try optimizing for the shape of an antenna using hill climbing. Thing about evolutionary networks is that they can evolve topology, which usually is just a thing decided by people implementing NNs (and not guaranteed to be anywhere near most optimal). I don't see why given enough time GAs (or EAs) could not outperform simple backpropagation based on differentiating simple cost functions.

Why wouldn't you be able to do that with hill climbing?

Because you can't establish any gradient information from the design variables (antenna topology) to the objective function (i.e. antenna gain). So most of the time you might end up climbing towards some wrong direction (i.e. local optima).

Re: Genetic Algorithms for Training Deep Neural Networks for Reinforcement Learning

#40

Post by tree_of_item »

ramgorur wrote: post #38
tree_of_item wrote: post #13

Earlier quoted context omitted.

Because as far as I know, genetic algorithms perform poorly compared to basically every other optimization technique. I'm under the impression that they're not actually used for anything interesting these days, now that the initial hype about implementing natural selection on a computer has died down. Stochastic hill climbing as a baseline method for evaluating genetic algorithms: http://papers.nips.cc/paper/1172-sto…

What do you do if you don't have any computable "gradient"? Many real world optimization problems don't come with any gradient information.

Yes, I'm aware many problems don't have a gradient. Before this paper I'd have said you don't do anything, you just throw compute at it but still don't really solve it well.
Post reply on HN