Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Evolution Is the New Deep Learning

sentient.ai

Post reply on HN
Print view View original

31–40 of 242 posts

Re: Evolution Is the New Deep Learning

#31

Post by xants »

yorwba wrote: post #29
ewjordan wrote: post #19

Earlier quoted context omitted.

Wait, I'm confused...you're saying genetic approaches fell out of favor because they're basically just stochastic gradient descent? Most of modern DL relies heavily on SGD at various points during training. My impression is that they fell out of favor precisely because don't actually use any gradients, and end up converging on good maxima slower than you could if you used the gradients from the net. Am I off base her…

If you mutate genomes by small additive modifications to a vector of continuous parameters, then taking lots of samples and keeping the best is essentially a stochastic approximation to gradient descent. However, unlike the SGD used in deep learning, it doesn't make use of calculus and therefore requires many more samples (exponentially more, in the worst case) to get a gradient of equivalent accuracy. I.e. it's slow…

Is it true that genetic algorithms have the benefit of being able to find the global optima more consistently due to the incorporation of randomness in subsequent generations? Whereas DL models often get stuck at local optima?

Re: Evolution Is the New Deep Learning

#33

Post by deepnet »

peterstoziek wrote: post #2

As expected, the article seems to be a typical content marketing piece. If you're looking for real insights into evolutionary algorithms, specifically "neuroevolution", I highly recommend to read this article: https://www.oreilly.com/ideas/neuroevolution-a-different-kin... I enjoyed it much more than - what feels like - a quickly thrown together marketing piece with no real value for the reader.

Thanks for the great link surveying Evolution and NEAT ( Neuroevolution of Neural Nets ) by Ken Stanley who pioneered the method.

The OP is by Risto Miikkulainen who was Stanley's collaborator on NEAT and should not be summarily dismissed.

Re: Evolution Is the New Deep Learning

#34

Post by TheOtherHobbes »

boffinism wrote: post #8

Interesting to see that we're a long way from this technique actually generating anything valuable. E.g. if you try the LSTM Music Maker they link to ( https://www.sentient.ai/sentient-labs/ea/lstm-music/ ) and enter a melody, the resulting 'improvisation' will neither have any of the hallmarks of your initial input nor obey any of the conventions of any genre of music I recognise. It'll just spew out a random-seemin…

Evolutionary approaches won't necessarily produce better music. This is a well-researched area.

The challenge with all GA work is finding the right fitness function. If you don't understand your domain well enough to define a good fitness function you're going to be wasting your time, and no amount of algorithmic and/or AI magic is going to help you.

Re: Evolution Is the New Deep Learning

#36

Post by TheOtherHobbes »

goatlover wrote: post #13
cvaidya1986 wrote: post #11

Earlier quoted context omitted.

What’s the definition of best

350,000 species of beetles.

In music, evolution produced Bach, so it can't really be considered a failure.

It took a while though, and it's possible alternative approaches would have been produced a similar result more efficiently.

Re: Evolution Is the New Deep Learning

#37

Post by oftenwrong »

koube wrote: post #26
mav3r1ck wrote: post #15

Having studied this extensively back when they were called Genetic Algorithms, I would like to offer a few insights. 1) One of the biggest reasons they fell out of favor for more "mathematical" approaches was that no one could really explain why exactly they worked. It makes sense on the surface that "survival of the fittest" and doing something akin to multiple stochastic gradient descents would work, but no one has…

Where's the posts on explainable AI? I did a ctrl+f but didn't see any.

https://distill.pub/2018/building-blocks/

https://news.ycombinator.com/item?id=16531246

Re: Evolution Is the New Deep Learning

#38

Post by vanderZwan »

mav3r1ck wrote: post #15

Having studied this extensively back when they were called Genetic Algorithms, I would like to offer a few insights. 1) One of the biggest reasons they fell out of favor for more "mathematical" approaches was that no one could really explain why exactly they worked. It makes sense on the surface that "survival of the fittest" and doing something akin to multiple stochastic gradient descents would work, but no one has…

Regarding 3), do you know of any work on genetic programming as a method of doing research into evolvability itself? So basically, as a form of simulation?

Tierra obviously counts, but I was thinking of more specific examples. Say, something like this paper, which showed that adding a tiny cost-function to a network spontaneously makes it more modular:

[0] http://rspb.royalsocietypublishing.org/content/280/1755/2012...

Re: Evolution Is the New Deep Learning

#40

Post by IAmEveryone »

If only wishing made it so...

Evolutionary approaches have always had one big feature in their favour: they are far more fun to work with.

They produce all these fascinating oddities, like the one that learnt to outsmart it’s opponents at infinite tic-tac-toe by playing coordinates 102312 and 47875 and watching them run out of memory trying to build a data structure for the board.

They are also far easier to combine with human expertise, i. e. “I can do this! Let’s throw some of my ideas into the gene pool”.

But empirically it’s hard to deny that neural nets have been used for some incredible things over the last years. I think it’s the rare hype that is deserved.

Post reply on HN