Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Evolution Is the New Deep Learning

sentient.ai

Post reply on HN
Print view View original

71–80 of 242 posts

Re: Evolution Is the New Deep Learning

#71

Post by andreyk »

"Like Deep Learning (DL), EC was introduced decades ago, and it is currently experiencing a similar boost from the available big compute and big data. However, it addresses a distinctly different need: Whereas DL focuses on modeling what we already know, EC focuses on creating new knowledge."

What utter nonsense. Genetic Algorithms do exactly the same thing that Deep Learning methods do: optimize a function for a particular criterion. Genetic Algorithms are useful when taking gradients is not viable, as with RL methods - and RL methods can use Deep Learning! Seriously misleading.

Also, this: "Remarkably, although several human-designed LSTM variations have been proposed, they have not improved performance much—LSTM structure was essentially unchanged for 25 years. Our neuroevolution experiments showed that it can, as a matter of fact, be improved significantly by adding more complexity, i.e. memory cells and more nonlinear, parallel pathways." - this is not at all a new idea? How can you even pretend what you're doing is novel.

Re: Evolution Is the New Deep Learning

#72

Post by evolutioner »

mav3r1ck wrote: post #15

Having studied this extensively back when they were called Genetic Algorithms, I would like to offer a few insights. 1) One of the biggest reasons they fell out of favor for more "mathematical" approaches was that no one could really explain why exactly they worked. It makes sense on the surface that "survival of the fittest" and doing something akin to multiple stochastic gradient descents would work, but no one has…

Sentient employee here. I'll give an example on a problem for which we use evolutionary algorithms: website optimization.

Say you want to try many various changes like the title of your page, the color of the background, the position of your buy button etc. We solve this problem by trying out random variations of these websites - like A/B testing with more candidates - and by crossing the best performing ones to create a new generation of websites. This helps us find good performing variations in a very big search space.

This would be hard to do with deep learning as we start with no data at all, measuring the performance is quite noisy and you can't compute a gradient to know how to evolve your website. There is no smoothness between a title and another.

Also you could try to make a linear model for to see what effect each change has, but that doesn't take into account all the dependencies that can be complex, for example what title goes well with what background color. Evolutionary computation helps implicitly optimize without having to formulate a model.

Re: Evolution Is the New Deep Learning

#73

Post by pas »

resu_nimda wrote: post #64
mav3r1ck wrote: post #15

Having studied this extensively back when they were called Genetic Algorithms, I would like to offer a few insights. 1) One of the biggest reasons they fell out of favor for more "mathematical" approaches was that no one could really explain why exactly they worked. It makes sense on the surface that "survival of the fittest" and doing something akin to multiple stochastic gradient descents would work, but no one has…

1) One of the biggest reasons they fell out of favor for more "mathematical" approaches was that no one could really explain why exactly they worked. Kind of like how nobody can really explain how the brain works, or life in general. My gut feeling is that it is hubris to think that we are going to "figure out" intelligence with increasingly sophisticated mathematical models anytime soon. We are not giving proper cre…

> We are not giving proper credit to how complex it is, and the multi-billion year developmental process that it took.

Or we are simply not ready to accept that it's simply a big book of heuristics fine-tuned over biological eons.

It's just big. We have too many interwoven, interdependent, synergistic faculties. Input, output, and a lot of mental stuff for making the right connections between the ins and the outs. Theory of mind, basic reasoning, the whole limbic system (emotions, basic behavior, dopaminergic motivaton), the executive functions in the prefrontal cortex, all are very specialized things, and we have a laundry list of those, all fine-tuned for each other.

And there's no big magic. Nothing to "understand", no closed formula for consciousness. It's simply a faculty that makes the "all's good, you're conscious" light go green, and it's easy to do that after all the other stuff are working well that does the heavy lifting to make sense of reality.

Re: Evolution Is the New Deep Learning

#74

Post by [deleted] »

andreyk wrote: post #71

"Like Deep Learning (DL), EC was introduced decades ago, and it is currently experiencing a similar boost from the available big compute and big data. However, it addresses a distinctly different need: Whereas DL focuses on modeling what we already know, EC focuses on creating new knowledge." What utter nonsense. Genetic Algorithms do exactly the same thing that Deep Learning methods do: optimize a function for a par…

[deleted]

Re: Evolution Is the New Deep Learning

#76

Post by antirez »

andreyk wrote: post #71

"Like Deep Learning (DL), EC was introduced decades ago, and it is currently experiencing a similar boost from the available big compute and big data. However, it addresses a distinctly different need: Whereas DL focuses on modeling what we already know, EC focuses on creating new knowledge." What utter nonsense. Genetic Algorithms do exactly the same thing that Deep Learning methods do: optimize a function for a par…

Genetic algorithms yes but genetic programming is actually able to invent new things. Possibly optimizing the set of weight of a neural network to solve a given problem is more similar to genetic programming, if you see the NN as a computational unit.

Re: Evolution Is the New Deep Learning

#77

Post by andreyk »

antirez wrote: post #76
andreyk wrote: post #71

"Like Deep Learning (DL), EC was introduced decades ago, and it is currently experiencing a similar boost from the available big compute and big data. However, it addresses a distinctly different need: Whereas DL focuses on modeling what we already know, EC focuses on creating new knowledge." What utter nonsense. Genetic Algorithms do exactly the same thing that Deep Learning methods do: optimize a function for a par…

Genetic algorithms yes but genetic programming is actually able to invent new things. Possibly optimizing the set of weight of a neural network to solve a given problem is more similar to genetic programming, if you see the NN as a computational unit.

Genetic algorithms 'invent new things' by doing exactly the same thing as other optimization methods - tweaking the values of particular parameters. The only difference in is how they do so.

Re: Evolution Is the New Deep Learning

#78

Post by api »

andreyk wrote: post #77
antirez wrote: post #76

Earlier quoted context omitted.

Genetic algorithms yes but genetic programming is actually able to invent new things. Possibly optimizing the set of weight of a neural network to solve a given problem is more similar to genetic programming, if you see the NN as a computational unit.

Genetic algorithms 'invent new things' by doing exactly the same thing as other optimization methods - tweaking the values of particular parameters. The only difference in is how they do so.

Isn't this kind of a banal statement? Doesn't everything that learns in any way whatsoever "tweak values of particular parameters"?
Post reply on HN