Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Evolution Is the New Deep Learning

sentient.ai

Post reply on HN
Print view View original

141–150 of 242 posts

Re: Evolution Is the New Deep Learning

#141

Post by nrub »

evolutioner wrote: post #124
avilay wrote: post #116

Earlier quoted context omitted.

What is the advantage of using evolutionary algos in this case over using something like Thompson Sampling or Contextual Multi-Armed Bandit? ( http://www.kdd.org/kdd2017/papers/view/an-efficient-bandit-a... )

They can actually work together. Thompson Sampling / Multi-armed bandit algorithms are traffic allocation strategies to use the least amount of traffic to find the optimal variant. So say you have version A/B/../N variants of a website, you can split the traffic equally or use a a bandit algorithm and find the best performing variant of it. But that only helps you test N variants. If you want to test for 4 titles and…

The paper linked above does directly address the case of multiple experiments occurring in the same context. They address this with hill-climbing over those 180 different variations. The use of a bayesian linear regression takes place of the exploration found with Thompson sampling.

Re: Evolution Is the New Deep Learning

#142

Post by jacques_chester »

andreyk wrote: post #77
antirez wrote: post #76

Earlier quoted context omitted.

Genetic algorithms yes but genetic programming is actually able to invent new things. Possibly optimizing the set of weight of a neural network to solve a given problem is more similar to genetic programming, if you see the NN as a computational unit.

Genetic algorithms 'invent new things' by doing exactly the same thing as other optimization methods - tweaking the values of particular parameters. The only difference in is how they do so.

Antirez distinguished between genetic algorithms and genetic programming.

As you say, GAs evolve parameters to the fitness function. But GP does, in fact, evolve programs.

They are very distinct schools of work that evolved independently. Some other major schools are Learning Classifier Systems, Evolution Strategies and Evolutionary Programming.

The high-level name these days is Evolutionary Computation. In turn typically lumped under Nature-Inspired Computing, or sometimes Metaheuristics.

Re: Evolution Is the New Deep Learning

#143

Post by eafan »

glup wrote: post #10
XnoiVeX wrote: post #6

A philosophical tangent: Evolution did not always produce the best outcomes. Would neuroevolution be vulnerable to similar effects? Probably a good research area. https://www.wired.com/2009/07/st-best-5/

Biological evolution seem pretty susceptible to overfitting to ecological circumstances—I worry that there would be an analog for these techniques with NNs

There are well known problems that evolutionary systems are much better at solving than NNs. The multiplexer problem, for instance, or sorting networks, where the search space is too rugged or deceptive for neural network solutions.

Re: Evolution Is the New Deep Learning

#144

Post by Maybestring »

diyseguy wrote: post #133
mav3r1ck wrote: post #15

Having studied this extensively back when they were called Genetic Algorithms, I would like to offer a few insights. 1) One of the biggest reasons they fell out of favor for more "mathematical" approaches was that no one could really explain why exactly they worked. It makes sense on the surface that "survival of the fittest" and doing something akin to multiple stochastic gradient descents would work, but no one has…

I don't understand why A.I. can't be explainable. Can't they just add logging every time it makes a decision and then trace through the trail of decisions to the final result?

If you are willing to accept that as an explanation. Yes, it's easily explainable. A big list of anonymous decisions is easy. Naming those decisions is hard.

Re: Evolution Is the New Deep Learning

#145

Post by Houshalter »

breuleux wrote: post #126
yorwba wrote: post #29

Earlier quoted context omitted.

If you mutate genomes by small additive modifications to a vector of continuous parameters, then taking lots of samples and keeping the best is essentially a stochastic approximation to gradient descent. However, unlike the SGD used in deep learning, it doesn't make use of calculus and therefore requires many more samples (exponentially more, in the worst case) to get a gradient of equivalent accuracy. I.e. it's slow…

On the other hand, even if evolutionary algorithms require a lot of samples, they are embarrassingly parallel: you can easily try all samples simultaneously. If you have enough resources to throw at the problem, it can be faster (although more resource-intensive) to estimate the gradient this way than to compute an accurate gradient analytically.

SGD is embarrassingly parallel as well. You can train a net on several different examples simultaneously and combine the gradients or learned weights.

The reason it's not done so much is because the bandwidth of moving huge numbers of gradients or weights between computers is pretty significant. There's been all sorts of research into compressing them or reducing the precision. However this is a problem for evolutionary algorithms as well.

Re: Evolution Is the New Deep Learning

#146

Post by Houshalter »

PeterisP wrote: post #54
xants wrote: post #31

Earlier quoted context omitted.

Is it true that genetic algorithms have the benefit of being able to find the global optima more consistently due to the incorporation of randomness in subsequent generations? Whereas DL models often get stuck at local optima?

DL models don't often get stuck at local optima. In theory, they could be vulnerable to that, but in practice they are not, it simply doesn't happen in most practical supervised learning applications. I'm not up to date on theoretical research about this topic, but as far as I recall there are some interesting demonstrations on realistic problems showing that all the different "local" optima resulting from different…

This is common wisdom I think is false. You absolutely do get stuck in local optima frequently with reinforcement learning. OpenAI has a good example somewhere of a robot trying to put a peg through a hole. Trained with regular gradient descent it just gets stuck putting the peg pretty close to the hole, but not through it.

I'm not even sure that it's not a problem in general. I know I've watched NNs frequently get stuck in local minima even on incredibly simple datasets like xor or spirals. SGD and dropout are widely used in part because they add noise to the gradients that can help break out of local optima. But that's not a perfect method

Re: Evolution Is the New Deep Learning

#147

Post by dmix »

IanCal wrote: post #96
api wrote: post #78

Earlier quoted context omitted.

Isn't this kind of a banal statement? Doesn't everything that learns in any way whatsoever "tweak values of particular parameters"?

It's banal but sensible in response to someone claiming a large difference between two fundamentally similar things. Either both or neither "invents".

As a non-expert in the field my impression from reading the article wasn't that they were saying there was a large difference, or it's a whole new approach, only that is was a progression over DL.

The entire article repeatedly relates everything in the context of what DL accomplishes and only introduces EC as an better optimized version.

Which doesn't lead one to think they are significantly different approaches but variations or tweaked versions of the same general thing...

Re: Evolution Is the New Deep Learning

#148

Post by eafan »

Eliezer wrote: post #112
andreyk wrote: post #77

Earlier quoted context omitted.

Genetic algorithms 'invent new things' by doing exactly the same thing as other optimization methods - tweaking the values of particular parameters. The only difference in is how they do so.

Gradient descent: Run the neural net, evaluate the fitness function, shift parameters in the direction of the derivative of the fitness function. Genetic algorithm: Shift parameters in a batch of random ways, evaluate using fitness function, keep the best. There is nothing more fundamentally 'creative' about one than the other; these are both just attacks on the problem "here is this function, find parameter settings…

An important difference is the manner by which 'exploration' is codified in evolutionary algorithms, which allows for these algorithms to have a tune-able mix of hill-climbing and niche-finding. RL-based systems, Evolutionary Strategies, Swarm Intelligence, ... all are essentially hill-climbing and are typically much better at the 'exploit' aspect than the 'explore' aspect of the search. When you pair evolutionary algorithms with novelty search, for instance, you get very good coverage, as these guys have done with their sorting network work.

Re: Evolution Is the New Deep Learning

#149

Post by Houshalter »

guest12268 wrote: post #132
vanderZwan wrote: post #38

Earlier quoted context omitted.

Regarding 3), do you know of any work on genetic programming as a method of doing research into evolvability itself? So basically, as a form of simulation? Tierra obviously counts, but I was thinking of more specific examples. Say, something like this paper, which showed that adding a tiny cost-function to a network spontaneously makes it more modular: [0] http://rspb.royalsocietypublishing.org/content/280/1755/2012.…

Yes there has been some very interesting recent work. In particular, how evolvability emerges and is harnessed in evolutionary computation. A few papers come to mind: 1. Evolvability is Inevitable: http://journals.plos.org/plosone/article?id=10.1371/journal.... 2. Extinction Events can Accelerate Evolution (2015): http://journals.plos.org/plosone/article?id=10.1371/journal.... 3. Evolvability Search: Directly selecti…

Evolvability can also evolve away too. For instance, a gene that decreases the mutation rate to 0. Most mutations are harmful, so any organism with the gene will be more likely to have successful children. And eventually the gene will become dominant and there will be no more mutations. And evolution will stop.

Re: Evolution Is the New Deep Learning

#150

Post by eafan »

Cacti wrote: post #70
marcosdumay wrote: post #68

Earlier quoted context omitted.

> no one could really explain why exactly they worked That is the first time I have heard that claim, and since we have a large body of knowledge describing how evolution works (that sampo description is one the clearest I've seen) and how it can be optimized, I imagine you are talking about some other problem. Is it about predicting the causes of some learned trait? Is there some interesting research on that?

They are referring to Genetic Algorithms. There was a theory called the "building block hypothesis" but no one could prove it (turned out it was impossible to prove). The field was sort of run on hand waving for several decades.

Clearly lack of a sound theoretical basis or proof for why deep learning works has not stopped its proliferation. For a practitioner, the proof is in the pudding: generalized results, novel solutions that provably work, new designs that fulfill the given objective(s). At the end of the day, those are what really matter for practical applications.
Post reply on HN