Neuroevolution of Self-Interpretable Agents
attentionagent.github.io
Neuroevolution of Self-Interpretable Agents
1–5 of 5 posts
Re: Neuroevolution of Self-Interpretable Agents
#2A twitter thread with a short summary of this paper from the author: https://twitter.com/hardmaru/status/1240444144704557056
Re: Neuroevolution of Self-Interpretable Agents
#3Is neuroevolution considered competitive with reinforcement learning? (In terms of... sample efficiency I guess)
Re: Neuroevolution of Self-Interpretable Agents
#4Is neuroevolution considered competitive with reinforcement learning? (In terms of... sample efficiency I guess)
In terms of sample efficiency, it's not - we're looking at 10x difference. But in terms of final performance, after training, it is (and even gets higher scores) for many tasks.
But here, we are not looking at the problem of sample efficiency, but rather at performance at generalization, and we find that evolution allows us to design and train systems that don't need to be differentiable (i.e. have many discrete operations) that may help with generalization.
Re: Neuroevolution of Self-Interpretable Agents
#5Is neuroevolution considered competitive with reinforcement learning? (In terms of... sample efficiency I guess)
This is my personal opinion, it is summarized from experience.
It is probably arguable though.
ES needs to learn from a population, the size of which is non-trivial for hard problems. To better evaluate population fitness, each individual performs multiple roll-outs and report the average scores as its fitness. Therefore, when an ES algorithm and an RL algorithm are both guaranteed to learn, ES is probably not as sample efficient as (some) RL algorithms.
On the other hand, ES tends to be stable and robust, one needs less trials to determine if the algorithm/hyper-parameters/network work for an agent when compared with RL algorithms. So if we count the samples needed for one to be confident that her/his method works reliably, ES is better.