Playing Atari with Six Neurons
arxiv.org
Playing Atari with Six Neurons
1–10 of 33 posts
Re: Playing Atari with Six Neurons
#2Cool article, lots to digest, one thing caught my eye:
"To the best of our knowledge, the only prior work using unsupervised learning as a pre-processor for neuroevolution is (cite)."
Just amazing how much low-hanging fruit there still is in the space.
Re: Playing Atari with Six Neurons
#3And the Github library:
Re: Playing Atari with Six Neurons
#4Cool article, lots to digest, one thing caught my eye: "To the best of our knowledge, the only prior work using unsupervised learning as a pre-processor for neuroevolution is (cite)." Just amazing how much low-hanging fruit there still is in the space.
[deleted]
Re: Playing Atari with Six Neurons
#5Cool article, lots to digest, one thing caught my eye: "To the best of our knowledge, the only prior work using unsupervised learning as a pre-processor for neuroevolution is (cite)." Just amazing how much low-hanging fruit there still is in the space.
Author here. The idea is low-hanging indeed, several friends (including @togelius!) commented "I always wanted to do that -- eventually". Realization is another matter. Have a look at the mess necessary to make it work: we had to discard UL initialization for online learning, accept that the encoding would grow in size, adapt the network sensibly to these changes, and tweak the ES to account for the extra weights.
Re: Playing Atari with Six Neurons
#6And the Github library: https://github.com/giuse/DNE/tree/nips2018
Ruby should be perfectly legible with a Python background, but for any question just ping me on twitter. I would be happy to build a dialog :)
Re: Playing Atari with Six Neurons
#7I can post on hacker news with only 4.
Re: Playing Atari with Six Neurons
#8As someone working on a reinforcement learning/neuroevolution problem right now, I find this to be extremely exciting. Fewer parameters, ceteris paribus, is always better—the fact that the experiments in this paper were run on one workstation, rather than on a massive farm of TPUs à la AlphaGo, implies quicker development iteration time and more accessibility to the average researcher.
The staging of components in this paper (compressor/controller), where neuroevolution is only applied to a low-dimensional controller, reminds me of Ha and Schmidhuber's recent paper on world models (which is briefly cited) [1]. They employ a variational autoencoder with ~4.4M parameters, an RNN with ~1.7M parameters, and a final controller with just 1,088 parameters! Though it's recently been shown that neuroevolution can scale to millions of parameters [2], the technique of applying evolution to as few parameters as possible and supplementing with either autoencoders or vector quantization seems to be gaining traction. I hope to apply some of the ideas in this paper to multiple co-evolving agents...
Re: Playing Atari with Six Neurons
#9I can post on hacker news with only 4.
And...ya know...I at least really try to post things related to the stories....and yet...
Also I really enjoy reading non hyped up stories about machine learning. This is really cool. Artificial neurons intrigue me.
Re: Playing Atari with Six Neurons
#10... and that's three more than the average Atari marketing exec had back then. No wonder they had trouble understanding the game industry :-)