Fast implementation of DeepMind's AlphaZero algorithm in Julia
1–10 of 73 posts
Re: Fast implementation of DeepMind's AlphaZero algorithm in Julia
#2Disclaimer: I'm not the author. Just want to share this awesome project.
Re: Fast implementation of DeepMind's AlphaZero algorithm in Julia
#3I don't know anything about Julia...how hard would this be to port to python or a c-style language?
Edit: I was mainly asking because I was curious about the relative expressiveness Julia...
Re: Fast implementation of DeepMind's AlphaZero algorithm in Julia
#4I don't know anything about Julia...how hard would this be to port to python or a c-style language? Edit: I was mainly asking because I was curious about the relative expressiveness Julia...
There are already implementations out there in Python. [1]
The point of that project is to be a very fast alternative to those implementations while being more accessible than a C++ implementation.
Re: Fast implementation of DeepMind's AlphaZero algorithm in Julia
#5I've been working on a Python implementation that uses Gradient Boosted Decision Trees (LightGBM/Treelite) instead of using a neural network for the value/policy models:
https://github.com/cgreer/alpha-zero-boosted
It's mostly to understand how AlphaZero&Friends work. I'm also curious about how well a GBDT could do, and if there are self-play techniques that can accelerate training.
The nice thing about a GBDT is that, unlike when using a NN, you can do thousands of value/policy lookups per second on a single core. So it should be cheaper to scale self-play and run a lot of self-play experiments (assuming the self-play learnings when using the GBDT model transfer to when you use the more-powerful NN in these environments).
If you're curious about accelerating self-play training, check out David Wu's work (https://arxiv.org/pdf/1902.10565.pdf). He's the creator of KataGo. I implemented his "Playout Cap Randomization" technique in my implementation above and, sure enough, it's much more efficient: https://imgur.com/a/epaKtDY. It seems like it's still early days in terms of how efficient self-play training is.
Re: Fast implementation of DeepMind's AlphaZero algorithm in Julia
#6I don't know anything about Julia...how hard would this be to port to python or a c-style language? Edit: I was mainly asking because I was curious about the relative expressiveness Julia...
I was going through this project over the weekend. And while I can't recall where exactly in the docs I read this, I am quite sure the author mentioned that there are various python projects but they are quite slow. Other implementations such as leela chess zero have a lot of C++ and are difficult to follow.
In fact, one of the things we want to do is maximize the performance of the Julia implementation. We hope to co-develop the compiler and ML stack to address these issues as they come up.
Re: Fast implementation of DeepMind's AlphaZero algorithm in Julia
#7I've been working on a Python implementation that uses Gradient Boosted Decision Trees (LightGBM/Treelite) instead of using a neural network for the value/policy models: https://github.com/cgreer/alpha-zero-boosted It's mostly to understand how AlphaZero&Friends work. I'm also curious about how well a GBDT could do, and if there are self-play techniques that can accelerate training. The nice thing about a GBDT is tha…
how good is your AI so far?
Re: Fast implementation of DeepMind's AlphaZero algorithm in Julia
#8Author here: I am happy to answer any question you may have about AlphaZero.jl. :-)
Re: Fast implementation of DeepMind's AlphaZero algorithm in Julia
#9I don't know anything about Julia...how hard would this be to port to python or a c-style language? Edit: I was mainly asking because I was curious about the relative expressiveness Julia...
Why would you? Julia shines in this use case.
Re: Fast implementation of DeepMind's AlphaZero algorithm in Julia
#10First of all this is very cool. Dunno if author is on here, but I’m curious why both Flux and Knet are used rather than just one of them (Flux seems the most Julianic?).
Also, is this really faster than PyTorch/TF? Last time I benchmarked Flux for non-trivial networks, the speed was quite good with small models but memory usage was ~5x higher than pytorch, and I couldn’t fit my models on the GPU for flux. For large models, I had to compromise on batch size in Julia, although maybe with Zygote.jl the memory issues have been resolved?