this is hilarious and also a great idea. i dont see any reason why you can't play a few million games against itself and other engines and see where it takes you. less efficient than alpha zero probably, but how much so?
Training GPT-2 to Play Chess
11–20 of 39 posts
Re: Training GPT-2 to Play Chess
#12GPT-2 is byte pair encoding and transformer. Is there any indication that BPE plays any role here, because the vocabulary is fixed? If not, then it is only the transformer that is interesting and this post is just trying to use the name of the model, because it sounds cool. And actually giving moves directly to transformer may improve the results.
Re: Training GPT-2 to Play Chess
#13this is hilarious and also a great idea. i dont see any reason why you can't play a few million games against itself and other engines and see where it takes you. less efficient than alpha zero probably, but how much so?
It should be similarly efficient. AlphaZero used 1,000 TPUv1's to generate self-play games, and a much smaller number of TPUs to train the model on the previous self-play results. Whenever it generated a model that was >= 55% better, that became the new model.
The same algorithm could be applied here.
Re: Training GPT-2 to Play Chess
#14Seems like it just memorized openings.
Like all good human players do
Re: Training GPT-2 to Play Chess
#15this is hilarious and also a great idea. i dont see any reason why you can't play a few million games against itself and other engines and see where it takes you. less efficient than alpha zero probably, but how much so?
It should be similarly efficient. AlphaZero used 1,000 TPUv1's to generate self-play games, and a much smaller number of TPUs to train the model on the previous self-play results. Whenever it generated a model that was >= 55% better, that became the new model. The same algorithm could be applied here.
It would not be close to similarly efficient. They have completely different loss functions.
Re: Training GPT-2 to Play Chess
#16GPT-2 is byte pair encoding and transformer. Is there any indication that BPE plays any role here, because the vocabulary is fixed? If not, then it is only the transformer that is interesting and this post is just trying to use the name of the model, because it sounds cool. And actually giving moves directly to transformer may improve the results.
It's unknown what role if any BPE plays. I was surprised to discover that the final probability of a move is equal to the probability of each token from the root prompt, i.e. even though "1.Nf3 e5" is encoded as ['1', '.', 'N', 'f', '3', ' e', '5'] the probability of e5 seems unaffected by the fact that Nf3 is 3 tokens as opposed to one.
You're right that coming up with a token mapping could help things. It's a bit tricky to do that right now. Your options for fitting a custom vocab seems to be "use sentencepiece to fit a vocab, then modify the gpt-2 codebase to use the sentencepiece library for decoding".
I am honestly not sure if the output of sentencepiece is compatible with traditional encoders. What I mean is, it doesn't seem to generate an encoder.json + vocab.bpe file. It seemed to be some other kind of format. So I'm not sure if the tooling that has evolved around OpenAI's encoder format would be applicable there. I really don't know, though.
According to this slatestarcodex comment, someone got superior results on solely algebraic notation (which looks like g1f3 instead of Nf3): https://www.reddit.com/r/slatestarcodex/comments/el87vo/a_ve...
Another extension that might help is to periodically inject the full FEN board state. This was the format we were going to try next, which injects the full FEN after every move: https://gist.github.com/shawwn/318606c112774ad070f94de9c8288...
I'm so happy to get to work with GPT-2 1.5B. It's been a lot of fun to train.
By the way, if you like this kind of thing, you'll love Elo World. https://www.youtube.com/watch?v=DpXy041BIlA
Re: Training GPT-2 to Play Chess
#17Earlier quoted context omitted.
It should be similarly efficient. AlphaZero used 1,000 TPUv1's to generate self-play games, and a much smaller number of TPUs to train the model on the previous self-play results. Whenever it generated a model that was >= 55% better, that became the new model. The same algorithm could be applied here.
It would not be close to similarly efficient. They have completely different loss functions.
You're right, "efficient" should be substituted with "possible". We're certainly not claiming that this is a smart way to do it, just that you can.
Still, I think that there's a chance it could work well. Each move could be prefixed with the final outcome of the game, which is the technique either alphazero or muzero uses.
Re: Training GPT-2 to Play Chess
#18This is the chess version of all those "type these two words into your phone and keep clicking the next word" memes it's not going to generate anything meaningful, it's meant to get close enough to realistic to be either funny or interesting I was very tickled.
Similar, but GPT-2 is better at text prediction than the Markov chains used on your phone.
Do phones use Markov chains at this point? It feels like they've gotten better recently, and I wonder if maybe they're using something fancier?
Re: Training GPT-2 to Play Chess
#191.e4 e5 2.Ke2 Nc6 3.Kf3 g6 4.Kg4 Bg7 5.Nf3 h6 6.Nxe5 Bxe5 7.d4 Bg7 8. e5 at this point the notebook started allocating more memory and after it became irresponsive.
Odd. If you click the stop button on the cell, it'll turn into a play button. If you click that, it should resume where it left off.
If you happen to reproduce this, let me know.
Re: Training GPT-2 to Play Chess
#20I made similar chess engine using FastText: https://github.com/thomasahle/fastchess
It is surprising to me that you can predict optimal/strong engine moves with 27% accuracy using a completely trivial linear model, that is by a single matrix multiplication.
I wonder how well it would compete with this GPT-2 engine.