31–39 of 39 posts
Earlier quoted context omitted.
Like all good human players do
It memorizes openings like an expert level player, then plays the rest of the game like a six year old who just learned the rules.
For the record, you can do the same things with a Hidden Markov Model (or hand-crafted rules) and the results won't be very different. Except that they won't elicit breatheless articles about being a "step towards general intelligence".
Reminds me of an old project of mine: n-gram chess. Similarly ok in openings, awful in endgames, and generally bad at chess. https://github.com/ElliotPenson/n-gram-chess
Earlier quoted context omitted.
It’s a pity that the common annotation for a “surprising” move is to follow the move code with an examination mark (or two) rather than preceding it; otherwise we would have a simple way of making the model generate surprising moves on command. :)
You could always train another model off reversed move notation and wrap it with something that reverses the output again. ;)
An amusing point from the comments > It’s not even trying to be competitive, it’s just guessing how the game will continue. If you blunder, it might guess that this must be a game between two blundering fools, and play accordingly. In a certain sense, GPT-2 is optimized to "look good to people interested in AI." Above all else it tries to generate plausibly-human-looking things, while completely oblivious of any othe…
Plenty of human behavior is generating plausibly-human-looking things completely oblivious of any other goal.
>> How impressed should we be that the same AI can write poems, compose music, and play chess, without having been designed for any of those tasks? I still don’t know. For the record, you can do the same things with a Hidden Markov Model (or hand-crafted rules) and the results won't be very different. Except that they won't elicit breatheless articles about being a "step towards general intelligence".
Not to mention that the text generated by GPT-2 can often fool an online reader whereas HMMs have the problem of being long-term incoherent and don’t reference back to subjects of the sentence like GPT-2 often does.
I’m not staying you should believe the AI hype in news media. But the paper does contain a lot of thorough analysis and comparison to the previous state of the art.
https://cdn.openai.com/better-language-models/language_model...
>> How impressed should we be that the same AI can write poems, compose music, and play chess, without having been designed for any of those tasks? I still don’t know. For the record, you can do the same things with a Hidden Markov Model (or hand-crafted rules) and the results won't be very different. Except that they won't elicit breatheless articles about being a "step towards general intelligence".
The text generated by GPT-2 is far superior to HMMs. GPT-2 was able to perform unsupervised machine translation and answered more than 5x as many questions correct on the SQUAD Q&A dataset than the previous best pure neural model. Not to mention that the text generated by GPT-2 can often fool an online reader whereas HMMs have the problem of being long-term incoherent and don’t reference back to subjects of the sente…
I guess people think "it's a powerful model so it should do well in any task" but that's typically not the case for neural nets. I know what OpenAI claims about how it can do a little bit of everything, machine translation benchmarks are borked and I bet so are question answering ones (which I confess I don't know much about).