21–30 of 39 posts
Shocking. Our AI overlords will soon stumble into power, if we only point out where they're slipping up.
[0] https://twitter.com/theshawwn/status/1213559429293060099
Earlier quoted context omitted.
Similar, but GPT-2 is better at text prediction than the Markov chains used on your phone.
Do phones use Markov chains at this point? It feels like they've gotten better recently, and I wonder if maybe they're using something fancier?
An amusing point from the comments > It’s not even trying to be competitive, it’s just guessing how the game will continue. If you blunder, it might guess that this must be a game between two blundering fools, and play accordingly. In a certain sense, GPT-2 is optimized to "look good to people interested in AI." Above all else it tries to generate plausibly-human-looking things, while completely oblivious of any othe…
It’s a pity that the common annotation for a “surprising” move is to follow the move code with an examination mark (or two) rather than preceding it; otherwise we would have a simple way of making the model generate surprising moves on command. :)
One method would be to just take the 3rd or 4th best move option. It wouldn't be a winning strategy but it would probably be pretty surprising and still moderately effective.
Also, I'd imagine that the most surprising moves would require the opponent to make a very precise series of moves to avoid a losing position. While the best moves usually slowly improve your position, the most surprising moves might polarize the position by giving the opponent a chance to improve but also making the chance of a costly blunder more likely.
In that sense you could look for moves that give the opponent the fewest number of positive expected value moves rather than finding moves that give your opponent the lowest possible EV response.
This is amusing but doesn't really prove anything special about GPT-2 or general intelligence. You can probably get similar results with an n-gram model.
An amusing point from the comments > It’s not even trying to be competitive, it’s just guessing how the game will continue. If you blunder, it might guess that this must be a game between two blundering fools, and play accordingly. In a certain sense, GPT-2 is optimized to "look good to people interested in AI." Above all else it tries to generate plausibly-human-looking things, while completely oblivious of any othe…
Reminds me of an old project of mine: n-gram chess. Similarly ok in openings, awful in endgames, and generally bad at chess. https://github.com/ElliotPenson/n-gram-chess
Big fan of the Lisp too. Gorgeous code.
This is amusing but doesn't really prove anything special about GPT-2 or general intelligence. You can probably get similar results with an n-gram model.
Though this is not particularly strong, I don't think you would get similar strength from an n-gram model. You need longer-term correlations, which is generally where transformers do well.
Someone else tried this with GPT-2 a few months ago on algebraic notation and their engine seems to get to move 40 without blundering: https://www.reddit.com/r/slatestarcodex/comments/el87vo/a_ve...
Board state + algebraic notation might be the trick to make a strong engine.