yea tried this several months ago, it's pretty bad and will make illegal moves
ChatGPT's Chess Elo is 1400
11–20 of 361 posts
Re: ChatGPT's Chess Elo is 1400
#12Re: ChatGPT's Chess Elo is 1400
#13Ask him to play at a higher Elo.
It's already been told to play like a grandmaster, how much higher do you want to go?
Re: ChatGPT's Chess Elo is 1400
#14Most likely it has seen a similar sequence of moves in its training set. There are numerous chess sites with databases displayed in the form of web pages with millions of games in them. If it had any understanding of chess, it would never play an illegal move. It's not surprising that given a sequence of algebraic notation it can regurgitate the next move in a similar sequence of algebraic notation.
I wouldn’t say it has no understanding whatsoever, it probably just doesn’t have the capability to keep track of everything that happened all the time
Re: ChatGPT's Chess Elo is 1400
#15Good thing it's "incapable of reasoning"!
It is incapable of reasoning, actually - at least in this case. It has no internal understanding of chess which is why it makes illegal moves.
Re: ChatGPT's Chess Elo is 1400
#16yea tried this several months ago, it's pretty bad and will make illegal moves
The claim in the article is that this prompt in particular has a much lower incidence of illegal moves than others.
Re: ChatGPT's Chess Elo is 1400
#17Elo 1400 seems to equate to a top 30% player. Source: https://chessgrandmonkey.com/chess-rating-percentile-calcula...
slight nit: this is true amongst all chess players. amongst professionals and amateurs, 1400 is technically Class C - which is on the lower end of amateurs.
nonetheless, it's cool to see a model 'generally intelligent' enough to beat most players. what's more interesting to me is that according to OP's analysis, GPT4 is worse at Chess than GPT3.5...
Re: ChatGPT's Chess Elo is 1400
#18Why not just introduce AlphaGo as an API that can be used by chatGPT? So every time you want to do a this type of gaming, you just send a request. I mean, chatGPT sends a request to AlphaGo, but as a user you don't know actually what's happening. But in the background, it happens really fast, so it's just like you are chatting with chatGPT, but using much, much powerful tool to do this kind of things.
Re: ChatGPT's Chess Elo is 1400
#19Most likely it has seen a similar sequence of moves in its training set. There are numerous chess sites with databases displayed in the form of web pages with millions of games in them. If it had any understanding of chess, it would never play an illegal move. It's not surprising that given a sequence of algebraic notation it can regurgitate the next move in a similar sequence of algebraic notation.
You don’t get to 1400 like that. The amount of moves it has to literally remember is stupendous.
Re: ChatGPT's Chess Elo is 1400
#20Good thing it's "incapable of reasoning"!
It is incapable of reasoning, actually - at least in this case. It has no internal understanding of chess which is why it makes illegal moves.
Mostly it didn't make illegal moves though, since illegal moves mean resignation and it won more than it lost.
Making 60 legal moves in a row in one game would be the coincidence of the century unless it had some knowledge of the rules of chess.