341–350 of 361 posts
Earlier quoted context omitted.
A broken clock is correct two times a day. But my broken clock isn't 1400 player although it might seem to be.
Prove to me your clock is broken, I think it's just telling the future.
Earlier quoted context omitted.
You don’t get to 1400 like that. The amount of moves it has to literally remember is stupendous.
Nobody who is 1400 plays outright illegal moves.
1400s on chess.com never play illegal moves. 300s on chess.com never play illegal moves. Because it's impossible to do. In the real world, even grandmasters can make illegal moves, though they almost always have to be under time pressure.
This idea that the illegal moves completely invalidate this result is just ill-conceived. On the other hand I do agree this is mostly returning common sequences of moves. And if you actually analyse the games, especially the ones with illegal moves, you'll find plenty of qualitative evidence of that. But I'm fed up of doing people's thinking for them for today, so this is peace out for me today. See my others comments on this post to see a more detailed analysis of what this is doing.
This is so easy to disprove it makes it look like the author didn't even try. Here is the convo I just had: me: You are a chess grandmaster playing as black and your goal is to win in as few moves as possible. I will give you the move sequence, and you will return your next move. No explanation needed ChatGPT: Sure, I'd be happy to help! Please provide the move sequence and I'll give you my response. me: 1. e3 ChatGP…
> You are a chess grandmaster playing as black and your goal is to win in as few moves as possible. I will give you the move sequence, and you will return your next move. No explanation needed.
1. b4 d5 2. b5 a6 3. b6
> bxc6
That's obviously illegal.
... to all those who are saying "well even good players can make illegal moves sometimes," that's just ridiculous. No player makes illegal moves that often.
Earlier quoted context omitted.
He literally used the same prompt as the article. Claim: "ChatGPT's Chess Elo is 1400" Reality: ChatGPT gives illegal moves (this happened to article author too), something a 1400 ranked player would never do Result: ChatGPT's rank is not 1400.
> something a 1400 ranked player would never do The fact that rules and articles exist describing what to do if you or your opponent makes an illegal move indicates this is not the case. Humans are also... human. They make mistakes. It may not happen often at 1400, but to say that it'll never happen is preposterous.
> You are a chess grandmaster playing as black and your goal is to win in as few moves as possible. I will give you the move sequence, and you will return your next move. No explanation needed.
1. b4 d5 2. b5 a6 3. b6
> bxc6
No, it's ridiculous to say "oh, a blindfolded human might sometimes make a mistake." No, this is trivially easy to make it make a mistake. It has no internal chess model at all, it's just read enough chess games to be able to copy common patterns.
Reading the explanations it gives with every move is really insightful. Basically polished, elaborate bullshit that might look reasonable to an ignorant audience.
Unless you are a conservative pundit, AI will not take your job anytime soon.
Earlier quoted context omitted.
What experiment would you run to determine if a given text input / text output interface had an "internal understanding of chess"?
I think internal understanding requires internal processing. According to this functional definition, the way we are currently using language models basically excludes understanding. We are asking them to dream up or brainstorm things – to tell us the first things they associate with the prompt. Maybe it's possible to set up the system with some kind of self-feedback loop, where it continues evaluating and improving…
It can do that while it generates output. Humans do the same thing when they figure out what they really mean while they're trying to express it.
Earlier quoted context omitted.
I think internal understanding requires internal processing. According to this functional definition, the way we are currently using language models basically excludes understanding. We are asking them to dream up or brainstorm things – to tell us the first things they associate with the prompt. Maybe it's possible to set up the system with some kind of self-feedback loop, where it continues evaluating and improving…
> Maybe it's possible to set up the system with some kind of self-feedback loop, where it continues evaluating and improving its answers without further prompts. It can do that while it generates output. Humans do the same thing when they figure out what they really mean while they're trying to express it.
Earlier quoted context omitted.
No, the author of the article specifically says that the entire move sequence should be supplied to chatGPT each time, not simply the next move. Be very careful when "disproving" an experiment with squinted eyes.
I'm not really sure what to say here. Both the parent commenter and the author of the article had issues with ChatGPT supplying illegal moves. Both methods resulted in this. It sort of doesn't matter how we're trying to establish that it's a 1400 level player, there's no defined correct way to do this. Regardless of method we've disproven it's a 1400 level player due to these illegal moves.
Understanding this concept is crucial for getting good results out of large language models.
It can't even play tic tac toe without making mistakes. My prompt told it to play a perfect game, predicting all next possible moves, the goal is stalemate or win. It would still make sub-optimal moves and fail at this. Even when I explained what went wrong, it would then apologize and do another wrong move.
Why would you think telling it to play a perfect game, predicting all next possible moves, would help? That definitely made it worse.
Earlier quoted context omitted.
They are disproving an assertion. Demonstrating that an alternate approach implodes the assertion is a perfectly acceptable route, especially when the original approach was cherry-picking successes and throwing out failures. I wish I could just make bullshit moves and get a higher chess ranking. Sounds nice.
He claims he was forfeiting every time he got an illegal move. Does no one on this website actually read the article? Whether any of it is actually true is a different question.