I mean obviously yes. LLMs aren't intelligent and they don't understand anything.
OP is trying to argue against this but it's nonsense. ChatGPT also cannot tell me who the 8th chancellor was. It tells me it was Gerhard Schröder.
Is the reversal curse in LLMs real?
81–90 of 211 posts
Re: Is the reversal curse in LLMs real?
#82Earlier quoted context omitted.
You are right. You are thinking correctly. But "the apple is red" might not mean that this particular apple belongs ---to put it in your wording--- to the category red, but that the category of things we call apple also belongs to the category of things that are red. And generally speaking, I think that is the meaning.
> but that the category of things we call apple also belongs to the category of things that are red No, in English this would be “apples are red”, not “the apple is red”.
I don't disagree though perhaps it's worth mentioning that sentences such as "the great spotted woodpecker is a medium-sized woodpecker" are used in English with the meaning "great spotted woodpeckers are medium-sized woodpeckers" so it seems to me that "apples are red" is grammatically a possible meaning of "the apple is red" even if it would be stylistically and pragmatically so weird that nobody would ever do that except perhaps as some kind of joke.
Re: Is the reversal curse in LLMs real?
#83I mean obviously yes. LLMs aren't intelligent and they don't understand anything. OP is trying to argue against this but it's nonsense. ChatGPT also cannot tell me who the 8th chancellor was. It tells me it was Gerhard Schröder.
It also cannot tell me the 7th.
Re: Is the reversal curse in LLMs real?
#84I fall somewhere on the skeptic side of the LLM spectrum. But this "flaw" just does not seem to have the force that its proponents seem to think it does, unless I'm missing something significant. Simply because in the context of natural language (Rather than formal logical statements), "A is B" does not imply "B is A" in the first place. "Is" can encompass a wide variety of logical relationships in colloquial usage,…
You're thinking of LLMs as basically pattern matching "a is b", but that's not really how they work. In fact, the incredible thing that LLMs can do is that they can understand (some) colloquial language and shades of meaning. LLMs can grasp all sorts of strange textual nuance; they can't grasp the cases "Foo is Bar's mother" strictly implies "Bar is Foo's son"?
>they can't grasp the cases "Foo is Bar's mother" strictly implies "Bar is Foo's son"?
Correction. The Model can. Training cannot.
Re: Is the reversal curse in LLMs real?
#85I mean obviously yes. LLMs aren't intelligent and they don't understand anything. OP is trying to argue against this but it's nonsense. ChatGPT also cannot tell me who the 8th chancellor was. It tells me it was Gerhard Schröder.
IDK, I still some of it boil down to "B -> A" being extremely ambiguous. Other than the annoying Tom Cruise example (I finally know where it came from!), I've seen people frequently bring up "The color of the sky is blue" vs. "Blue is the color of the sky" - but this illustrates my hypothesis perfectly. In training data (and in the totality of what humans ever said or wrote), "Color of the sky is " is almost certain to be followed by "blue" - meanwhile, "Blue is the color of the " has a lot of possible completions, of which "sky" is by far not the most likely.
Re: Is the reversal curse in LLMs real?
#86The Olaf Scholz exemple from this article is just another exemple of how LLMs can’t count. If you try this prompt: “In this list of words: bike, apple, phone, dirt, tee, sun, glass; which is the fifth word?” it will fail as well. “Fifth” is not connected to any counting ability in LLMs the way it is for us.
If you now try this prompt: “Who’s Tom Cruise’s mother in this exemple: “Mary Lee Pfeiffer is Tom Cruise’s mother.”?” It will give you the right answer.
IMO this is a sign that it can understand reversibility.
The exemple used are not present enough in the dataset and the “confidence” of the model is not high enough for it to give the right answer. Or, as stated at the beginning they use counting mechanisms (or others) that the model just doesn’t have.
Re: Is the reversal curse in LLMs real?
#87Earlier quoted context omitted.
You're thinking of LLMs as basically pattern matching "a is b", but that's not really how they work. In fact, the incredible thing that LLMs can do is that they can understand (some) colloquial language and shades of meaning. LLMs can grasp all sorts of strange textual nuance; they can't grasp the cases "Foo is Bar's mother" strictly implies "Bar is Foo's son"?
>they can't grasp the cases "Foo is Bar's mother" strictly implies "Bar is Foo's son"? Correction. The Model can. Training cannot.
I'm starting to wonder, why should it in the first place? Very late stages, or fine-tuning, I'd perhaps understand. But early on? Everything is possible, and "Foo is Bar's mother" doesn't imply much over "these tokens come together, sometimes".
It's not that humans aren't suffering from this too. It's possible to spot it when you're learning, or even easier, when you're teaching someone a new thing. A person who learned that "A is B" does not automatically learn that "B is A"; they need to first process it, perhaps run the reversion explicitly in their head. I'd say that checking if the student can infer "B is A" having learned "A is B" is a good way to tell whether they're starting to comprehend the material, vs. just memorizing it.
Re: Is the reversal curse in LLMs real?
#88The Olaf Scholz exemple from this article is just another exemple of how LLMs can’t count. If you try this prompt: “In this list of words: bike, apple, phone, dirt, tee, sun, glass; which is the fifth word?” it will fail as well. “Fifth” is not connected to any counting ability in LLMs the way it is for us. If you now try this prompt: “Who’s Tom Cruise’s mother in this exemple: “Mary Lee Pfeiffer is Tom Cruise’s moth…
It can’t count because what the LLM sees is a bunch of tokens not words
Re: Is the reversal curse in LLMs real?
#89I mean obviously yes. LLMs aren't intelligent and they don't understand anything. OP is trying to argue against this but it's nonsense. ChatGPT also cannot tell me who the 8th chancellor was. It tells me it was Gerhard Schröder.
IDK, I still some of it boil down to "B -> A" being extremely ambiguous. Other than the annoying Tom Cruise example (I finally know where it came from!), I've seen people frequently bring up "The color of the sky is blue" vs. "Blue is the color of the sky" - but this illustrates my hypothesis perfectly. In training data (and in the totality of what humans ever said or wrote), "Color of the sky is " is almost certain…
Nah. The context of all of this is people claiming LLMs are AI. If so they would be able to reverse A is B, and also know when not to, so your example is irrelevant.
The paper shows that it cannot reverse. Then the blog posts goes around different disingenuous ways of making excuses for why it can't reverse at the same time as trying to show that it can reverse in the Tom Cruise training example (extremely disingenuous because it's just the same data over and over with synonyms replaced, and what he gets out of it is just completions, not logical deduction).
Re: Is the reversal curse in LLMs real?
#90Earlier quoted context omitted.
> Even when allowed to surf the web, ChatGPT gets it wrong: > Who was the eighth Federal Chancellor of the Federal Republic of Germany? - Gerhard Schröder Its probably counting Walter Scheel, who was Acting Chancellor in 1974, and you are probably not. When I asked ChatGPT to list the chancellors in order and identify the eighth, it listed eight ending in Schröder, with Scheel and his ten day Acting Chancellor tenure…
It's funny because you ascribe some reasoning in ChatGPT's answer “Gerhard Schröder”, but somehow missed that it's also able to give you two other answers that are unambiguously wrong…
> It's funny because you ascribe some reasoning in ChatGPT's answer “Gerhard Schröder”, but somehow missed that it's also able to give you two other answers that are unambiguously wrong
Schröder is the only answer it gave when using functionality that would bring some representation if a list into its context first, and 10it did it with different prompts and mechanisms for bringing a list into its context.
That LLMs are bad at counting-related tasks without doing that is well-known, and not a point I felt needed belaboring.