Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A" (2023)

arxiv.org

Post reply on HN
Print view View original

41–50 of 54 posts

Re: The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A" (2023)

#41

Post by YetAnotherNick »

trumpdong wrote: post #13
ralferoo wrote: post #9

Not only is the inverse not generally true (as others have pointed out), their examples requires several mental leaps. "Who is Tom Cruise's mother? [A: Mary Lee Pfeiffer]" and the reverse "Who is Mary Lee Pfeiffer's son?" The word "mother" has no relationship to "son" in terms of the model, and so while the model might be able to infer a proximity relationship between "Tom Cruise" and "Mary Lee Pfeiffer" just because…

Pretraining could be reasonably expected to make it learn that mother/father and son/daughter are inverse relationships and Tom is usually a male name.

So by extension, the question "The first letter of Oman is _", and "O is the first letter of country _" the same for humans?

Re: The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A" (2023)

#42

Post by 1718627440 »

YetAnotherNick wrote: post #41
trumpdong wrote: post #13

Earlier quoted context omitted.

Pretraining could be reasonably expected to make it learn that mother/father and son/daughter are inverse relationships and Tom is usually a male name.

So by extension, the question "The first letter of Oman is _", and "O is the first letter of country _" the same for humans?

This is obviously not as symmetrical as the initial problem, but yes you are expected to be able to easily answer the second after you read the first in a text. That is the concept of quite a lot of early secondary education level tests and also used when learning another language.

Re: The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A" (2023)

#43

Post by 1718627440 »

zmgsabst wrote: post #7
gipp wrote: post #5

As I and several other people pointed out last time this was posted, "A is B," in natural language, does not imply "B is A." "Is" can denote any of many different shades of relationship weaker than logical identity.

Even in strict logic, “is” can denote membership, as in, “squares are rectangles” does not entail “rectangles are squares”.

Because it is "(all)squares are (some)rectangles", so the reverse "(some)rectangles are (all)squares" is also true.

The difference is how you speak about a group term, not the meaning of the word 'to be'.

Re: The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A" (2023)

#44

Post by 1718627440 »

IshKebab wrote: post #15
WithinReason wrote: post #11

Earlier quoted context omitted.

That completely misses the point. The point is that "Valentina Tereshkova was the first woman to travel to space" does imply "The first woman to travel to space was Valentina Tereshkova", which LLMs fail to recognise.

Right, but that could be because the fact that that implication exists is not actually as trivial as they are implying. Does "Flargbler was blorglargh" imply "blorglargh was Flargbler"? Maybe. You need more context to know.

> You need more context to know.

Yes, which is what the model is. That's the point here, it doesn't has or accesses this context, even though it should.

Re: The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A" (2023)

#46

Post by spacebacon »

ralferoo wrote: post #31
trumpdong wrote: post #13

Earlier quoted context omitted.

Pretraining could be reasonably expected to make it learn that mother/father and son/daughter are inverse relationships and Tom is usually a male name.

I'd argue that that's not an easy task in and of itself, but even if someone adds a special exception, there's still the issue that there are many other types of inverse relationship that we understand, but a machine that's just doing pattern matching can't be expected to understand. For instance "boss" and "employee". For instance "waiter" and "customer". For instance "manager" and "player" (in a football context) o…

[dead]

Re: The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A" (2023)

#47

Post by nh23423fefe »

bigstrat2003 wrote: post #45

Yes, because they can't reason. This is well known, and should be completely unsurprising. LLMs don't "learn" anything except that some token is statistically likely to be followed by some other token.

cool assertion from 2023

Re: The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A" (2023)

#48

Post by ralferoo »

1718627440 wrote: post #42
YetAnotherNick wrote: post #41

Earlier quoted context omitted.

So by extension, the question "The first letter of Oman is _", and "O is the first letter of country _" the same for humans?

This is obviously not as symmetrical as the initial problem, but yes you are expected to be able to easily answer the second after you read the first in a text. That is the concept of quite a lot of early secondary education level tests and also used when learning another language.

But again, you can only make that determination when you know that Oman is a country. LLM's don't know this as a fact, even if they're able to regurgitate a sentence that states this.

Re: The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A" (2023)

#49

Post by IshKebab »

1718627440 wrote: post #44
IshKebab wrote: post #15

Earlier quoted context omitted.

Right, but that could be because the fact that that implication exists is not actually as trivial as they are implying. Does "Flargbler was blorglargh" imply "blorglargh was Flargbler"? Maybe. You need more context to know.

> You need more context to know. Yes, which is what the model is. That's the point here, it doesn't has or accesses this context, even though it should.

It seems to me that the point was usually "these models are fundamentally not intelligent because they have this incredibly dumb failure modes" rather than "these models happen to have this interesting weakness".

Maybe the original authors didn't make that mistake, but that was certainly the case whenever it was brought up by HN commenters of the "stochastic parrot" persuasion.

Post reply on HN