Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A" (2023)

arxiv.org

Post reply on HN
Print view View original

31–40 of 54 posts

Re: The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A" (2023)

#31

Post by ralferoo »

trumpdong wrote: post #13
ralferoo wrote: post #9

Not only is the inverse not generally true (as others have pointed out), their examples requires several mental leaps. "Who is Tom Cruise's mother? [A: Mary Lee Pfeiffer]" and the reverse "Who is Mary Lee Pfeiffer's son?" The word "mother" has no relationship to "son" in terms of the model, and so while the model might be able to infer a proximity relationship between "Tom Cruise" and "Mary Lee Pfeiffer" just because…

Pretraining could be reasonably expected to make it learn that mother/father and son/daughter are inverse relationships and Tom is usually a male name.

I'd argue that that's not an easy task in and of itself, but even if someone adds a special exception, there's still the issue that there are many other types of inverse relationship that we understand, but a machine that's just doing pattern matching can't be expected to understand. For instance "boss" and "employee". For instance "waiter" and "customer". For instance "manager" and "player" (in a football context) or "manager" and "artist" (in a music context) or "manager" and "customer" (in a bank context). And what's the inverse of "customer" now? And so on and so on...

All of this context works because we build up an extensive model of the world through the course of our lifetimes. LLM models don't do that, they pattern match based on stats.

Somebody would have to decide each of these things is important and create training data sets for each of them. But we implicitly understand so much context about the world that it's practically impossible to document everything we know in the form that a model can actually learn from.

Re: The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A" (2023)

#32

Post by [deleted] »

trumpdong wrote: post #13
ralferoo wrote: post #9

Not only is the inverse not generally true (as others have pointed out), their examples requires several mental leaps. "Who is Tom Cruise's mother? [A: Mary Lee Pfeiffer]" and the reverse "Who is Mary Lee Pfeiffer's son?" The word "mother" has no relationship to "son" in terms of the model, and so while the model might be able to infer a proximity relationship between "Tom Cruise" and "Mary Lee Pfeiffer" just because…

Pretraining could be reasonably expected to make it learn that mother/father and son/daughter are inverse relationships and Tom is usually a male name.

[deleted]

Re: The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A" (2023)

#33

Post by johnsonjo »

latexr wrote: post #18
beardyw wrote: post #8

Earlier quoted context omitted.

Yes, "Who is Mary Lee Pfeiffer's son?" happens to have just one answer, whereas "Who is Mary Lee Pfeiffer's child?" would have several.

> "Who is Mary Lee Pfeiffer's child?" would have several. Might have several. It only has one answer if there is only one child, which appears to be the case here. They are measuring against what they told the model, not necessarily facts mapping to the real world. In this case, the correct would always include “Tom Cruise” even if it needed a clarifying “there might be others I have no knowledge of”.

> Might have several. It only has one answer if there is only one child, which appears to be the case here.

With context. It does have several which was probably GPs point, and she did not have only one child.

Quick google search turns out that Mary Lee Pfeiffer had 4 children:

Lee Ann DeVette (born 1959) (daughter) Marian Henry (born 1960) (daughter) Tom Cruise (born 1962) (son) Cass Mapother (born 1964) (daughter)

So saying "Who is Mary Lee Pfeiffer's child?" would have 4 possible answers (which is several) with all known context. Whereas like GP was saying "Who is Mary Lee Pfeiffer's son?" would have 1 identifying answer, Tom Cruise with the same context.

> In this case, the correct would always include “Tom Cruise” even if it needed a clarifying “there might be others I have no knowledge of”.

I agree with this by the way.

Re: The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A" (2023)

#34

Post by [deleted] »

trumpdong wrote: post #13
ralferoo wrote: post #9

Not only is the inverse not generally true (as others have pointed out), their examples requires several mental leaps. "Who is Tom Cruise's mother? [A: Mary Lee Pfeiffer]" and the reverse "Who is Mary Lee Pfeiffer's son?" The word "mother" has no relationship to "son" in terms of the model, and so while the model might be able to infer a proximity relationship between "Tom Cruise" and "Mary Lee Pfeiffer" just because…

Pretraining could be reasonably expected to make it learn that mother/father and son/daughter are inverse relationships and Tom is usually a male name.

[deleted]

Re: The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A" (2023)

#35

Post by niam »

altmanaltman wrote: post #23
niam wrote: post #17

Earlier quoted context omitted.

Your question arguably doesn't have multiple answers. There's one answer: the set of children (discounting different orders of the set). The question "Who is her child" has multiple answers because it asks you to deliver a single answer.

But wouldn't every one of those multiple answers be the correct one in this case? Like it can say child a or child b or child c (hypothetical) and while there are mutiple answers, each of them is a logically right one for the question "Who is her child?" no? So how do we judge what is the absolute right answer to that? its ambigious when you say child

Zooming out to the original complaint that "A is B" doesn't imply "B is A" in common English, and then further -- to the goal of having an LLM predict tokens that map closely to truth/logic/helpfulness:

I don't think a person speaking plain English in most contexts should be seen as "correct" to answer the question with a non-list answer, even if the question is shaped to expect one, unless there's an established confidence that the shape of the question wasn't made in error.

If someone asked me in real life who "my child" is on stage, and I had multiple children on stage, I would first say that I had multiple children there, rather than choosing one from the set. It would be most helpful for an LLM in my position to do the same, rather than infer that [because Timmy is niam's child, niam's child ought to be Timmy when queried].

Re: The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A" (2023)

#36

Post by soco »

trumpdong wrote: post #13
ralferoo wrote: post #9

Not only is the inverse not generally true (as others have pointed out), their examples requires several mental leaps. "Who is Tom Cruise's mother? [A: Mary Lee Pfeiffer]" and the reverse "Who is Mary Lee Pfeiffer's son?" The word "mother" has no relationship to "son" in terms of the model, and so while the model might be able to infer a proximity relationship between "Tom Cruise" and "Mary Lee Pfeiffer" just because…

Pretraining could be reasonably expected to make it learn that mother/father and son/daughter are inverse relationships and Tom is usually a male name.

We have both Sean Young and Sean Bean. Black swans still exists and the pretraining cannot rely on assumptions - provided if you want answers, not hallucinations.

Re: The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A" (2023)

#37

Post by johnsonjo »

altmanaltman wrote: post #28
johnsonjo wrote: post #21

Earlier quoted context omitted.

The space of possible answers is more than one is all GP is saying. Mary Lee Pfeiffer had 1 son, Tom Cruise, and 3 daughters. That's why saying Who is Mary Lee Pfeiffer's son was an identity, because it strictly identifies (or singles out) Mary's son, Tom. Kind of a weird way to draw an analogy, but in math it's kind of like |x|=2 (the absolute value of x is 2) the answer for the value of x is -2 and 2 sure you could…

Yeah i understand that but it is logically right to reply with "Tom Cruise" or any of the girls to that question because by its structure it requires only 1 of the 4 answers since it asks for a singular child right? Or is it like we are saying while that is logically correct, its not the actual answer and the model should reply "they have more than one child, here is the list of children" and that would be a more acc…

> Yeah i understand that but it is logically right to reply with "Tom Cruise" or any of the girls to that question because by its structure it requires only 1 of the 4 answers since it asks for a singular child right?

Yes that is correct it is logically correct to reply Tom Cruise or any of the girls, and that was their point that there are four possible answers to one question.

> Why would it have several answer when you are asking for a singular child?

By the way this quote was the focus of my GP comment since I didn't quote it there.

> Or is it like we are saying while that is logically correct, its not the actual answer and the model should reply "they have more than one child, here is the list of children" and that would be a more accurate one even though the prompt strictly asked for just 1 child?

This was not his point so I feel like we are moving the goal posts a bit, so no though that could have been what's said I don't think that's what was really being said.

Re: The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A" (2023)

#39

Post by [deleted] »

Dylan16807 wrote: post #20
gipp wrote: post #5

As I and several other people pointed out last time this was posted, "A is B," in natural language, does not imply "B is A." "Is" can denote any of many different shades of relationship weaker than logical identity.

That's a fine reaction to the title alone but a very bad reaction to the abstract, where the failure they're criticizing is real and not a childish misunderstanding of the word "is". Is the abstract misleading and the full paper is stupider than the upfront examples? If not this criticism seems like a total waste of time.

[deleted]

Re: The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A" (2023)

#40

Post by [deleted] »

whilenot-dev wrote: post #14
beardyw wrote: post #8

Earlier quoted context omitted.

Yes, "Who is Mary Lee Pfeiffer's son?" happens to have just one answer, whereas "Who is Mary Lee Pfeiffer's child?" would have several.

It could have several correct answers, yes, where one should be logically deducible. "Mary Lee Pfeiffer (A) is Tom Cruise's (B's) mother" and "Tom Cruise (B) is Mary Lee Pfeiffer's (A's) son" are two statements of the same relation. EDIT: I mean if you'd go so far, even Tom Cruise could have multiple mothers, but that doesn't make "Mary Lee Pfeiffer is Tom Cruise's mother" a wrong statement, just because "one of ...…

[deleted]
Post reply on HN