Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A" (2023)

arxiv.org

Post reply on HN
Print view View original

21–30 of 54 posts

Re: The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A" (2023)

#21

Post by johnsonjo »

altmanaltman wrote: post #12
beardyw wrote: post #8

Earlier quoted context omitted.

Yes, "Who is Mary Lee Pfeiffer's son?" happens to have just one answer, whereas "Who is Mary Lee Pfeiffer's child?" would have several.

Why would it have several answer when you are asking for a singular child? Isn't the right way to phrase that question be "Who are Mary Lee Pfeiffer's children?" to get multiple answers?

The space of possible answers is more than one is all GP is saying. Mary Lee Pfeiffer had 1 son, Tom Cruise, and 3 daughters. That's why saying Who is Mary Lee Pfeiffer's son was an identity, because it strictly identifies (or singles out) Mary's son, Tom.

Kind of a weird way to draw an analogy, but in math it's kind of like |x|=2 (the absolute value of x is 2) the answer for the value of x is -2 and 2 sure you could reply that the answer is 2 and be correct (even though you would still be missing something, because the space of possible answers includes both 2 and -2). To relay that back to Mary Lee Pfeiffer saying she has Tom Cruise as a child is correct, but the actual answer could include any 4 of her children (including Tom or one of the 3 daughters) and still be correct.

Re: The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A" (2023)

#22

Post by clcaev »

gipp wrote: post #5

As I and several other people pointed out last time this was posted, "A is B," in natural language, does not imply "B is A." "Is" can denote any of many different shades of relationship weaker than logical identity.

As a stylistic comment, on HN I am seeing more and more of "As I have always said..." or similar opening constructs. Concisely documenting a phenomena or affecting change requires detailed, sustained effort. Merely observing a pattern isn't sufficient to be notable.

A succinct point, ideally noting counter arguments, is most welcome. Further, if there is substantial prior discussion or relevant literature, a link is productive.

Re: The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A" (2023)

#23

Post by altmanaltman »

niam wrote: post #17
altmanaltman wrote: post #12

Earlier quoted context omitted.

Why would it have several answer when you are asking for a singular child? Isn't the right way to phrase that question be "Who are Mary Lee Pfeiffer's children?" to get multiple answers?

Your question arguably doesn't have multiple answers. There's one answer: the set of children (discounting different orders of the set). The question "Who is her child" has multiple answers because it asks you to deliver a single answer.

But wouldn't every one of those multiple answers be the correct one in this case? Like it can say child a or child b or child c (hypothetical) and while there are mutiple answers, each of them is a logically right one for the question "Who is her child?" no? So how do we judge what is the absolute right answer to that? its ambigious when you say child

Re: The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A" (2023)

#24

Post by bayindirh »

clcaev wrote: post #22
gipp wrote: post #5

As I and several other people pointed out last time this was posted, "A is B," in natural language, does not imply "B is A." "Is" can denote any of many different shades of relationship weaker than logical identity.

As a stylistic comment, on HN I am seeing more and more of "As I have always said..." or similar opening constructs. Concisely documenting a phenomena or affecting change requires detailed, sustained effort. Merely observing a pattern isn't sufficient to be notable. A succinct point, ideally noting counter arguments, is most welcome. Further, if there is substantial prior discussion or relevant literature, a link is…

As a better version of this, I like to link my prior comments, or if I'm referring to another comment in the same thread, I like to link that one.

H in HTML & third W in WWW is meant to denote connections.

Re: The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A" (2023)

#25

Post by Dylan16807 »

sigmoid10 wrote: post #19
beardyw wrote: post #8

Earlier quoted context omitted.

Yes, "Who is Mary Lee Pfeiffer's son?" happens to have just one answer, whereas "Who is Mary Lee Pfeiffer's child?" would have several.

I wouldn't resort to language examples, as the other comments show how this gets imprecise and lost in semantic details quickly. Instead think of a basic logic example: Consider an OR gate with inputs A and B and output X. If B=1 that means X=1. But if X=1 you can't infer that B=1, because there is an alternative (i.e. A=1,B=0). So from B=1→X=1, the inverse simply does not follow. This extends to all statements where…

Except the researchers are not asking about the inverse of arbitrary logic statements.

Re: The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A" (2023)

#26

Post by ACCount37 »

eurekin wrote: post #3

I remember seeing this popping up in discussions the first time, but never noticed any resolution (other than to train both sides). Has SOTA advanced?

No, it's just that no one really cares. It doesn't seem to cause identifiable faults in model reasoning in the real world.

It could be fun to try to make the model pre-learn a "reversal prior" that would cause a greater degree of generalization there, but I'm yet to see a published result like this. Let alone one that would demonstrate such a prior to be useful.

Re: The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A" (2023)

#28

Post by altmanaltman »

johnsonjo wrote: post #21
altmanaltman wrote: post #12

Earlier quoted context omitted.

Why would it have several answer when you are asking for a singular child? Isn't the right way to phrase that question be "Who are Mary Lee Pfeiffer's children?" to get multiple answers?

The space of possible answers is more than one is all GP is saying. Mary Lee Pfeiffer had 1 son, Tom Cruise, and 3 daughters. That's why saying Who is Mary Lee Pfeiffer's son was an identity, because it strictly identifies (or singles out) Mary's son, Tom. Kind of a weird way to draw an analogy, but in math it's kind of like |x|=2 (the absolute value of x is 2) the answer for the value of x is -2 and 2 sure you could…

Yeah i understand that but it is logically right to reply with "Tom Cruise" or any of the girls to that question because by its structure it requires only 1 of the 4 answers since it asks for a singular child right?

Or is it like we are saying while that is logically correct, its not the actual answer and the model should reply "they have more than one child, here is the list of children" and that would be a more accurate one even though the prompt strictly asked for just 1 child?

Re: The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A" (2023)

#29

Post by beardyw »

latexr wrote: post #18
beardyw wrote: post #8

Earlier quoted context omitted.

Yes, "Who is Mary Lee Pfeiffer's son?" happens to have just one answer, whereas "Who is Mary Lee Pfeiffer's child?" would have several.

> "Who is Mary Lee Pfeiffer's child?" would have several. Might have several. It only has one answer if there is only one child, which appears to be the case here. They are measuring against what they told the model, not necessarily facts mapping to the real world. In this case, the correct would always include “Tom Cruise” even if it needed a clarifying “there might be others I have no knowledge of”.

> It only has one answer if there is only one child

My understanding is she has four.

Re: The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A" (2023)

#30

Post by Dibby053 »

zmgsabst wrote: post #6

“A is B” doesn’t generally entail “B is A”. “A square is a rectangle” does not entail “a rectangle is a square”. Similarly, “Socrates is alive” doesn’t entail “alive is Socrates”. Notably, they mention when context is included, LLM performance rises — ie, exactly when we include extra information that allows it to recognize what kind of information is being conveyed. But the LLM is correct not to generalize that patt…

"A is the B" does entail "the B is A", because "the" establishes an identity/bijection.
Post reply on HN