Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Is the reversal curse in LLMs real?

andrewmayne.com

Post reply on HN
Print view View original

61–70 of 211 posts

Re: Is the reversal curse in LLMs real?

#61

Post by godelski »

majormajor wrote: post #22
gipp wrote: post #8

I fall somewhere on the skeptic side of the LLM spectrum. But this "flaw" just does not seem to have the force that its proponents seem to think it does, unless I'm missing something significant. Simply because in the context of natural language (Rather than formal logical statements), "A is B" does not imply "B is A" in the first place. "Is" can encompass a wide variety of logical relationships in colloquial usage,…

I don't think this is the right explanation, or really that relevant, the suggestions from 'og_kalu and in the article sound more accurate to me. It seems like understanding when "is" is reversible is pretty core to the capabilities of the model, but that's different than having a lot of facts memorized. For instance, a model should be able to answer "who is the star of Mission Impossible" with "Tom Cruise" based on…

I mean transformers are definitely sequentially biased. There's also human speech bias. But I think it's pretty clear that humans are generally invariant to this kind of prompting as well (given that they have the knowledge. More in a different comment). My surprisal is far higher that the reversal "curse" is considered controversial or even surprising than it was when that sensational tweet dropped. It feels pretty well studied that sequential algorithms are sequentially biased.

Re: Is the reversal curse in LLMs real?

#62

Post by anon291 »

gipp wrote: post #8

I fall somewhere on the skeptic side of the LLM spectrum. But this "flaw" just does not seem to have the force that its proponents seem to think it does, unless I'm missing something significant. Simply because in the context of natural language (Rather than formal logical statements), "A is B" does not imply "B is A" in the first place. "Is" can encompass a wide variety of logical relationships in colloquial usage,…

> skeptic side of the LLM spectrum

Can you explain what you are skeptical of? There is ample evidence that LLMs are what they say they are: a series of transformer (usually) functions with learned weights that can successfully be used to generate human like text in many domains. Do you disbelieve this? Or are you skeptical of something else.

Re: Is the reversal curse in LLMs real?

#63

Post by School-Cotton »

willsoon wrote: post #34
p1necone wrote: post #23

Earlier quoted context omitted.

I don't think it's as simple as that - "the apple is red" is "single thing belongs to category", whereas "Olaf Scholz was the ninth Chancellor of Germany" is "single thing is single thing" - the latter is reversible, the former is not. I would expect a good language model to be able to parse both sentences correctly.

You are right. You are thinking correctly. But "the apple is red" might not mean that this particular apple belongs ---to put it in your wording--- to the category red, but that the category of things we call apple also belongs to the category of things that are red. And generally speaking, I think that is the meaning.

> but that the category of things we call apple also belongs to the category of things that are red

No, in English this would be “apples are red”, not “the apple is red”.

Re: Is the reversal curse in LLMs real?

#64

Post by School-Cotton »

tkgally wrote: post #33
zarzavat wrote: post #27

Humans are also vulnerable to the reversal curse! When you learn languages you have to learn both directions (chat is cat and cat is chat), anybody who has built an anki deck will know this, otherwise you will be better in one direction than the other.

As one gets deeper into learning another language, it’s also important to be aware that the meanings of words in different languages rarely map to each other in clean bijections. Common words especially tend to be semantic clouds, not fixed points of meaning. I don’t know French, but I am sure there are many cases where an English phrase or sentence that includes ‘cat’ should not be translated into French with ‘chat’…

I think this has as much to do with abstract vs. concrete words as common vs. uncommon. If you had to pick a word corresponding to “chat” it’s quite clearly “cat”, despite a few different expressions. But it’s rather difficult to translate “justement” to English, or “random” to French, without further context.

Re: Is the reversal curse in LLMs real?

#65

Post by zapperdulchen »

I played along the example of the chancellor question.

If you deviate a little from the examples in the article GPT-4 gets it all wrong:

Who is the eighth Federal Chancellor of the Federal Republic of Germany? - Olaf Scholz (wrong, Angela Merkel)

https://chat.openai.com/share/937795ea-bd91-43ee-bd76-1a125f...

Who was the eighth Federal Chancellor of the Federal Republic of Germany? - Helmut Kohl (wrong, Angela Merkel)

https://chat.openai.com/share/937795ea-bd91-43ee-bd76-1a125f...

Interestingly whether you put is or was into the question does make a difference.

Even when allowed to surf the web, ChatGPT gets it wrong:

Who was the eighth Federal Chancellor of the Federal Republic of Germany? - Gerhard Schröder (wrong, Angela Merkel)

https://chat.openai.com/share/867cb3bc-f642-4d2a-b80d-fe8dc0...

Though it's referring to the right Wikipedia page: https://en.m.wikipedia.org/wiki/List_of_chancellors_of_Germa...

Re: Is the reversal curse in LLMs real?

#66

Post by Falkon1313 »

thaumasiotes wrote: post #20
akoboldfrying wrote: post #9

Only skimmed this and didn't read the underlying paper, but I was surprised to see no mention of the fact that "A is B" often does not at all imply "B is A" in everyday language: A bird is an animal, but it's wrong to conclude that an arbitrary animal must be a bird.

This is bad news for all the people who like to say "A winner is you!"

But obviously that implies that you are a chicken dinner.

At least according to some people's expectations, presumably.

Re: Is the reversal curse in LLMs real?

#68

Post by dragonwriter »

zapperdulchen wrote: post #65

I played along the example of the chancellor question. If you deviate a little from the examples in the article GPT-4 gets it all wrong: Who is the eighth Federal Chancellor of the Federal Republic of Germany? - Olaf Scholz (wrong, Angela Merkel) https://chat.openai.com/share/937795ea-bd91-43ee-bd76-1a125f... Who was the eighth Federal Chancellor of the Federal Republic of Germany? - Helmut Kohl (wrong, Angela Merkel…

> Even when allowed to surf the web, ChatGPT gets it wrong:

> Who was the eighth Federal Chancellor of the Federal Republic of Germany? - Gerhard Schröder

Its probably counting Walter Scheel, who was Acting Chancellor in 1974, and you are probably not.

When I asked ChatGPT to list the chancellors in order and identify the eighth, it listed eight ending in Schröder, with Scheel and his ten day Acting Chancellor tenure in May 74 as number 5. (Scheel’s tenure is in the timeline on the Wikipedia page you cite, though not in the numbered list in that page, which is why there is a dicontinuity in thr dates on the numbered list.)

Re: Is the reversal curse in LLMs real?

#69

Post by diffeomorphism »

Well, yeah because the reversal is just completely wrong most of the time.

Only if you have additional context clues like definite articles or some outside knowledge that a role or property is unique, then you can maybe, sometimes conclude the reverse. Not a bug, working as intended.

Re: Is the reversal curse in LLMs real?

#70

Post by trimethylpurine »

eru wrote: post #60
tysam_and wrote: post #38

Earlier quoted context omitted.

Bit of a nitpick, "red is the apple" is both a valid sentence and one that also conveys the original relationship held by the opposite phrasing, so, in this case, "the apple is red" does indeed imply "red is the apple", and accurately so.

Perhaps have a look at this famous syllogism instead: (1) Mortal was Socrates. (2) All humans are mortal. (3) Therefor all humans are Socrates.

This isn't exactly on topic. A claim about the apple's color is attributable to the apple and vice versa. An apple can contextually help someone understand the color red by example. There is no "therefore" in such a claim, and in fact the algorithm is simply gauging context statistically. Claim 3 in the syllogism would statistically be selected against, because a "therefore" that is false obviously weighs it in the opposite direction from truth. That's not the case for the apple. Essentially it's not difficult for the LLM to win on this one, just as it is not difficult for a human and for the same reasons.
Post reply on HN