Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Is the reversal curse in LLMs real?

andrewmayne.com

Post reply on HN
Print view View original

161–170 of 211 posts

Re: Is the reversal curse in LLMs real?

#161

Post by famouswaffles »

godelski wrote: post #61
majormajor wrote: post #22

Earlier quoted context omitted.

I don't think this is the right explanation, or really that relevant, the suggestions from 'og_kalu and in the article sound more accurate to me. It seems like understanding when "is" is reversible is pretty core to the capabilities of the model, but that's different than having a lot of facts memorized. For instance, a model should be able to answer "who is the star of Mission Impossible" with "Tom Cruise" based on…

I mean transformers are definitely sequentially biased. There's also human speech bias. But I think it's pretty clear that humans are generally invariant to this kind of prompting as well (given that they have the knowledge. More in a different comment). My surprisal is far higher that the reversal "curse" is considered controversial or even surprising than it was when that sensational tweet dropped. It feels pretty…

>But I think it's pretty clear that humans are generally invariant to this kind of prompting as well (given that they have the knowledge. More in a different comment).

I think anyone who's used Anki to learn anything will tell you reversal is by no means free.

고양이 on the front deck, Cat on the back.

You can see this direction and remember cat is the correct answer every single time but completely blank in the other direction. Cat on the front. What Korean word is at the back ?

Re: Is the reversal curse in LLMs real?

#162

Post by getpost »

zapperdulchen wrote: post #65

I played along the example of the chancellor question. If you deviate a little from the examples in the article GPT-4 gets it all wrong: Who is the eighth Federal Chancellor of the Federal Republic of Germany? - Olaf Scholz (wrong, Angela Merkel) https://chat.openai.com/share/937795ea-bd91-43ee-bd76-1a125f... Who was the eighth Federal Chancellor of the Federal Republic of Germany? - Helmut Kohl (wrong, Angela Merkel…

I asked for a list of Federal Chancellors of the Federal Republic of Germany and GPT4 included Scheel ("acting"), which makes Schröder the 8th. I often troubleshoot GPT4 replies by asking it for more context. "Is" vs "was" is interesting an find!

https://chat.openai.com/share/ddd2800e-630c-4ff7-9992-14b265...

Re: Is the reversal curse in LLMs real?

#163

Post by RandomLensman »

Kiro wrote: post #158
RandomLensman wrote: post #148

Earlier quoted context omitted.

There is zero ambiguity under German as law as to who was a Chancellor and who was not. Not a matter of wikipedia etc. The way to be Chancellor is through article 63 of the Grundgesetz, while Scheel was put into the caretaker role via article 69. This explains it a bit https://de.wikipedia.org/wiki/Vizekanzler_(Deutschland) - Scheel was only taking on the function, not the office. This kind of giving some machine the…

He's also included in this list between 4 and 5: https://de.wikipedia.org/wiki/Bundeskanzler_(Deutschland)#De... When you need to bring up article 63 and 69 of the Grundgesetz to prove that the claim is ludicrous, maybe the reasonable thing to say instead is "I understand why you might think that".

You can say "I consider Scheel to be Chancellor", but that isn't the same as Scheel having been a Chancellor.

I honestly don't understand why in situations with a clear ground truth there is a need to debate and why we would want machines to bungle that.

Re: Is the reversal curse in LLMs real?

#164

Post by Kiro »

RandomLensman wrote: post #163
Kiro wrote: post #158

Earlier quoted context omitted.

He's also included in this list between 4 and 5: https://de.wikipedia.org/wiki/Bundeskanzler_(Deutschland)#De... When you need to bring up article 63 and 69 of the Grundgesetz to prove that the claim is ludicrous, maybe the reasonable thing to say instead is "I understand why you might think that".

You can say "I consider Scheel to be Chancellor", but that isn't the same as Scheel having been a Chancellor. I honestly don't understand why in situations with a clear ground truth there is a need to debate and why we would want machines to bungle that.

I don't think it bungled it and I don't agree that there's a clear ground truth here. Quite the opposite: you've only convinced me that the semantic ambiguity is real. It's like debating whether interim CEOs should be included in a list of CEOs.

Re: Is the reversal curse in LLMs real?

#165

Post by bastawhiz »

gipp wrote: post #8

I fall somewhere on the skeptic side of the LLM spectrum. But this "flaw" just does not seem to have the force that its proponents seem to think it does, unless I'm missing something significant. Simply because in the context of natural language (Rather than formal logical statements), "A is B" does not imply "B is A" in the first place. "Is" can encompass a wide variety of logical relationships in colloquial usage,…

> "A is B" does not imply "B is A" in the first place

Not to Bill Clinton this, but I think you are considering this for the wrong interpretation of "is". In this case, "RMS is the founder of the FSF" is the sort of statement implied here. Nobody else was or ever will be the founder of the FSF. So yes, for what the author is describing, "B is A" should always hold. B, in this case, isn't a category, it's an exact identity. The author doesn't explicitly state this, but it's clear this is what they're talking about.

Re: Is the reversal curse in LLMs real?

#166

Post by mistermann »

bastawhiz wrote: post #165
gipp wrote: post #8

I fall somewhere on the skeptic side of the LLM spectrum. But this "flaw" just does not seem to have the force that its proponents seem to think it does, unless I'm missing something significant. Simply because in the context of natural language (Rather than formal logical statements), "A is B" does not imply "B is A" in the first place. "Is" can encompass a wide variety of logical relationships in colloquial usage,…

> "A is B" does not imply "B is A" in the first place Not to Bill Clinton this, but I think you are considering this for the wrong interpretation of "is". In this case, "RMS is the founder of the FSF" is the sort of statement implied here. Nobody else was or ever will be the founder of the FSF. So yes, for what the author is describing, "B is A" should always hold. B, in this case, isn't a category, it's an exact ide…

You are using your mind to determine what the proper usage is in this specific concrete instance, LLM's do not have your mind to perform that step. LLM's have to work at higher levels of abstraction, and there the multiple meanings of "is" (which is also very commonly used to mean it is my opinion that X "is" Y) makes sorting things out very difficult.

This problem exists for both LLM's and humans, I think it may be fundamental to reality itself, as it currently is at least.

I wonder if they changed the training data to consistently use "equals" where possible (even if it sounds weird to the reader) would make a difference.

Re: Is the reversal curse in LLMs real?

#167

Post by OJFord »

RandomLensman wrote: post #150
OJFord wrote: post #146

Earlier quoted context omitted.

> Walter Scheel was never Chancellor, he was fulfilling the duties of the Chancellor while I think this is really an issue of semantics & translation, because that is just what ' acting X' means? (Admittedly not always while still doing something else too, but I don't see that as significant - if it had gone on long perhaps a junior minister in the foreign office would've been named acting FM in his place too.)

That is not semantics. The law is very clear: Scheel was not a Chancellor (and indeed he is not counted as one in Germany: https://www.bundeskanzler.de/bk-en/chancellery/federal-chanc... ). Article 69 Grundgesetz has a sort of "Caretaker Chancellor" that has the function but - importantly - not the office. They way to have the office is through article 63. It works differently in Germany to the US, for example, where…

What distinction are you making between 'acting X' and 'caretaker X'?

I'm British, not American, so I'm not assuming something like the vice president automatically becomes president, we don't have that either. If anything it's even less than Germany since (at least in theory and history, modern media etc. makes it a bit different in practice) there's nothing special about the PM, it's just the governing party's leader. I suppose though you could say the heir apparent to the throne immediately becoming the monarch on the death of the previous one is like vice taking over - but I don't think that detracts from my point because nobody would call that 'acting monarch', they just are.

'Acting X' means doing the necessary duties, but not any major decisions that can be avoided/deferred, while the 'real' replacement is found. Sometimes it ends up being the same person, e.g. the head of some division is 'acting CEO' for a while as the board searches for a new CEO, ultimately ends up going with that person and title changes to just 'CEO' - or they don't, and go back to old job, or maybe quit in a huff, and the person they found is just 'CEO' taking over from the 'acting'.

Re: Is the reversal curse in LLMs real?

#168

Post by jameshart »

bjornsing wrote: post #94

I didn’t read the whole thing, but the first part about Tom Cruise and his mother sounded very flawed: Of course the LLM could learn the reverse relationship if it had better data where the reverse relationship is a common occurrence. The point of the reversal curse argument (I guess) is that it should be able to learn the reverse relationship from entirely other examples [1], but seemingly is not. 1. That is, the LL…

LLM training teaches it that ‘Mary’,’Lee’, and ‘Pfeiffer’ are words that appear in association with ‘Tom’, ‘Cruise’, and ‘mother’. Probably also in association with ‘son’, and ‘family’.

But the association between the words ‘Mary’, ‘Lee’, ‘Pfeiffer’ and ‘son’ or ‘mother’ point to other words more strongly than they do to ‘Tom’ or ‘Cruise’. Sure, Tom Cruise is probably in there but so is probably John Henry Kelley (Michelle Marie Pfeiffer’s son), and George Washington Custis Lee (whose father was Robert E Lee and whose mother was called Mary).

A random piece of text containing the words Mary Lee Pfeiffer is just, based on its training, not likely to be about Tom Cruise. There’s nothing there anchoring it particularly to those words.

It’s possible that the ‘Pfeiffer’ in there screws it up more, as well - it’s like it vaguely knows there’s a Hollywood connection but of course it crosses its wires and thinks it’s probably Michelle Pfeiffer.

Be honest: if a pub quiz question came up asking “which Hollywood superstar’s mother is Mary Lee Pfeiffer“, you’d probably not guess Tom Cruise either.

But here’s the thing: once we go beyond training into specific completions, if you ask GPT ‘who is Tom Cruise’s mother’, it answers correctly; and then once it has that A is B in context, you can ask it ‘who is Mary Lee Pfeiffer’s son?’ And of course it knows how to complete that B is A.

So I definitely agree with the piece here - there’s no ‘reversal curse’, there’s just asymmetric information relevance.

Re: Is the reversal curse in LLMs real?

#169

Post by RandomLensman »

OJFord wrote: post #167
RandomLensman wrote: post #150

Earlier quoted context omitted.

That is not semantics. The law is very clear: Scheel was not a Chancellor (and indeed he is not counted as one in Germany: https://www.bundeskanzler.de/bk-en/chancellery/federal-chanc... ). Article 69 Grundgesetz has a sort of "Caretaker Chancellor" that has the function but - importantly - not the office. They way to have the office is through article 63. It works differently in Germany to the US, for example, where…

What distinction are you making between 'acting X' and 'caretaker X'? I'm British, not American, so I'm not assuming something like the vice president automatically becomes president, we don't have that either. If anything it's even less than Germany since (at least in theory and history, modern media etc. makes it a bit different in practice) there's nothing special about the PM, it's just the governing party's lead…

The real issue here is that people stick to the word "Chancellor", not the "acting" as such.

There is no named role of "Acting Chancellor" created in the basic law, instead someone is just tasked with performing certain duties (and people might sometimes call it "acting Chancellor" or whatever). However, the position "Chancellor" is a well defined & special role (unlike perhaps PM) and whatever that other "acting" role is, it isn't a Chancellor. It is a bit like adding Oliver Cromwell to list of English Kings or Kamala Harris to the list of US Presidents - you can do it but you then move beyond the "standard" definitions.

Re: Is the reversal curse in LLMs real?

#170

Post by mistermann »

croes wrote: post #155
gipp wrote: post #8

I fall somewhere on the skeptic side of the LLM spectrum. But this "flaw" just does not seem to have the force that its proponents seem to think it does, unless I'm missing something significant. Simply because in the context of natural language (Rather than formal logical statements), "A is B" does not imply "B is A" in the first place. "Is" can encompass a wide variety of logical relationships in colloquial usage,…

But humans know when "is" means equal.

It only takes two humans getting it correct once for your statement to be technically true though, demonstrating how easily seemingly innocuous and straightforward language can be misleading.

The opposite statement, humans don't know when "is" means equal is also a true statement, perhaps even more true.

Post reply on HN