Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Is the reversal curse in LLMs real?

andrewmayne.com

Post reply on HN
Print view View original

171–180 of 211 posts

Re: Is the reversal curse in LLMs real?

#171

Post by mistermann »

amayne wrote: post #48
Imnimo wrote: post #18

>If you start a query with “Mary Lee Pfeiffer”, you’re not going to get very far because neural networks aren’t equidistant grids of points (besides the fact that she may not appear very often under that version of her name.) They’re networks of nodes, some with many connections, some with few. One of the ways you optimize large models is by pruning off weakly connected regions. This may come at the expense of destro…

I've added your suggested test to the blog post: https://andrewmayne.com/2023/11/14/is-the-reversal-curse-rea... Spoiler: It doesn't say "Tom Cruise."

Have to tried replacing "is" with "equals" in the training set where it is valid? I did a search in the article for the word and got no hits.

I'm thinking this might work because "equals" is always logically reversible whereas "is" is not, assuming the statements are technically truthful.

Re: Is the reversal curse in LLMs real?

#172

Post by dragonwriter »

OJFord wrote: post #146
RandomLensman wrote: post #131

Earlier quoted context omitted.

Walter Scheel was never Chancellor, he was fulfilling the duties of the Chancellor while being Foreign Minister, but he was not a Chancellor. So counting him is wrong.

> Walter Scheel was never Chancellor, he was fulfilling the duties of the Chancellor while I think this is really an issue of semantics & translation, because that is just what ' acting X' means? (Admittedly not always while still doing something else too, but I don't see that as significant - if it had gone on long perhaps a junior minister in the foreign office would've been named acting FM in his place too.)

It is exactly what “acting X” means in English.

Re: Is the reversal curse in LLMs real?

#173

Post by RandomLensman »

Kiro wrote: post #164
RandomLensman wrote: post #163

Earlier quoted context omitted.

You can say "I consider Scheel to be Chancellor", but that isn't the same as Scheel having been a Chancellor. I honestly don't understand why in situations with a clear ground truth there is a need to debate and why we would want machines to bungle that.

I don't think it bungled it and I don't agree that there's a clear ground truth here. Quite the opposite: you've only convinced me that the semantic ambiguity is real. It's like debating whether interim CEOs should be included in a list of CEOs.

You are, of course, free to ignore German law and its definitions, but that doesn't change the fact who actually was a Chancellor and who was not. Chancellor is a very well defined role.

Re: Is the reversal curse in LLMs real?

#174

Post by mistermann »

saithound wrote: post #11
famouswaffles wrote: post #4

Earlier quoted context omitted.

It's "obvious" because A is B -> B is A is not a thing that is actually true for the vast majority of text (or really any kind) constructions. It's only a truth of formal logic.

Logician here. We're well aware that "A=B" is not a good formalization of the colloquial/grammarical "A is B", we don't formalize it that way, and we don't regard "if A is B then B is A" is a truth of formal logic. The people making the argument about reversal curses are not logicians, and most of them don't know anything more about formal logic than what anyone would pick up in an undergraduate "Intro to Proofs" cou…

> That said, the semantics of the word "is" in natural language really doesn't matter in this debate.

As a logician, do you consider this necessarily factual:

- using ternary logic

and/or (I am asking both independently and in combination)

- considering the numerous possibilities for ambiguity in "doesn't matter"?

Re: Is the reversal curse in LLMs real?

#175

Post by marcosdumay »

mistermann wrote: post #166
bastawhiz wrote: post #165

Earlier quoted context omitted.

> "A is B" does not imply "B is A" in the first place Not to Bill Clinton this, but I think you are considering this for the wrong interpretation of "is". In this case, "RMS is the founder of the FSF" is the sort of statement implied here. Nobody else was or ever will be the founder of the FSF. So yes, for what the author is describing, "B is A" should always hold. B, in this case, isn't a category, it's an exact ide…

You are using your mind to determine what the proper usage is in this specific concrete instance, LLM's do not have your mind to perform that step. LLM's have to work at higher levels of abstraction, and there the multiple meanings of "is" (which is also very commonly used to mean it is my opinion that X "is" Y) makes sorting things out very difficult. This problem exists for both LLM's and humans, I think it may be…

LLMs are supposed to derive a huge amount of those rules from their training sets. That's the one thing they are good at. They should be pretty good at differentiating the reflexive from the non-reflexive usage of "is" from the context pretty fine.

The fact that the current ones don't is surprising.

Re: Is the reversal curse in LLMs real?

#176

Post by RandomLensman »

dragonwriter wrote: post #172
OJFord wrote: post #146

Earlier quoted context omitted.

> Walter Scheel was never Chancellor, he was fulfilling the duties of the Chancellor while I think this is really an issue of semantics & translation, because that is just what ' acting X' means? (Admittedly not always while still doing something else too, but I don't see that as significant - if it had gone on long perhaps a junior minister in the foreign office would've been named acting FM in his place too.)

It is exactly what “acting X” means in English.

You can clearly see it is confusing the heck out of people here by seeing "acting X" as a version of X when it is not in this case here.

Re: Is the reversal curse in LLMs real?

#177

Post by bastawhiz »

mistermann wrote: post #166
bastawhiz wrote: post #165

Earlier quoted context omitted.

> "A is B" does not imply "B is A" in the first place Not to Bill Clinton this, but I think you are considering this for the wrong interpretation of "is". In this case, "RMS is the founder of the FSF" is the sort of statement implied here. Nobody else was or ever will be the founder of the FSF. So yes, for what the author is describing, "B is A" should always hold. B, in this case, isn't a category, it's an exact ide…

You are using your mind to determine what the proper usage is in this specific concrete instance, LLM's do not have your mind to perform that step. LLM's have to work at higher levels of abstraction, and there the multiple meanings of "is" (which is also very commonly used to mean it is my opinion that X "is" Y) makes sorting things out very difficult. This problem exists for both LLM's and humans, I think it may be…

> there the multiple meanings of "is" (which is also very commonly used to mean it is my opinion that X "is" Y) makes sorting things out very difficult.

I'm not going to hype the technology more than it needs, but it seems to do a pretty damn good job of just this. And to the author's point, it does get this right.

And in fact, I'd suspect many of the cases where the LLM gets this wrong are caused by lots of examples in the training data which were written at different times, where someone else was chancellor. Or where the LLM gets numbering wrong (which is a well established problem).

Re: Is the reversal curse in LLMs real?

#178

Post by thomastjeffery »

mistermann wrote: post #166
bastawhiz wrote: post #165

Earlier quoted context omitted.

> "A is B" does not imply "B is A" in the first place Not to Bill Clinton this, but I think you are considering this for the wrong interpretation of "is". In this case, "RMS is the founder of the FSF" is the sort of statement implied here. Nobody else was or ever will be the founder of the FSF. So yes, for what the author is describing, "B is A" should always hold. B, in this case, isn't a category, it's an exact ide…

You are using your mind to determine what the proper usage is in this specific concrete instance, LLM's do not have your mind to perform that step. LLM's have to work at higher levels of abstraction, and there the multiple meanings of "is" (which is also very commonly used to mean it is my opinion that X "is" Y) makes sorting things out very difficult. This problem exists for both LLM's and humans, I think it may be…

LLMs don't derive or follow rules. LLMs don't have anything to do with rules.

They implicitly recognize (and model) patterns in text, then continue those patterns.

While the patterns that an LLM has modeled may align to the high abstraction we call language, LLMs actually work at a much lower level of abstraction: plain text.

It's the content of that text that is significant. We encode language into text, and LLMs infer something "close enough" to language by blindly modeling it.

Re: Is the reversal curse in LLMs real?

#179

Post by naasking »

mistermann wrote: post #166
bastawhiz wrote: post #165

Earlier quoted context omitted.

> "A is B" does not imply "B is A" in the first place Not to Bill Clinton this, but I think you are considering this for the wrong interpretation of "is". In this case, "RMS is the founder of the FSF" is the sort of statement implied here. Nobody else was or ever will be the founder of the FSF. So yes, for what the author is describing, "B is A" should always hold. B, in this case, isn't a category, it's an exact ide…

You are using your mind to determine what the proper usage is in this specific concrete instance, LLM's do not have your mind to perform that step. LLM's have to work at higher levels of abstraction, and there the multiple meanings of "is" (which is also very commonly used to mean it is my opinion that X "is" Y) makes sorting things out very difficult. This problem exists for both LLM's and humans, I think it may be…

LLMs effectively disambiguate in an arguably similar way humans would: look at the same term in multiple contexts to reduce confidence in incorrect interpretations, leaving only correct interpretations.

Re: Is the reversal curse in LLMs real?

#180

Post by naasking »

RandomLensman wrote: post #138
Kiro wrote: post #137

Earlier quoted context omitted.

He was acting Chancellor. It's like saying a red apple is not an apple. It's a technicality. The fact that we have a completely reasonable explanation to why GPT-4 includes him in the list and you guys still make it sound like the AI is a complete moron is just hilarious to me.

There is no "Acting Chancellor", that is a poor translation of what the job is. There is only a Chancellor, not an "Acting Chancellor" - you have to trust that the Germans and their institutions kind of know who was Chancellor and who was not. The machine is a complete moron for not being able to get basic data from basic sources right.

> The machine is a complete moron for not being able to get basic data from basic sources right.

I'm just going to point out that you're calling an AI moronic for not getting this right when a bunch of humans are also disagreeing with you. Frankly, this really undermines your case that this is an AI failure.

Post reply on HN