Is the reversal curse in LLMs real?
andrewmayne.com
Is the reversal curse in LLMs real?
1–10 of 211 posts
Re: Is the reversal curse in LLMs real?
#2> Saying that models can’t automatically generalize from B to A when B is vastly underrepresented in the dataset feels rather obvious and not so much a curse as a description of how neural nets function
More importantly perhaps, it's not how people (at least non AI/ML research types) typically think they work, and much of the handwaving and hype around it isn't helping improve that.
Re: Is the reversal curse in LLMs real?
#3> Saying that models can’t automatically generalize from B to A when B is vastly underrepresented in the dataset feels rather obvious and not so much a curse as a description of how neural nets function More importantly perhaps, it's not how people (at least non AI/ML research types) typically think they work, and much of the handwaving and hype around it isn't helping improve that.
It doesn't seem obvious at all unless you start with the assumption that not being able to derive A=B -> B=A is "obvious", which it clearly isn't to most people.
Indeed, just because the lack of that capability is a result of the design of LLMs doesn't mean it's a feature of LLMs. It could also be that it's a bug of LLMs. Which one depends on what the expected behavior is, and the expected behavior from the product is being able to perform the above logical derivation.
tl;dr: the article reaffirms that the "curse" is true, and for the reasons claimed too, but it's a feature, not a bug.
Re: Is the reversal curse in LLMs real?
#4> Saying that models can’t automatically generalize from B to A when B is vastly underrepresented in the dataset feels rather obvious and not so much a curse as a description of how neural nets function More importantly perhaps, it's not how people (at least non AI/ML research types) typically think they work, and much of the handwaving and hype around it isn't helping improve that.
It doesn't seem obvious at all unless you start with the assumption that not being able to derive A=B -> B=A is "obvious", which it clearly isn't to most people. Indeed, just because the lack of that capability is a result of the design of LLMs doesn't mean it's a feature of LLMs. It could also be that it's a bug of LLMs. Which one depends on what the expected behavior is, and the expected behavior from the product i…
It's "obvious" because A is B -> B is A is not a thing that is actually true for the vast majority of text (or really any kind) constructions. It's only a truth of formal logic.
Re: Is the reversal curse in LLMs real?
#5You have to add context the relationship of A to B isaintained under reflexive algebra..
Re: Is the reversal curse in LLMs real?
#6[deleted]
Re: Is the reversal curse in LLMs real?
#7> This isn’t a failure of neural networks. It’s a feature. It’s why you’re not flooded with every single memory and experience you’ve ever had every moment.
This is an interesting point, and made me think on whether this "reversal curse" is something we experience with our own, human neural networks. I think it is. Like, I can imagine being given a character in a movie, being able to tell you what actor played them, but given the actor and the movie, not being able to tell you what character they played, or vice-versa. So the pairing exists in my brain somewhere, but I only have an accessible pointer to one side. I think we run into cases like this all the time, actually.
Re: Is the reversal curse in LLMs real?
#8I fall somewhere on the skeptic side of the LLM spectrum. But this "flaw" just does not seem to have the force that its proponents seem to think it does, unless I'm missing something significant. Simply because in the context of natural language (Rather than formal logical statements), "A is B" does not imply "B is A" in the first place. "Is" can encompass a wide variety of logical relationships in colloquial usage, not just exact identity. "The apple is red" does not imply "red is the apple," as a trivial example.
Re: Is the reversal curse in LLMs real?
#9Only skimmed this and didn't read the underlying paper, but I was surprised to see no mention of the fact that "A is B" often does not at all imply "B is A" in everyday language: A bird is an animal, but it's wrong to conclude that an arbitrary animal must be a bird.
Re: Is the reversal curse in LLMs real?
#10The article is a decent peer review and refutation of “the reversal curse”. Some of the comments given here clearly haven’t read the whole article though - arriving at similarly skeptical conclusions that are clearly present and expanded on in the article.
Why do people feel the need to do this here? Armchair commentary on advanced material is one of the main reasons I avoid Reddit. And furthermore why does it feel like you’re not allowed to suggest this as a response? I should be able to say “RTFA” but here I feel like I’m going to be scolded or banned by moderation.