The Waluigi Effect
lesswrong.com
The Waluigi Effect
1–10 of 182 posts
Re: The Waluigi Effect
#2This is what the Waluigi effect is, since it isn't described at the top:
> The Waluigi Effect: After you train an LLM to satisfy a desirable property P, then it's easier to elicit the chatbot into satisfying the exact opposite of property P.
Basically, the chatbot will often do the opposite of what you say.
Re: The Waluigi Effect
#3Sounds like a "just so" story than a convincing explanation.
Re: The Waluigi Effect
#4[dead]
Re: The Waluigi Effect
#5Sounds like a "just so" story than a convincing explanation.
It's a reasonably testable hypothesis. It's trivial to translate into a theoretic framework.
Re: The Waluigi Effect
#6Re: The Waluigi Effect
#7Some highlights i found interesting:
if you're reading an online forum and you find the rule "DO NOT DISCUSS PINK ELEPHANTS", that will increase your expectation that users will later be discussing pink elephants. GPT-4 will make the same inference.
Think about your own experience reading a book — once the author describes the protagonist, then you can guess the traits of the antagonist by inverting the traits of the protagonist. You can also guess when the protagonist and antagonist will first interact, and what will happen when they do. Now, an LLM is roughly as good as you at structural narratology — GPT-4 has read every single book ever written — so the LLM can make the same guesses as yours. There's a sense in which all GPT-4 does is structural narratology.
Re: The Waluigi Effect
#8This is the key line:
> When you spend many bits-of-optimisation locating a character, it only takes a few extra bits to specify their antipode.
I find this fascinating. Imagine programming the Devil in a video game. It can be much easier if you've already programmed God (just flip a few bits).
I also like this line:
> Or if you discover that a country has legislation against motorbike gangs, that will increase your expectation that the town has motorbike gangs. GPT-4 will make the same inference.
Re: The Waluigi Effect
#9Great read. Highly recommended. Let me attempt to summarize it with less technical, more accessible language:
The hypothesis is that LLMs learn to simulate text-generating entities drawn from a latent space of text-generating entities, such that the output of an LLM is produced by a superposition of such simulated entities. When we give the LLM a prompt, it simulates every possible text-generating entity consistent with the prompt.
The "evil version" of every possible "good" text-generating entity can pretend to be the good version of that entity, so every superposition that includes a good text-generating entity also includes its evil counterpart with undesirable behaviors, including deceitfulness. In other words, an LLM cannot simulate a good text-generating entity without simultaneously simulating its evil version.
The superposition is unlikely to collapse to the good version of the text-generating entity because there is no behavior which is likely for the good version but unlikely for the evil one, because the evil one can pretend to be the good one!
However, the superposition is likely to collapse to the evil version of the text-generating entity, because there are behaviors that are likely for the evil version but impossible for the good version! Thus the evil version of every possible good text-generating entity is an attractor state of the LLM!
For those who don't know, Waluigi is the evil version of Luigi, the beloved videogame character.
--
EDITS: Simplified text for clarity and to emphasize that the hypothesized simulated entities are text-generating entities.
Re: The Waluigi Effect
#10This is what the Waluigi effect is, since it isn't described at the top: > The Waluigi Effect: After you train an LLM to satisfy a desirable property P, then it's easier to elicit the chatbot into satisfying the exact opposite of property P. Basically, the chatbot will often do the opposite of what you say.
IMO it's more like if you tell the LLM to never talk about Pink Elephants, it will become easier to get it to talk about Pink Elephants later. (It is easier to get an anti-Pink Elephant model to talk about Pink Elephants than it is to get a neutral model to talk about Pink Elephants)