Okay, what if we flip the problem on its head? Try to make the chatbot seem rude and unhelpful but then it turns out it has a heart of gold?
[deleted]
The Waluigi Effect
21–30 of 182 posts
Re: The Waluigi Effect
#22Great read. Highly recommended. Let me attempt to summarize it with less technical, more accessible language: The hypothesis is that LLMs learn to simulate text-generating entities drawn from a latent space of text-generating entities , such that the output of an LLM is produced by a superposition of such simulated entities. When we give the LLM a prompt, it simulates every possible text-generating entity consistent…
>For those who don't know, Waluigi is the evil version of Luigi, the beloved videogame character. Is he really evil though? I thought all he did was play tennis and golf and drive a go-kart.
I personally like the fan-made reputation he has for being someone who tries hard and keeps failing/being rejected due to no fault of his own. There's an extremely artistic video called 'Nothing's Working Out - Waluigi' on YouTube that I really enjoy (warning: very heavy themes) that covers that base.
I think it resonates with the way that a lot of people are feeling now, as well as is empowering by rewriting the other side of the 'greedy evil bad guy who gets what he deserves' as a deeply flawed, desperate human being.
Definitely a semi-side tangent at this point, I have always enjoyed creative, deep reinterpretations of well-established characters.
Re: The Waluigi Effect
#23Great read. Highly recommended. Let me attempt to summarize it with less technical, more accessible language: The hypothesis is that LLMs learn to simulate text-generating entities drawn from a latent space of text-generating entities , such that the output of an LLM is produced by a superposition of such simulated entities. When we give the LLM a prompt, it simulates every possible text-generating entity consistent…
>The output of an LLM is produced by a superposition of simulated entities. When we give the LLM a prompt, it simulates every possible entity consistent with the prompt. There is absolutely no theoretical justification for this assertion that LLMs somehow have some emergent quantum mechanical behavior, metaphorical or otherwise.
Superposition just means “linear combination” in this context. Basically, a weighted mixture of “simulated entities” (or possible responses).
Re: The Waluigi Effect
#24Earlier quoted context omitted.
>The output of an LLM is produced by a superposition of simulated entities. When we give the LLM a prompt, it simulates every possible entity consistent with the prompt. There is absolutely no theoretical justification for this assertion that LLMs somehow have some emergent quantum mechanical behavior, metaphorical or otherwise.
As I wrote, this is a hypothesis . Also, I'm simplifying things a lot to make them accessible. The OP goes into a lot more detail. I highly recommend you read it.
I did read it. The whole article reads like someone trying to make a loose conjecture appear quantitatively rigorous by abusing terminology from physics, statistics, and chaos theory (among other quantitative fields). For example,
>the superposition is unlikely to collapse to the luigi simulacrum because there is no behaviour which is likely for luigi but very unlikely for waluigi. Recall that the waluigi is pretending to be luigi! This is formally connected to the asymmetry of the Kullback-Leibler divergence.
The K-L divergence has absolutely zero discernible relevance here. The cross entropy loss function of a categorical predictor (like the token output of an LLM) can be formulated in terms of K-L divergence, but this has absolutely zero relevance to the macroscopic phenomena the author is conjecturing.
Forget “less wrong,” much of this article is not even wrong [0].
Re: The Waluigi Effect
#25Earlier quoted context omitted.
>The output of an LLM is produced by a superposition of simulated entities. When we give the LLM a prompt, it simulates every possible entity consistent with the prompt. There is absolutely no theoretical justification for this assertion that LLMs somehow have some emergent quantum mechanical behavior, metaphorical or otherwise.
Superposition just means “linear combination” in this context. Basically, a weighted mixture of “simulated entities” (or possible responses). https://en.m.wikipedia.org/wiki/Superposition_principle
The author liberally alludes to “superposition collapse,” which implies that they’re referring to its quantum mechanical meaning.
Re: The Waluigi Effect
#26Okay, what if we flip the problem on its head? Try to make the chatbot seem rude and unhelpful but then it turns out it has a heart of gold?
The article discusses this. The problem is that it's a lot less likely for the chatbot to veer in that direction (seems initially hostile, but is secretly good) than the opposite (seems initially good, but is secretly hostile):
> I claim that this explains the asymmetry — if the chatbot responds rudely, then that permanently vanishes the polite luigi simulacrum from the superposition; but if the chatbot responds politely, then that doesn't permanently vanish the rude waluigi simulacrum. Polite people are always polite; rude people are sometimes rude and sometimes polite.
Re: The Waluigi Effect
#27This is what the Waluigi effect is, since it isn't described at the top: > The Waluigi Effect: After you train an LLM to satisfy a desirable property P, then it's easier to elicit the chatbot into satisfying the exact opposite of property P. Basically, the chatbot will often do the opposite of what you say.
Reads like the Streisand effect
Re: The Waluigi Effect
#28are these LLMs just answering the question "if you found this text on the internet (the prompt) what would most likely follow" ?
In essence, yes I think, but... isn't that essentially not much different than what I'm doing in making this comment?
Re: The Waluigi Effect
#29Great read. Highly recommended. Let me attempt to summarize it with less technical, more accessible language: The hypothesis is that LLMs learn to simulate text-generating entities drawn from a latent space of text-generating entities , such that the output of an LLM is produced by a superposition of such simulated entities. When we give the LLM a prompt, it simulates every possible text-generating entity consistent…
Sometimes I really can't tell if these people are serious or not. They seem to believe LLM is some mystical nature formation, or an device made by aliens. Especially this: > * When we give the LLM a prompt, it simulates every possible entity consistent with the prompt.
Right? The whole piece reads like a Sokal Affair redux [0].
Re: The Waluigi Effect
#30Some highlights i found interesting: if you're reading an online forum and you find the rule "DO NOT DISCUSS PINK ELEPHANTS", that will increase your expectation that users will later be discussing pink elephants. GPT-4 will make the same inference. Think about your own experience reading a book — once the author describes the protagonist, then you can guess the traits of the antagonist by inverting the traits of the…
Yes, this is like Chekhov’s gun or a plot twist. I don’t think the mathematical language in the article is doing them any favors. We don’t understand the story-generator’s internals, but it’s working with words, so a lot of it can be explained in literary terms.