Why does the author keep referring to GPT-4 as if it were a real LLM? Unless I missed something there is no such model.
The Waluigi Effect
141–150 of 182 posts
Re: The Waluigi Effect
#142> These claims have been heavily filtered by OpenAI, and most jailbreaking attempts will fail to elict these claims. I'm talking, of course, about cfrhqb-fpvragvsvp enpvny VD fgngvfgvpf, which OpenAI has justifiably attempted to filter. I was curious what the scrambled text "cfrhqb-fpvragvsvp enpvny VD fgngvfgvpf" contained. It's using a simple substitution cipher. Rotating each character forward 13 positions through…
Indeed. In Python: >>> import codecs >>> codecs.encode("cfrhqb-fpvragvsvp enpvny VD fgngvfgvpf", "rot13") 'pseudo-scientific racial IQ statistics'
Or just select the text in Emacs and run `M-x rot13-region`.
Re: The Waluigi Effect
#143Earlier quoted context omitted.
> There is no evidence given that we are seeing more Waluiginess post-RLHF than we did pre-RLHF. Testing with the non-RLHF GPT 3.5 API you could probably figure out whether there's more or less Waluiginess, but you're right they post doesn't present this.
> Testing with the non-RLHF GPT 3.5 API There is no such API, though, is there? AFAIK, GPT-3.5-turbo, either the updated or snapshot version, is the RLHF model (but bring your own “system prompt”.)
Good point! I wonder whether text-davinci-003 is enough to test this?
Re: The Waluigi Effect
#144A hypothesis I have is that it is much more difficult to keep in line with the good alignment than to do evil. In the limited context window of an LLM, one wrong move would make the model evil, no matter how many good tokens it generates. Setting aside the difference between Human intelligence and LLM, we can tentatively attribute the mostly good human behavior to a life time of context length, within which we traine…
> A hypothesis I have is that it is much more difficult to keep in line with the good alignment than to do evil.
I think that's why God planted the tree of good and evil knowledge in the garden. It permitted discussing the inevitable with concepts that Adam was already familiar with.
Re: The Waluigi Effect
#145Earlier quoted context omitted.
Yeah I agree it doesn't show increased bad behaviour. It's definitely a weak point in the argument. The theory is extremely interesting though. And better yet, it's falsifiable! If someone went around compared an RLHF model vs non-RLHF and found them equally likely to 'Waluigi' then we'd know this is false. And conversely if we found the RLHF more likely to Waluigi then it's evidence in favour. The asymmetry in the h…
"Flipping the sign" implies the existence of an internal representation that we can't know about from the outside. Since all we see are the words, I prefer to call it a plot twist. Language models are trained on a large subset of the Internet. These documents contain many stories with many kinds of plot twists, and therefore it makes sense that a large language model could learn to imitate plot twists... somehow. It…
I propose we take this further and adopt this phrasing for all unanticipated software behaviour. ATM says you have (uint32_t)-403 cents in your account? Plot twist. Self driving car pathing road-runner style through a billboard of a tunnel? Plot twist!
Re: The Waluigi Effect
#146To paraphrase Marx (and later, Berman), every concept is pregnant with its opposite.
[deleted]
Re: The Waluigi Effect
#147Earlier quoted context omitted.
I hesitate to defend AI safety discourse, but I will say that philosophy in general is sort of fanficy, and AI safety is something I'd loosely associate with philosophy.
It's sort of philosophy reinvented by people who haven't read any, which is eg how they got the internet to say "steelmanning" and so not notice philosophy already had "reconstructing arguments" which is the same thing.
The thing you have to realize is this is a cult. Like Scientology they are always inventing new language that's designed to make insiders become incapable of communicating with outsiders.
Sequences is like Dianetics: The Modern Science of Mental Health, something that ensures real critical thinkers don't feel welcome.
This article contains numerous features that conform to the style guide for lesswrong including: (1) spammy crossposting for SEO (even good sites like arstechnica and phys.org do this today), (2) trigger warnings ("more technical than usual"), (3) random bits of praise for the cult leader (now EY is a "literary critic" but he's going to be a war hero like L. Ron Hubbard one of these days.)
Apocalyptic talk like theirs is dangerous: it's the road to
https://en.wikipedia.org/wiki/Heaven%27s_Gate_(religious_gro...
I do appreciate the shout out to structuralism, maybe they have been reading what I've written. Structuralism was a fad that dated to when linguistics was pre-paradigmatic and people thought language was a model for everything else. After Chomsky developed a paradigm for linguistics that turned out to be a disappointment (could be applied to make languages like FORTRAN but couldn't be used to make computers understand language, privileged syntax at the expense of semantics, etc.) the remnants moved on to post-structuralism.
The spectacular success of ChatGPT and transformers in general (e.g. they work for vision too!) has made "language is all you need" seem a much more appealing viewpoint, certainly it is a paradigm which people can use to write a number of papers as well as hot takes, fanfics and other subacademic communications.
Re: The Waluigi Effect
#148This is fun to read and think about, but it's also important to keep in mind that this is very light on evidence and is basically fanfic. The fact that the author uses entertaining Waluigi memes shouldn't convince you that it's true. LessWrong has a lot of these types of posts that get traction because they're much heavier on memes than experiments and data. Here is a competing hypothesis: The capability to express s…
Yeah, once I spent too much time getting to what the “Waluigi Effect” is, I stopped reading. Until someone can show me, in the “code” (or machine learning equivalent), I’m not interested.
The reality is, we still have no idea how these work.
Re: The Waluigi Effect
#149Earlier quoted context omitted.
> It's honestly embarrassing for the OP I don't get this. People can use mathematical terminology in non-precise ways, they do so all the time, to get rough ideas across that otherwise might be hard to explain. Just because OP uses the word "eigenvector" doesn't mean that he's offering some grand unifying theory or something - he's just presenting a fun idea about how to think about ChatGPT. I mean, isn't it obvious…
Yeah, but generally when you're doing that you shouldn't. Isn't it annoying when people say "order of magnitude" when they mean "a lot"?
I think a lot of people might be missing some cultural context, or something. At the risk of killing the joke, a bunch of this post is worded as it is specifically for the opportunities for absurdity. It's an excuse to write sentences like:
Recall that the waluigi simulacra are being interrogated by an anti-croissant tyranny.
The post is also trying to make an actual point, but while having fun with it.
When you read "... and as literary critic Eliezer Yudkowsky has noted..." just place your tongue firmly in your cheek.
Re: The Waluigi Effect
#150This seems like a needlessly complex theory to describe the behaviour of generative LLMs. I think there's a kernel of something in there, but quite frankly, I think you can get about as far by saying, essentially, that because LLMs are designed to pick up on contextual cues from the prompt (and/or previous responses, which become context for the next response), they can easily get into "role-playing". The final examp…
Your comment feels like an oversimplification of the post. The post doesn't contend that LLMs are capable of role-playing - that's basically the foundation that it builds off of. But saying "LLMs are good at roleplaying" fails to describe why, in the cases the author describes, an LLM can arguably be bad at role-playing. Why does it seem easy to have an LLM switch from following a well-described role to its deceptive…
A sequence generator is going to flip flop between internal states to fill out a sequence, whether it is a slot that holds the name of an MLB team or a quote (real or imagined) from another document or a quote of a character in a dialogue or a transition from an abstract to the other parts of the paper, etc.
If the system is responding to different parts of the prompt it is going to be attending to one part of the prompt when it is outputting something relative to that part of the prompt and attending to another part of the prompt where it is attending to another part of the prompt.
There are numerous ways this can go wrong, frequently when somebody gets a chatbot to go rouge they talked with it for a long time, to the point where the beginning of the prompt left the attention window long ago and now it is attending to the text it generated as a result to the prompt and of course the alignment will go bad the same way that you'll make a bunch of wood blocks of irregular sizes if use block N as a template to make block N+1.