The Waluigi Effect • Hacker News Board

Re: The Waluigi Effect

#41

Post by notpachet » Mon, Mar 06, 2023, 6:02 PM UTC

> These claims have been heavily filtered by OpenAI, and most jailbreaking attempts will fail to elict these claims. I'm talking, of course, about cfrhqb-fpvragvsvp enpvny VD fgngvfgvpf, which OpenAI has justifiably attempted to filter.

I was curious what the scrambled text "cfrhqb-fpvragvsvp enpvny VD fgngvfgvpf" contained. It's using a simple substitution cipher. Rotating each character forward 13 positions through the alphabet (c -> p, f -> s, etc) yields "pseudo-scientific racial IQ statistics".

https://en.wikipedia.org/wiki/Caesar_cipher

Re: The Waluigi Effect

#42

Post by CelticBard » Mon, Mar 06, 2023, 6:04 PM UTC

The guy who wrote the article is very clearly neither a computer scientist nor a mathematician nor even a machine learning engineer and it very very clearly shows.

Also where did they all that info on GPT-4? Pure speculation with zero theoretical basis. But then again that’s the sort of stuff you expect from lesswrong anyway

Re: The Waluigi Effect

#43

Post by blurbleblurble » Mon, Mar 06, 2023, 6:07 PM UTC

MontyCarloHall wrote: post #13

cs702 wrote: post #9

Great read. Highly recommended. Let me attempt to summarize it with less technical, more accessible language: The hypothesis is that LLMs learn to simulate text-generating entities drawn from a latent space of text-generating entities , such that the output of an LLM is produced by a superposition of such simulated entities. When we give the LLM a prompt, it simulates every possible text-generating entity consistent…

>The output of an LLM is produced by a superposition of simulated entities. When we give the LLM a prompt, it simulates every possible entity consistent with the prompt. There is absolutely no theoretical justification for this assertion that LLMs somehow have some emergent quantum mechanical behavior, metaphorical or otherwise.

You're giving "quantum" a lot of categorical definition but "quantum" is just as much a set of tools for modeling intractably probabilistic systems as it is for objectively describing microscopic realms.

And anyhow there's _plenty_ of theoretical justification for modeling things like this with various tools from quantum theory:

https://philpapers.org/rec/BUSQMO-2 https://link.springer.com/book/10.1007/978-3-642-05101-2

Re: The Waluigi Effect

#44

Post by jrochkind1 » Mon, Mar 06, 2023, 6:08 PM UTC

atomicnumber3 wrote: post #33

BugsJustFindMe wrote: post #31

> If you ask GPT- ∞ "what's brown and sticky?", then it will reply "a stick", even though a stick isn't actually sticky. Isn't it though?

Also, isn't this a really common joke? I assume ChatGPT will have absorbed some amount of a sense of humor from its trawls of the internet.

I mean, that's exactly what the OP assumes too. Well, not that it's absorbed a "sense of humor", but that it's going to reply with this because it's a very common joke so statistically likely text.

Re: The Waluigi Effect

#45

Post by jxy » Mon, Mar 06, 2023, 6:10 PM UTC

A hypothesis I have is that it is much more difficult to keep in line with the good alignment than to do evil. In the limited context window of an LLM, one wrong move would make the model evil, no matter how many good tokens it generates.

Setting aside the difference between Human intelligence and LLM, we can tentatively attribute the mostly good human behavior to a life time of context length, within which we trained ourselves to do good, while the RLHF for a limited context length LLM lack such continuous reinforcement within a big context.

Re: The Waluigi Effect

#46

Post by CuriouslyC » Mon, Mar 06, 2023, 6:14 PM UTC

cs702 wrote: post #9

Great read. Highly recommended. Let me attempt to summarize it with less technical, more accessible language: The hypothesis is that LLMs learn to simulate text-generating entities drawn from a latent space of text-generating entities , such that the output of an LLM is produced by a superposition of such simulated entities. When we give the LLM a prompt, it simulates every possible text-generating entity consistent…

The hole in that argument is that we can annotate token sequences with a "good/bad" score, train a model on annotated token sequences, then add "high good score" as a condition during inference.

Re: The Waluigi Effect

#47

Post by CuriouslyC » Mon, Mar 06, 2023, 6:16 PM UTC

captainmuon wrote: post #39

I would question the assumption that there is a simulacrum of anything in a LLM, not even implicit. Any simulacrum, identity, self-consistency etc. is a projection of the "reader", i.e. user. (I guess it is an interesting philosophical question whether a convincing presentation of a simulation of a mind is a mind, or at least an acceptable simulation. One meta level higher as the turing test, so to speak. If so, I'm…

Do you think an ant have a subjective experience? If not, why? If so, why wouldn't a computer, or parts of a computer?

Re: The Waluigi Effect

#48

Post by attemptone » Mon, Mar 06, 2023, 6:21 PM UTC

mitthrowaway2 wrote: post #34

raincole wrote: post #20

Earlier quoted context omitted.

Sometimes I really can't tell if these people are serious or not. They seem to believe LLM is some mystical nature formation, or an device made by aliens. Especially this: > * When we give the LLM a prompt, it simulates every possible entity consistent with the prompt.

Would you object if the sentence read "it approximates simulating every possible entity consistent with the prompt"?

Not OP, but I see the also in problem with 'every possible entity'.

If you formulate it like that the prompt is decoupled from the LLM capabilities and can be anything. And if you restrict the prompt to cover only what the LLM understands the sentence becomes trivial.

Train a LLM with ASCII and try to get it to simulate anything that is outside of that (ancient sumerian script for example). If you only input ASCII it can generate every possible output in ASCII, most with very low probability but still.

After writing this, I'm not even sure what 'simulating' means in this context.

Re: The Waluigi Effect

#49

Post by GlickWick » Mon, Mar 06, 2023, 6:21 PM UTC

MontyCarloHall wrote: post #25

jointpdf wrote: post #23

Earlier quoted context omitted.

Superposition just means “linear combination” in this context. Basically, a weighted mixture of “simulated entities” (or possible responses). https://en.m.wikipedia.org/wiki/Superposition_principle

The author liberally alludes to “superposition collapse,” which implies that they’re referring to its quantum mechanical meaning.

Would you say the commonly-used random level generation algorithm in gamedev, Wave Function Collapse, implies it’s using quantum mechanics? Most people would disagree with you, I suspect.

Re: The Waluigi Effect

#50

Post by ineptech » Mon, Mar 06, 2023, 6:22 PM UTC

raincole wrote: post #20

cs702 wrote: post #9

Great read. Highly recommended. Let me attempt to summarize it with less technical, more accessible language: The hypothesis is that LLMs learn to simulate text-generating entities drawn from a latent space of text-generating entities , such that the output of an LLM is produced by a superposition of such simulated entities. When we give the LLM a prompt, it simulates every possible text-generating entity consistent…

Sometimes I really can't tell if these people are serious or not. They seem to believe LLM is some mystical nature formation, or an device made by aliens. Especially this: > * When we give the LLM a prompt, it simulates every possible entity consistent with the prompt.

I think you're taking it way too literally. The author is proposing a model that [allegedly] does a good job of explaining why we see unexpected behavior, not a description of what they think is happening. I think charitable reading would require you to mentally insert a lot of qualifiers, e.g. "When we give the LLM a prompt, it [behaves like something that] simulates every possible entity consistent with the prompt." That's certainly how I read it, anyway.

edit to add: this is similar to how people discussing evolutionary biology will often use "evolution wants to..." as shorthand for something like "evolution, which obviously cannot want things due to being a process and not an entity, nevertheless can be accurately modeled as an entity that wants to...". Someone will invariably come along in the comments and say, "Nonsense, how can evolution 'want' anything? You must have failed Bio 101!"