Claude mixes up who said what • Hacker News Board

Re: Claude mixes up who said what

#21

Post by cyanydeez » Thu, Apr 09, 2026, 10:18 AM UTC

human memories dont exist as fundamental entities. every time you rember something, your brain reconstructs the experience in "realtime". that reconstruction is easily influence by the current experience, which is why eue witness accounts in police records are often highly biased by questioning and learning new facts.

LLMs are not experience engines, but the tokens might be thought of as subatomic units of experience and when you shove your half drawn eye witness prompt into them, they recreate like a memory, that output.

so, because theyre not a conscious, they have no self, and a pseudo self like is all theyre given.

lastly, like memories, the more intricate the memory, the more detailed, the more likely those details go from embellished to straight up fiction. so too do LLMs with longer context start swallowing up the and missing the and anyone whose raw dogged html parsing knows bad things happen when you forget closing tags. if there was a block in there, congrats, the LLM now thinks its instructions are divine right, because its instructions are user simulcra. it is poisoned at that point and no good will come.

Re: Claude mixes up who said what

#22

Post by sixhobbits » Thu, Apr 09, 2026, 10:19 AM UTC

xg15 wrote: post #13

> This class of bug seems to be in the harness, not in the model itself. It’s somehow labelling internal reasoning messages as coming from the user, which is why the model is so confident that “No, you said that.” Are we sure about this? Accidentally mis-routing a message is one thing, but those messages also distinctly "sound" like user messages, and not something you'd read in a reasoning trace. I'd like to know if…

author here - yeah maybe 'reasoning' is the incorrect term here, I just mean the dialogue that claude generates for itself between turns before producing the output that it gives back to the user

Re: Claude mixes up who said what

#23

Post by perching_aix » Thu, Apr 09, 2026, 10:19 AM UTC

Latty wrote: post #5

Everything to do with LLM prompts reminds me of people doing regexes to try and sanitise input against SQL injections a few decades ago, just papering over the flaw but without any guarantees. It's weird seeing people just adding a few more "REALLY REALLY REALLY REALLY DON'T DO THAT" to the prompt and hoping, to me it's just an unacceptable risk, and any system using these needs to treat the entire LLM as untrusted t…

It's less about security in my view, because as you say, you'd want to ensure safety using proper sandboxing and access controls instead.

It hinders the effectiveness of the model. Or at least I'm pretty sure it getting high on its own supply (in this specific unintended way) is not doing it any favors, even ignoring security.

Re: Claude mixes up who said what

#24

Post by supernes » Thu, Apr 09, 2026, 10:20 AM UTC

> after using it for months you get a ‘feel’ for what kind of mistakes it makes

Sure, go ahead and bet your entire operation on your intuition of how a non-deterministic, constantly changing black box of software "behaves". Don't see how that could backfire.

Re: Claude mixes up who said what

#25

Post by cyanydeez » Thu, Apr 09, 2026, 10:20 AM UTC

__alexs wrote: post #8

Why are tokens not coloured? Would there just be too many params if we double the token count so the model could always tell input tokens from output tokens?

you would have to train it three times for two colors.

each by itself, they with both interactions.

2!

Re: Claude mixes up who said what

#26

Post by loveparade » Thu, Apr 09, 2026, 10:21 AM UTC

xg15 wrote: post #13

> This class of bug seems to be in the harness, not in the model itself. It’s somehow labelling internal reasoning messages as coming from the user, which is why the model is so confident that “No, you said that.” Are we sure about this? Accidentally mis-routing a message is one thing, but those messages also distinctly "sound" like user messages, and not something you'd read in a reasoning trace. I'd like to know if…

Yeah, it looks like a model issue to me. If the harness had a (semi-)deterministic bug and the model was robust to such mix-ups we'd see this behavior much more frequently. It looks like the model just starts getting confused depending on what's in the context, speakers are just tokens after all and handled in the same probabilistic way as all other tokens.

Re: Claude mixes up who said what

#27

Post by okanat » Thu, Apr 09, 2026, 10:22 AM UTC

Congrats on discovering what "thinking" models do internally. That's how they work, they generate "thinking" lines to feed back on themselves on top of your prompt. There is no way of separating it.

Re: Claude mixes up who said what

#28

Post by hydroreadsstuff » Thu, Apr 09, 2026, 10:22 AM UTC

Latty wrote: post #5

Everything to do with LLM prompts reminds me of people doing regexes to try and sanitise input against SQL injections a few decades ago, just papering over the flaw but without any guarantees. It's weird seeing people just adding a few more "REALLY REALLY REALLY REALLY DON'T DO THAT" to the prompt and hoping, to me it's just an unacceptable risk, and any system using these needs to treat the entire LLM as untrusted t…

I like the Dark Souls model for user input - messages. https://darksouls.fandom.com/wiki/Messages Premeditated words and sentence structure. With that there is no need for moderation or anti-abuse mechanics. Not saying this is 100% applicable here. But for their use case it's a good solution.

Re: Claude mixes up who said what

#29

Post by vanviegen » Thu, Apr 09, 2026, 10:22 AM UTC

supernes wrote: post #24

> after using it for months you get a ‘feel’ for what kind of mistakes it makes Sure, go ahead and bet your entire operation on your intuition of how a non-deterministic, constantly changing black box of software "behaves". Don't see how that could backfire.

> bet your entire operation

What straw man is doing that?

Re: Claude mixes up who said what

#30

Post by __alexs » Thu, Apr 09, 2026, 10:23 AM UTC

cyanydeez wrote: post #25

__alexs wrote: post #8

Why are tokens not coloured? Would there just be too many params if we double the token count so the model could always tell input tokens from output tokens?

you would have to train it three times for two colors. each by itself, they with both interactions. 2!

The models are already massively over trained. Perhaps you could do something like initialise the 2 new token sets based on the shared data, then use existing chat logs to train it to understand the difference between input and output content? That's only a single extra phase.