Claude mixes up who said what • Hacker News Board

Re: Claude mixes up who said what

#121

Post by spprashant » Thu, Apr 09, 2026, 12:05 PM UTC

hacker_homie wrote: post #64

Latty wrote: post #5

Everything to do with LLM prompts reminds me of people doing regexes to try and sanitise input against SQL injections a few decades ago, just papering over the flaw but without any guarantees. It's weird seeing people just adding a few more "REALLY REALLY REALLY REALLY DON'T DO THAT" to the prompt and hoping, to me it's just an unacceptable risk, and any system using these needs to treat the entire LLM as untrusted t…

I have been saying this for a while, the issue is there's no good way to do LLM structured queries yet. There was an attempt to make a separate system prompt buffer, but it didn't work out and people want longer general contexts but I suspect we will end up back at something like this soon.

The problem is once you accept that it is needed, you can no longer push AI as general intelligence that has superior understanding of the language we speak.

A structured LLM query is a programming language and then you have to accept you need software engineers for sufficiently complex structured queries. This goes against everything the technocrats have been saying.

Re: Claude mixes up who said what

#122

Post by bravetraveler » Thu, Apr 09, 2026, 12:06 PM UTC

ares623 wrote: post #76

nathell wrote: post #68

I’ve hit this! In my otherwise wildly successful attempt to translate a Haskell codebase to Clojure [0], Claude at one point asks: [Claude:] Shall I commit this progress? [some details about what has been accomplished follow] Then several background commands finish (by timeout or completing); Claude Code sees this as my input, thinks I haven’t replied to its question, so it answers itself in my name: [Claude:] Yes, g…

I wonder if tools like Terraform should remove the message "Run terraform apply plan.out next" that it prints after every `terraform plan` is run.

I don't think so, feels like the wrong side is getting attention. Degrading the experience for humans (in one tool) because the bots are prone to injection (from any tool). Terraform is used outside of agents; somebody surely finds the reminder helpful.

If terraform were to abide, I'd hope at the very least it would check if in a pipeline or under an agent. This should be obvious from file descriptors/env.

What about the next thing that might make a suggestion relying on our discretion? Patch it for agent safety?

Re: Claude mixes up who said what

#123

Post by fathermarz » Thu, Apr 09, 2026, 12:06 PM UTC

I have seen this when approaching ~30% context window remaining.

There was a big bug in the Voice MCP I was using that it would just talk to itself back and forth too.

Re: Claude mixes up who said what

#124

Post by timcobb » Thu, Apr 09, 2026, 12:08 PM UTC

HeavyStorm wrote: post #102

Latty wrote: post #5

Everything to do with LLM prompts reminds me of people doing regexes to try and sanitise input against SQL injections a few decades ago, just papering over the flaw but without any guarantees. It's weird seeing people just adding a few more "REALLY REALLY REALLY REALLY DON'T DO THAT" to the prompt and hoping, to me it's just an unacceptable risk, and any system using these needs to treat the entire LLM as untrusted t…

The real issue is expecting an LLM to be deterministic when it's not.

they are deterministic, open a dev console and run the same prompt two times w/ temperature = 0

Re: Claude mixes up who said what

#125

Post by mynameisvlad » Thu, Apr 09, 2026, 12:10 PM UTC

I wouldn't exactly call three instances "widespread". Nor would the third such instance prompt me to think so.

"Widespread" would be if every second comment on this post was complaining about it.

Re: Claude mixes up who said what

#126

Post by [deleted] » Thu, Apr 09, 2026, 12:10 PM UTC

simianparrot wrote: post #79

satvikpendem wrote: post #72

Earlier quoted context omitted.

That is "fundamentally" not true, you can use a preset seed and temperature and get a deterministic output.

A single byte change in the input changes the output. The sentence "Please do this for me" and "Please, do this for me" can lead to completely distinct output. Given this, you can't treat it as deterministic even with temp 0 and fixed seed and no memory.

[deleted]

Re: Claude mixes up who said what

#127

Post by program_whiz » Thu, Apr 09, 2026, 12:11 PM UTC

satvikpendem wrote: post #100

HPsquared wrote: post #77

Earlier quoted context omitted.

I'll grant that you can guarantee the length of the output and, being a computer program, it's possible (though not always in practice) to rerun and get the same result each time, but that's not guaranteeing anything about said output.

What do you want to guarantee about the output, that it follows a given structure? Unless you map out all inputs and outputs, no it's not possible, but to say that it is a fundamental property of LLMs to be non deterministic is false, which is what I was inferring you meant, perhaps that was not what you implied.

Yeah I think there are two definitions of determinism people are using which is causing confusion. In a strict sense, LLMs can be deterministic meaning same input can generate same output (or as close as desired to same output). However, I think what people mean is that for slight changes to the input, it can behave in unpredictable ways (e.g. its output is not easily predicted by the user based on input alone). People mean "I told it don't do X, then it did X", which indicates a kind of randomness or non-determinism, the output isn't strictly constrained by the input in the way a reasonable person would expect.

Re: Claude mixes up who said what

#128

Post by silon42 » Thu, Apr 09, 2026, 12:12 PM UTC

HPsquared wrote: post #77

satvikpendem wrote: post #72

Earlier quoted context omitted.

That is "fundamentally" not true, you can use a preset seed and temperature and get a deterministic output.

I'll grant that you can guarantee the length of the output and, being a computer program, it's possible (though not always in practice) to rerun and get the same result each time, but that's not guaranteeing anything about said output.

You can guarantee what you have test coverage for :)

Re: Claude mixes up who said what

#129

Post by zbentley » Thu, Apr 09, 2026, 12:13 PM UTC

satvikpendem wrote: post #72

HPsquared wrote: post #69

Earlier quoted context omitted.

Fundamentally there's no way to deterministically guarantee anything about the output.

That is "fundamentally" not true, you can use a preset seed and temperature and get a deterministic output.

Practically, the performance loss of making it truly repeatable (which takes parallelism reduction or coordination overhead, not just temperature and randomizer control) is unacceptable to most people.

Re: Claude mixes up who said what

#130

Post by TeMPOraL » Thu, Apr 09, 2026, 12:15 PM UTC

hacker_homie wrote: post #64

Latty wrote: post #5

Everything to do with LLM prompts reminds me of people doing regexes to try and sanitise input against SQL injections a few decades ago, just papering over the flaw but without any guarantees. It's weird seeing people just adding a few more "REALLY REALLY REALLY REALLY DON'T DO THAT" to the prompt and hoping, to me it's just an unacceptable risk, and any system using these needs to treat the entire LLM as untrusted t…

I have been saying this for a while, the issue is there's no good way to do LLM structured queries yet. There was an attempt to make a separate system prompt buffer, but it didn't work out and people want longer general contexts but I suspect we will end up back at something like this soon.

I've been saying this for a while, the issue is that what you're asking for is not possible, period. Prompt injection isn't like SQL injection, it's like social engineering - you can't eliminate it without also destroying the very capabilities you're using a general-purpose system for in the first place, whether that's an LLM or a human. It's not a bug, it's the feature.