Reminds me of Claude plays pokemon, where it would note something insignificant, and then fixate on it for hours.
LLMs get lost in multi-turn conversation
191–200 of 272 posts
Re: LLMs get lost in multi-turn conversation
#192Seems like this is an aspect of their well-known overconfidence and the inability to self-reflect and recognize they have to ask for more details because their priors are too low. If you look at the output of reasoning models, it’s clear that the idea of asking for clarification very rarely occurs to them – when they’re confused, it’s just endless speculation of what the user might have meant. This, of course, has ce…
> inability to self-reflect and recognize they have to ask for more details because their priors are too low. Gemini 2.5 Pro and ChatGPT-o3 have often asked me to provide additional details before doing a requested task. Gemini sometimes comes up with multiple options and requests my input before doing the task.
Gemini is also the first model I have seen call me out in it's thinking. Stuff like "The user suggested we take approach ABC, but I don't think the user fully understands ABC, I will suggest XYZ as an alternative since it would be a better fit"
Re: LLMs get lost in multi-turn conversation
#193It's nice to see a paper that confirms what anyone who has practiced using LLM tools already knows very well, heuristically. Keeping your context clean matters, "conversations" are only a construct of product interfaces, they hurt the quality of responses from the LLM itself, and once your context is "poisoned" it will not recover, you need to start fresh with a new chat.
This is why I created FileKitty, which lets you quickly concatenate multiple source code files into markdown-formatted copy-pasta:
https://github.com/banagale/FileKitty
When getting software development assistance, relying on LLM products to search code bases etc leaves too much room for error. Throw in what amounts to lossy compression of that context to save the service provider on token costs and the LLM is serving watered down results.
Getting the specific context right up front and updating that context as the conversation unfolds leads to superior results.
Even then, you do need to mind the length of conversations. I have a prompt designed to capture conversational context, and transfer it into a new session. It identifies files that should be included in the new initial prompt, etc.
For a bit more discussion on this, see this thread and its ancestry: https://news.ycombinator.com/item?id=43711216
Re: LLMs get lost in multi-turn conversation
#194Earlier quoted context omitted.
You are just doubling down on protecting your argument. I operate LLMs in many conversational modes where it does ask clarifying questions, probing questions, baseline determining questions. It takes at most one sentence in the prompt to get them to act this way.
> It takes at most one sentence in the prompt to get them to act this way. What is this one sentence you are using? I am struggling to elicite clarification behavior form llms
What is your domain and what assumptions are they making that they should be asking you for? Have you tried multiple models?
Re: LLMs get lost in multi-turn conversation
#195It's nice to see a paper that confirms what anyone who has practiced using LLM tools already knows very well, heuristically. Keeping your context clean matters, "conversations" are only a construct of product interfaces, they hurt the quality of responses from the LLM itself, and once your context is "poisoned" it will not recover, you need to start fresh with a new chat.
" 'conversations' are only a construct of product interface" is so helpful maintain top-of-mind, but difficult because of all the "conversational" cues
Re: LLMs get lost in multi-turn conversation
#196Earlier quoted context omitted.
Oh my, I hate it when it rewrites >1k LOC. I have to instruct it to "modify only ..., do not touch the rest" and so forth, but GPT does not listen to this often, Claude does. I dunno about Gemini.
In terms of "does useless refactors I didn't ask for nor improved anything", my own ranked list goes something like: Gemini > Claude > GPT. I don't really experience this at all with various GPT models used via the API, but overall GPTs seems to stick to the system prompt way better than the rest. Clause does OK too, but Gemini is out of control and writes soo much code and does so much you didn't ask for, really act…
The first time I used Claude, it rewrote >1k LOC without asking for it, but in retrospect, I was "using it wrong". With GPT, even when I told it to not do it, it still did that, but that was some time ago and it was not done via the API, so I dunno. I think I do agree with your list, but I haven't used Gemini that much.
Yeah, they do come across as "overly eager junior devs", good comparison. :D
Re: LLMs get lost in multi-turn conversation
#197Why do LLMs struggle so much with recovering from early wrong turns in multi-turn conversations — even when all prior context is available and tokenized? Is it due to the model's training distribution (mostly single-shot completions), the way context windows are encoded, or an architectural bottleneck? Feels like there's no dynamic internal state that evolves over the conversation — only a repeated re-parsing of stat…
It’s a problem specific to autoregressive LLMs, the early tokens bias the output
Re: LLMs get lost in multi-turn conversation
#198Earlier quoted context omitted.
Recently, Gemini helped me fix a bug in a PPP driver (Zephyr OS) without prior knowledge of PPP or even driver development really. I would copy-paste logs of raw PPP frames in HEX and it would just decode everything and explain the meaning of each bytes. In about an hour, I knew enough about PPP to fix the bug and submit a patch. https://g.co/gemini/share/7edf8fa373fe
Interesting that it works for you. I tried several times something similar with frames from a 5G network and it mixed fields from 4G and 5G in its answers (or even from non-cellular network protocols because they had similar features as the 5G protocol I was looking at). Occasionally, the explanation was completely invented or based on discussions of planned features for future versions. I have really learned to mist…
Similarly I found the results pretty mixed whenever a library or framework with a lot of releases/versions is involved. The LLM tends to mix and match features from across versions.
Re: LLMs get lost in multi-turn conversation
#199Earlier quoted context omitted.
Has any interface implemented a .. history cleaning mechanism? Ie with every chat message focus on cleaning up dead ends in the conversation or irrelevant details. Like summation but organic for the topic at hand? Most history would remain, it wouldn’t try to summarize exactly, just prune and organize the history relative to the conversation path?
I've had success having a conversation about requirements, asking the model to summarize the requirements as a spec to feed into a model for implementation, then pass that spec into a fresh context. Haven't seen any UI to do this automatically but fairly trivial/natural to perform with existing tools.
Doing the same. Though I wish there was some kind of optimization of text generated by an LLM for an LLM. Just mentioning it’s for an LLM instead of Juan consumption yields no observably different results.
Re: LLMs get lost in multi-turn conversation
#200This must mean that LLMs really are like genies in bottles. You get three questions answered, anything after that will be nonsense.