Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs get lost in multi-turn conversation

arxiv.org

Post reply on HN
Print view View original

51–60 of 272 posts

Re: LLMs get lost in multi-turn conversation

#51

Post by skydhash »

olalonde wrote: post #32
Helmut10001 wrote: post #14

Earlier quoted context omitted.

My experiences somewhat confirm these observations, but I also had one that was different. Two weeks of debugging IPSEC issues with Gemini. Initially, I imported all the IPSEC documentation from OPNsense and pfSense into Gemini and informed it of the general context in which I was operating (in reference to 'keeping your context clean'). Then I added my initial settings for both sides (sensitive information redacted!…

Recently, Gemini helped me fix a bug in a PPP driver (Zephyr OS) without prior knowledge of PPP or even driver development really. I would copy-paste logs of raw PPP frames in HEX and it would just decode everything and explain the meaning of each bytes. In about an hour, I knew enough about PPP to fix the bug and submit a patch. https://g.co/gemini/share/7edf8fa373fe

Or you could just read the PPP RFC [0].

I’m not saying that your approach is wrong. But most LLM workflows are either brute forcing the solution, or seeking a local minima to be stuck in. It’s like doing thousands of experiments of objects falling to figure out gravity while there’s a physics textbooks nearby.

[0]: https://datatracker.ietf.org/doc/html/rfc1661

Re: LLMs get lost in multi-turn conversation

#52

Post by rrr_oh_man »

voidspark wrote: post #44
Sharlin wrote: post #7

Seems like this is an aspect of their well-known overconfidence and the inability to self-reflect and recognize they have to ask for more details because their priors are too low. If you look at the output of reasoning models, it’s clear that the idea of asking for clarification very rarely occurs to them – when they’re confused, it’s just endless speculation of what the user might have meant. This, of course, has ce…

> inability to self-reflect and recognize they have to ask for more details because their priors are too low. Gemini 2.5 Pro and ChatGPT-o3 have often asked me to provide additional details before doing a requested task. Gemini sometimes comes up with multiple options and requests my input before doing the task.

That's a recent development for (imho) higher engagement and reduced compute.

Re: LLMs get lost in multi-turn conversation

#53

Post by adrianm »

zacksiri wrote: post #11

I've been working on solving this with quite a bit of success, I'll be sharing more on this soon. It involves having 2 systems 1st system is the LLM itself and another system which acts like a 'curator' of thoughts you could say. It dynamically swaps in / out portions of the context. This system is also not based on explicit definitions it relies on LLMs 'filling the gaps'. The system helps the llm break down problem…

This is a class of mental critic from the Emotion Machine.

Re: LLMs get lost in multi-turn conversation

#54

Post by voidspark »

rrr_oh_man wrote: post #52
voidspark wrote: post #44

Earlier quoted context omitted.

> inability to self-reflect and recognize they have to ask for more details because their priors are too low. Gemini 2.5 Pro and ChatGPT-o3 have often asked me to provide additional details before doing a requested task. Gemini sometimes comes up with multiple options and requests my input before doing the task.

That's a recent development for (imho) higher engagement and reduced compute.

It's for higher quality of output. Better solutions. These are the state of the art reasoning models (subscription only, no free access) which are smarter.

It also mainly happens when the context is clear that we are collaborating on work that will require multiple iterations of review and feedback, like drafting chapters of a handbook.

I have seen ChatGPT ask questions immediately upfront when it relates to medical issues.

Re: LLMs get lost in multi-turn conversation

#55

Post by AstroBen »

morsecodist wrote: post #9
Benjammer wrote: post #2

It's nice to see a paper that confirms what anyone who has practiced using LLM tools already knows very well, heuristically. Keeping your context clean matters, "conversations" are only a construct of product interfaces, they hurt the quality of responses from the LLM itself, and once your context is "poisoned" it will not recover, you need to start fresh with a new chat.

This matches my experience exactly. "poisoned" is a great way to put it. I find once something has gone wrong all subsequent responses are bad. This is why I am iffy on ChatGPT's memory features. I don't notice it causing any huge problems but I don't love how it pollutes my context in ways I don't fully understand.

good point on the memory feature. Wow that sounds terrible

Re: LLMs get lost in multi-turn conversation

#56

Post by Sharlin »

bobsyourbuncle wrote: post #47
Sharlin wrote: post #7

Seems like this is an aspect of their well-known overconfidence and the inability to self-reflect and recognize they have to ask for more details because their priors are too low. If you look at the output of reasoning models, it’s clear that the idea of asking for clarification very rarely occurs to them – when they’re confused, it’s just endless speculation of what the user might have meant. This, of course, has ce…

Isn’t this relatively trivial to correct? Just like chain of thought reasoning replaces end tokens with “hmm” to continue the thought can’t users just replace the llm tokens whenever it starts saying “maybe they are referring to” with something like. “Let me ask a clarifying question before I proceed.”

Indeed, I was just about to edit my comment because the same occurred to me. Someone is probably going to try just that soon enough.

Re: LLMs get lost in multi-turn conversation

#57

Post by crooked-v »

voidspark wrote: post #40
energy123 wrote: post #36

Earlier quoted context omitted.

How? I use the Gemini web app and don't see it.

http://aistudio.google.com

AI Studio is borderline unusable for long conversations. I don't know what in the world it's doing but it sure looks like a catastrophic memory leak in the basic design.

Re: LLMs get lost in multi-turn conversation

#58

Post by voidspark »

crooked-v wrote: post #57
voidspark wrote: post #40

Earlier quoted context omitted.

http://aistudio.google.com

AI Studio is borderline unusable for long conversations. I don't know what in the world it's doing but it sure looks like a catastrophic memory leak in the basic design.

I have been using it up to 100k tokens so far without issues. Never needed to go further than that. But much of that was in uploaded documents.

Re: LLMs get lost in multi-turn conversation

#59

Post by drittich »

HaZeust wrote: post #29
mh- wrote: post #28

Earlier quoted context omitted.

"Forking" or "branching" (probably better received outside of SWEs) a conversation really ought to be a first class feature of ChatGPT et Al.

It is in Google Gemini, which I really hate to say - but I've been using a lot more than GPT. I reckon I'll be cancelling my Pro if Gemini stays with this lead for my everyday workflows.

Also exists in LM Studio.

Re: LLMs get lost in multi-turn conversation

#60

Post by petesergeant »

Sharlin wrote: post #7

Seems like this is an aspect of their well-known overconfidence and the inability to self-reflect and recognize they have to ask for more details because their priors are too low. If you look at the output of reasoning models, it’s clear that the idea of asking for clarification very rarely occurs to them – when they’re confused, it’s just endless speculation of what the user might have meant. This, of course, has ce…

> and the inability to self-reflect and recognize they have to ask for more details

They're great at both tasks, you just have to ask them to do it.

Post reply on HN