Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs get lost in multi-turn conversation

arxiv.org

Post reply on HN
Print view View original

101–110 of 272 posts

Re: LLMs get lost in multi-turn conversation

#101

Post by veunes »

jsemrau wrote: post #76

That's no surprise. When I was working on game theory and agent reasoning I reached the same conclusion a year ago. My conclusion was that context needs to be managed well for the LLMs to manage accuracy in replies. Also, it helps to have a planning process ("graph reasoning") before task execution because it guardrails the models thought process. This also introduces a discussion on general use vs workflow agent imp…

It's probably why workflow agents feel more reliable: they're built around structure, not just raw prediction

Re: LLMs get lost in multi-turn conversation

#102

Post by M4v3R »

CobrastanJorji wrote: post #62
Benjammer wrote: post #2

It's nice to see a paper that confirms what anyone who has practiced using LLM tools already knows very well, heuristically. Keeping your context clean matters, "conversations" are only a construct of product interfaces, they hurt the quality of responses from the LLM itself, and once your context is "poisoned" it will not recover, you need to start fresh with a new chat.

An interesting little example of this problem is initial prompting, which is effectively just a permanent, hidden context that can't be cleared. On Twitter right now, the "Grok" bot has recently begun frequently mentioning "White Genocide," which is, y'know, odd. This is almost certainly because someone recently adjusted its prompt to tell it what its views on white genocide are meant to be, which for a perfect chatb…

> This is almost certainly because someone recently adjusted its prompt to tell it what its views on white genocide are

Do you have any source on this? System prompts get leaked/extracted all the time so imagine someone would notice this

Edit: just realized you’re talking about the Grok bot, not Grok the LLM available on X or grok.com. With the bot it’s probably harder to extract its exact instructions since it only replies via tweets. For reference here’s the current Grok the LLM system prompt: https://github.com/asgeirtj/system_prompts_leaks/blob/main/g...

Re: LLMs get lost in multi-turn conversation

#103

Post by simianwords »

root_axis wrote: post #67
Sharlin wrote: post #20

Earlier quoted context omitted.

This is a tired semantic argument that does not bring any insight into the discussion. A token-predictor could still be trained to predict the tokens “I’m not sure what you mean because of points x, y, and z; could you elaborate?”

It could be trained to say that, but it's not exactly clear how you would reinforce the absence of certain training data in order to emit that response accurately, rather than just based on embedding proximity.

Why does it seem so hard to make training data for this? You can cook up a few thousands of training data and do an RLHF.

Re: LLMs get lost in multi-turn conversation

#104

Post by jsemrau »

veunes wrote: post #101
jsemrau wrote: post #76

That's no surprise. When I was working on game theory and agent reasoning I reached the same conclusion a year ago. My conclusion was that context needs to be managed well for the LLMs to manage accuracy in replies. Also, it helps to have a planning process ("graph reasoning") before task execution because it guardrails the models thought process. This also introduces a discussion on general use vs workflow agent imp…

It's probably why workflow agents feel more reliable: they're built around structure, not just raw prediction

Also you have more control points. It's not just a brain a vat.

Re: LLMs get lost in multi-turn conversation

#105

Post by simianwords »

Terr_ wrote: post #22
Sharlin wrote: post #20

Earlier quoted context omitted.

This is a tired semantic argument that does not bring any insight into the discussion. A token-predictor could still be trained to predict the tokens “I’m not sure what you mean because of points x, y, and z; could you elaborate?”

It means if you want something resembling a self-introspective theory of mind, you need to arrange the overall document to cohere to documents where such things are/appear-to-be happening. This leads us to new questions: How can we characterize and identify real-world documents which fit? How can we determine what features may be significant, and which of those can be easily transplanted to our use-case?

There are a lot of words but it feels like you have never really used LLM's (apologies for the bluntness).

We see LLM's introspecting all the time[1].

>Notably, DeepSeek-AI et al. report that the average response length and downstreamperformance of DeepSeek-R1-Zero increases as training progresses. They further report an “aha moment” during training, which refers to the “emergence” of the model’s ability to reconsider its previously generated content. As we show in Section 3.2, this reconsideration behaviour is often indicated by the generation of phrases such as ‘wait, ...’ or ‘alternatively, ...’

[1] https://arxiv.org/pdf/2504.07128

Re: LLMs get lost in multi-turn conversation

#106

Post by dragonwriter »

CobrastanJorji wrote: post #62
Benjammer wrote: post #2

It's nice to see a paper that confirms what anyone who has practiced using LLM tools already knows very well, heuristically. Keeping your context clean matters, "conversations" are only a construct of product interfaces, they hurt the quality of responses from the LLM itself, and once your context is "poisoned" it will not recover, you need to start fresh with a new chat.

An interesting little example of this problem is initial prompting, which is effectively just a permanent, hidden context that can't be cleared. On Twitter right now, the "Grok" bot has recently begun frequently mentioning "White Genocide," which is, y'know, odd. This is almost certainly because someone recently adjusted its prompt to tell it what its views on white genocide are meant to be, which for a perfect chatb…

> This is almost certainly because someone recently adjusted its prompt to tell it what its views on white genocide are meant to be

Well, someone did something to it; whether it was training, feature boosting the way Golden Gate Claude [0] was done, adjusting the system prompt, or assuring that it's internet search for contextual information would always return material about that, or some combination of those, is neither obvious nor, if someone had a conjecture as to which one or combination it was, easily falsifiable/verifiable.

[0] https://www.anthropic.com/news/golden-gate-claude

Re: LLMs get lost in multi-turn conversation

#107

Post by amelius »

Benjammer wrote: post #2

It's nice to see a paper that confirms what anyone who has practiced using LLM tools already knows very well, heuristically. Keeping your context clean matters, "conversations" are only a construct of product interfaces, they hurt the quality of responses from the LLM itself, and once your context is "poisoned" it will not recover, you need to start fresh with a new chat.

I suppose that the chain-of-thought style of prompting that is used by AI chat applications internally also breaks down because of this phenomenon.

Re: LLMs get lost in multi-turn conversation

#108

Post by stevedonovan »

CobrastanJorji wrote: post #62
Benjammer wrote: post #2

It's nice to see a paper that confirms what anyone who has practiced using LLM tools already knows very well, heuristically. Keeping your context clean matters, "conversations" are only a construct of product interfaces, they hurt the quality of responses from the LLM itself, and once your context is "poisoned" it will not recover, you need to start fresh with a new chat.

An interesting little example of this problem is initial prompting, which is effectively just a permanent, hidden context that can't be cleared. On Twitter right now, the "Grok" bot has recently begun frequently mentioning "White Genocide," which is, y'know, odd. This is almost certainly because someone recently adjusted its prompt to tell it what its views on white genocide are meant to be, which for a perfect chatb…

Ah, Elon paying attention to hid companies again!

Context poisoning is not a uniquely LLM problem

Re: LLMs get lost in multi-turn conversation

#109

Post by kfarr »

djmips wrote: post #42
Benjammer wrote: post #2

It's nice to see a paper that confirms what anyone who has practiced using LLM tools already knows very well, heuristically. Keeping your context clean matters, "conversations" are only a construct of product interfaces, they hurt the quality of responses from the LLM itself, and once your context is "poisoned" it will not recover, you need to start fresh with a new chat.

Happens with people too if you think about it.

Who gets lost in multi-turn conversations?

Re: LLMs get lost in multi-turn conversation

#110

Post by Capricorn2481 »

jumploops wrote: post #82

It's amazing that branching/forking isn't a core aspect of the main chat tools. You can edit responses, sure, but then a bunch of other context is lost. My flow is basically: 1. plan 2. build 3. branch (into some feature/esoteric dependency issue) 4. goto #2 Prompt pruning/branching should be a first-class tool for any LLM usage.

I've been kicking around making this for a while. BetterChatGPT at least has some good ergonomics around deleting history. But I agree that branching is the next step.
Post reply on HN