LLMs get lost in multi-turn conversation • Hacker News Board

Re: LLMs get lost in multi-turn conversation

#81

Post by layer8 » Thu, May 15, 2025, 6:14 AM UTC

Terr_ wrote: post #17

Sharlin wrote: post #7

Seems like this is an aspect of their well-known overconfidence and the inability to self-reflect and recognize they have to ask for more details because their priors are too low. If you look at the output of reasoning models, it’s clear that the idea of asking for clarification very rarely occurs to them – when they’re confused, it’s just endless speculation of what the user might have meant. This, of course, has ce…

> inability to self-reflect IMO the One Weird Trick for LLMs is recognizing that there's no real entity, and that users are being tricked into a suspended-disbelief story. In most cases cases you're contributing text-lines for a User-character in a movie-script document, and the LLM algorithm is periodically triggered to autocomplete incomplete lines for a Chatbot character. You can have an interview with a vampire D…

Not to mention that vampires don’t reflect. ;)

Re: LLMs get lost in multi-turn conversation

#82

Post by jumploops » Thu, May 15, 2025, 6:15 AM UTC

It's amazing that branching/forking isn't a core aspect of the main chat tools.

You can edit responses, sure, but then a bunch of other context is lost.

My flow is basically:

1. plan

2. build

3. branch (into some feature/esoteric dependency issue)

4. goto #2

Prompt pruning/branching should be a first-class tool for any LLM usage.

Re: LLMs get lost in multi-turn conversation

#83

Post by layer8 » Thu, May 15, 2025, 6:24 AM UTC

zacksiri wrote: post #11

I've been working on solving this with quite a bit of success, I'll be sharing more on this soon. It involves having 2 systems 1st system is the LLM itself and another system which acts like a 'curator' of thoughts you could say. It dynamically swaps in / out portions of the context. This system is also not based on explicit definitions it relies on LLMs 'filling the gaps'. The system helps the llm break down problem…

So, Map-Reduce-of-Thought?

Re: LLMs get lost in multi-turn conversation

#84

Post by ithkuil » Thu, May 15, 2025, 6:29 AM UTC

unshavedyak wrote: post #16

Benjammer wrote: post #2

It's nice to see a paper that confirms what anyone who has practiced using LLM tools already knows very well, heuristically. Keeping your context clean matters, "conversations" are only a construct of product interfaces, they hurt the quality of responses from the LLM itself, and once your context is "poisoned" it will not recover, you need to start fresh with a new chat.

Has any interface implemented a .. history cleaning mechanism? Ie with every chat message focus on cleaning up dead ends in the conversation or irrelevant details. Like summation but organic for the topic at hand? Most history would remain, it wouldn’t try to summarize exactly, just prune and organize the history relative to the conversation path?

"Every problem in computer science can be solved with another level of indirection."

One could argue that the attention mechanism in transformers is already designed to do that.

But you need to train it more specifically with that in mind if you want it to be better at damping attention to parts that are deemed irrelevant by the subsequent evolution of the conversation.

And that requires the black art of ML training.

While thinking of doing this as a hack on top of the chat product feels more like engineering and we're more familiar with that as a field.

Re: LLMs get lost in multi-turn conversation

#85

Post by SamPatt » Thu, May 15, 2025, 6:29 AM UTC

I always felt the derision around the term "prompt engineering" was partially due to people overestimating the importance of the initial prompt and underestimating the importance of managing the ongoing context.

You develop a knack for how to steer the models or start a new conversation through experience. The system or initial prompt are important, but nothing will save you if you naively keep a conversation going too long.

Re: LLMs get lost in multi-turn conversation

#86

Post by b800h » Thu, May 15, 2025, 6:29 AM UTC

Benjammer wrote: post #2

It's nice to see a paper that confirms what anyone who has practiced using LLM tools already knows very well, heuristically. Keeping your context clean matters, "conversations" are only a construct of product interfaces, they hurt the quality of responses from the LLM itself, and once your context is "poisoned" it will not recover, you need to start fresh with a new chat.

I've been saying for ages that I want to be able to fork conversations so I can experiment with the direction an exchange takes without irrevocably poisoning a promising well. I can't do this with ChatGPT, is anyone aware of a provider that offers this as a feature?

Re: LLMs get lost in multi-turn conversation

#87

Post by veunes » Thu, May 15, 2025, 6:30 AM UTC

Kind of wild how even the best models still struggle with keeping context straight over time. Definitely feels like a big challenge if we want these things to hold real conversations.

Re: LLMs get lost in multi-turn conversation

#88

Post by anonexpat » Thu, May 15, 2025, 6:32 AM UTC

b800h wrote: post #86

Benjammer wrote: post #2

It's nice to see a paper that confirms what anyone who has practiced using LLM tools already knows very well, heuristically. Keeping your context clean matters, "conversations" are only a construct of product interfaces, they hurt the quality of responses from the LLM itself, and once your context is "poisoned" it will not recover, you need to start fresh with a new chat.

I've been saying for ages that I want to be able to fork conversations so I can experiment with the direction an exchange takes without irrevocably poisoning a promising well. I can't do this with ChatGPT, is anyone aware of a provider that offers this as a feature?

I believe Claude has forking in their web interface.

Re: LLMs get lost in multi-turn conversation

#89

Post by hobofan » Thu, May 15, 2025, 6:32 AM UTC

unshavedyak wrote: post #16

Benjammer wrote: post #2

It's nice to see a paper that confirms what anyone who has practiced using LLM tools already knows very well, heuristically. Keeping your context clean matters, "conversations" are only a construct of product interfaces, they hurt the quality of responses from the LLM itself, and once your context is "poisoned" it will not recover, you need to start fresh with a new chat.

Has any interface implemented a .. history cleaning mechanism? Ie with every chat message focus on cleaning up dead ends in the conversation or irrelevant details. Like summation but organic for the topic at hand? Most history would remain, it wouldn’t try to summarize exactly, just prune and organize the history relative to the conversation path?

Not a history cleaning mechanism, but related to that, Cursor in the most recent release introduced a feature to duplicate your chat (so you can saveguard yourself against poisoning and go back to and unpoisoned point in history), which seems like an addmision of the same problem.

Re: LLMs get lost in multi-turn conversation

#90

Post by granra » Thu, May 15, 2025, 6:32 AM UTC

b800h wrote: post #86

Benjammer wrote: post #2

It's nice to see a paper that confirms what anyone who has practiced using LLM tools already knows very well, heuristically. Keeping your context clean matters, "conversations" are only a construct of product interfaces, they hurt the quality of responses from the LLM itself, and once your context is "poisoned" it will not recover, you need to start fresh with a new chat.

I've been saying for ages that I want to be able to fork conversations so I can experiment with the direction an exchange takes without irrevocably poisoning a promising well. I can't do this with ChatGPT, is anyone aware of a provider that offers this as a feature?

Some 3rd party UIs offer this, I use typingmind sometimes that does but AFAIK some open source ones do too.