Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs get lost in multi-turn conversation

arxiv.org

Post reply on HN
Print view View original

221–230 of 272 posts

Re: LLMs get lost in multi-turn conversation

#221

Post by Macuyiko »

Benjammer wrote: post #2

It's nice to see a paper that confirms what anyone who has practiced using LLM tools already knows very well, heuristically. Keeping your context clean matters, "conversations" are only a construct of product interfaces, they hurt the quality of responses from the LLM itself, and once your context is "poisoned" it will not recover, you need to start fresh with a new chat.

Weirdly it has gotten so far that I have embedded this into my workflow and will often prompt:

> "Good work so far, now I want to take it to another step (somewhat related but feeling it too hard): . Do you think we can do it in this conversation or is it better to start fresh? If so, prepare an initial prompt for your next fresh instantiation."

Sometimes the model says that it might be better to start fresh, and prepares a good summary prompt (including a final 'see you later'), whereas in other cases it assures me it can continue.

I have a lot of notebooks with "initial prompts to explore forward". But given the sycophancy going on as well as one-step RL (sigh) post-training [1], it indeed seems AI platforms would like to keep the conversation going.

[1] RL in post-training has little to do with real RL and just uses one shot preference mechanisms with an RL inspired training loop. There is very little work in terms of long-term preferences slash conversations, as that would increase requirements exponentially.

Re: LLMs get lost in multi-turn conversation

#222

Post by Macuyiko »

Retric wrote: post #212
wrasee wrote: post #148

Earlier quoted context omitted.

I could also walk everywhere, but sometimes technology can help. There’s no way I could fully read that RFC in an hour. And that’s before you even know what reading to focus your attention on, so you’re just being a worse LLM at that point.

The difference is you’d remember some of the context from reading the thing where an LLM is starting from scratch every single time it comes up.

All of the above is true, but between solving quicker, and admitting we gave context:

I do agree with you that an LLM should not always start from scratch.

In a way it is like an animal which we have given the ultimate human instinct.

What has nature given us? Homo Erectus is 2 million years ago.

A weird world we live in.

What is context.

Re: LLMs get lost in multi-turn conversation

#223

Post by senordevnyc »

Macuyiko wrote: post #221
Benjammer wrote: post #2

It's nice to see a paper that confirms what anyone who has practiced using LLM tools already knows very well, heuristically. Keeping your context clean matters, "conversations" are only a construct of product interfaces, they hurt the quality of responses from the LLM itself, and once your context is "poisoned" it will not recover, you need to start fresh with a new chat.

Weirdly it has gotten so far that I have embedded this into my workflow and will often prompt: > "Good work so far, now I want to take it to another step (somewhat related but feeling it too hard): . Do you think we can do it in this conversation or is it better to start fresh? If so, prepare an initial prompt for your next fresh instantiation." Sometimes the model says that it might be better to start fresh, and pre…

Is there any reason to think that LLMs have the introspection ability to be able to answer your question effectively? I just default to having them provide a summary that I can use to start the next conversation, because I’m unclear on how an LLM would know it’s losing the plot due to long context window.

Re: LLMs get lost in multi-turn conversation

#224

Post by jsnider3 »

joleyj wrote: post #203
jsnider3 wrote: post #158

Earlier quoted context omitted.

Seems easy. Have a set of vague requests and train it to ask for clarification instead of guessing.

If the solution were easy or obvious the problem would likely have already been solved no?

We've only had ChatGPT and the like for a few years. It took Ford longer to make automatic transmissions.

Re: LLMs get lost in multi-turn conversation

#225

Post by jsnider3 »

timdiggerm wrote: post #162
jsnider3 wrote: post #158

Earlier quoted context omitted.

Seems easy. Have a set of vague requests and train it to ask for clarification instead of guessing.

How does it identify what's vague?

Many ways. 1) Hire some humans to label the data. 2) Let the user give you feedback. 3) Ask another LLM.

Re: LLMs get lost in multi-turn conversation

#226

Post by Retric »

cgriswald wrote: post #218
Retric wrote: post #212

Earlier quoted context omitted.

The difference is you’d remember some of the context from reading the thing where an LLM is starting from scratch every single time it comes up.

There are opportunity costs to consider along with relevance. Suppose you are staying at my place. Are you going to read the manual for my espresso machine in total or are you going to ask me to show you how to use it or make one for you? In any case, LLMs are not magical forgetfulness machines. You can use a calculator to avoid learning arithmetic but using a calculator doesn’t necessitate failing to learn arithmeti…

There’s plenty to learn from using LLM’s including how to interact with an LLM.

However, even outside of using a LLM the temptation is always to keep the blinders on do a deep dive for a very specific bug and repeat as needed. It’s the local minima of effort and very slowly you do improve as those deep dives occasionally come up again, but what keeps it from being a global minimum is these systems aren’t suddenly going away. It’s not a friend’s expresso machine, it’s now sitting in your metaphorical kitchen.

As soon as you’re dealt with say a CSS bug the odds of seeing another in the future are dramatically higher. Thus optimizing for diminishing returns means spending a few hours learning the basics of any system or protocol you encounter is just a useful strategy. If you spend 1% of your time on a strategy that makes you 2% more efficient that’s a net win.

Re: LLMs get lost in multi-turn conversation

#227

Post by famouswaffles »

Terr_ wrote: post #214
famouswaffles wrote: post #147

Earlier quoted context omitted.

It's definitely a tired and semantical one because as he said, it brings no insight and is not even good at the analogy level. I can't have a conversation with Dracula and Dracula can't make decisions that affect the real world, so LLMs already break key aspects and assumptions of the 'Document Simulator'. Pre-trained LLMs will ask clarifying questions just fine. So I think this is just another consequence of post-tr…

> Dracula can't make decisions that affect the real world, so LLMs already break key aspects and assumptions of the 'Document Simulator'. Nonsense, we are already surrounded by mindless algorithms (and their outputs) that "affect the real world" because many of us have full-time jobs ensuring it happens ! " When someone uses a SimCity-esque program to generate a spreadsheet used for real-world bus schedules, does tha…

You’re talking past the point I was making.

My point about Dracula isn't just that he's fictional, but that he cannot make decisions that have unscripted consequences in the real world, nor can he engage in a novel, interactive conversation. Dracula, as a character, only "acts" or "speaks" as an author (or game designer, etc.) has already written or programmed him to. He has no independent capacity to assess a new situation and generate a novel response that affects anything beyond his fictional context. If I "talk" to Dracula in a game, the game developers have pre-scripted his possible responses. The text of Dracula is immutable.

A LLM, by contrast, performs fresh inference every time it’s prompted: it weighs competing continuations and selects one. That selection is a bona-fide decision (a branch taken at run-time). The “document-simulator” picture collapses that distinction, treating a dynamic decision process as if it were a block of pre-written prose. It's just nonsensical.

Your SimCity example is open loop: the simulation runs, a human inspects the results, and then decides whether to publish new bus schedules. Nothing in the simulator is tasked with interrogating the human, updating its model of their intent, or steering the outcome. In production LLM systems the loop is often closed: the model (often with tool-wrapper code) directly drafts emails, modifies configs, triggers API calls, or at minimum interrogates the user (“What city are we talking about?”) before emitting an answer.

Your argument is tired and semantical because it fails at the most fundamental level - It's not even a good analogy.

Re: LLMs get lost in multi-turn conversation

#228

Post by joleyj »

jsnider3 wrote: post #224
joleyj wrote: post #203

Earlier quoted context omitted.

If the solution were easy or obvious the problem would likely have already been solved no?

We've only had ChatGPT and the like for a few years. It took Ford longer to make automatic transmissions.

So it is hard? Not easy? I would agree with that position. I think the analogy with automatic transmissions misses though. Programming actual intelligence into a computer seems orders of magnitude more complex and difficult than building the gearbox for a car.

Re: LLMs get lost in multi-turn conversation

#229

Post by skydhash »

Retric wrote: post #226
cgriswald wrote: post #218

Earlier quoted context omitted.

There are opportunity costs to consider along with relevance. Suppose you are staying at my place. Are you going to read the manual for my espresso machine in total or are you going to ask me to show you how to use it or make one for you? In any case, LLMs are not magical forgetfulness machines. You can use a calculator to avoid learning arithmetic but using a calculator doesn’t necessitate failing to learn arithmeti…

There’s plenty to learn from using LLM’s including how to interact with an LLM. However, even outside of using a LLM the temptation is always to keep the blinders on do a deep dive for a very specific bug and repeat as needed. It’s the local minima of effort and very slowly you do improve as those deep dives occasionally come up again, but what keeps it from being a global minimum is these systems aren’t suddenly goi…

Sometimes learning means understanding, aka a deep dive on the domain. Only a few domains are worth that. For the others, it's only about placing landmark so you can quickly recognize a problem and find the relevant information before solving it. I believe the best use case of LLMs is when you have recognized the problem and know the general shape of the solution, but have no time to wrangle the specifics of the implementation. So you can provide the context and its constraint in order to guide the LLM's generation, as well as recognize wrong outputs.

But that's not learning or even problem's solving. It's just a time saving trick. And one that's not reliable.

And the fact is that there's a lot of information about pretty much anything. But I see people trying to skip the foundation (not glamorous enough, maybe) and go straight for the complicated stuff. And LLMs are good for providing the illusion that it can be the right workflow.

Re: LLMs get lost in multi-turn conversation

#230

Post by drewbitt »

tmountain wrote: post #159

I often ask the LLM for a concise summary of the discussion so far—formatted as a prompt. I then edit it appropriately and use it to start a new conversation without the baggage. I have found this to be a very effective technique, but I imagine it will be automated sometime soon.

Cursor tried doing this automatically - it may still if you're not on a large context model like gemini 2.5 pro - but I found the summary was just missing too many details to use out of the box.
Post reply on HN