Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs get lost in multi-turn conversation

arxiv.org

Post reply on HN
Print view View original

251–260 of 272 posts

Re: LLMs get lost in multi-turn conversation

#251

Post by cruffle_duffle »

dr_dshiv wrote: post #180
Benjammer wrote: post #2

It's nice to see a paper that confirms what anyone who has practiced using LLM tools already knows very well, heuristically. Keeping your context clean matters, "conversations" are only a construct of product interfaces, they hurt the quality of responses from the LLM itself, and once your context is "poisoned" it will not recover, you need to start fresh with a new chat.

The #1 tip I teach is to make extensive use of the teeny-tiny mostly hidden “edit” button in ChatGPT and Claude. When you get a bad response, stop and edit to get a better one, rather than letting crap start to multiply crap.

It is also a great way to branch conversations from some shared “initial context”.

They really need to make that edit feature much more prominent. It is such an important way to interact with the model.

Re: LLMs get lost in multi-turn conversation

#252

Post by cruffle_duffle »

b800h wrote: post #94
layer8 wrote: post #80

Earlier quoted context omitted.

This has been in ChatGPT from pretty early on? Just edit any prompt, it creates a new branch, and you can switch back and forth.

Blimey, I didn't realise the entire thread was saved when you edited a prompt. Very good! Mind you, it feels "unsafe". I'd like to be able to clone a thread.

You can toggle between different branches in both GPT and Claude.

Re: LLMs get lost in multi-turn conversation

#253

Post by shrewduser »

morsecodist wrote: post #9
Benjammer wrote: post #2

It's nice to see a paper that confirms what anyone who has practiced using LLM tools already knows very well, heuristically. Keeping your context clean matters, "conversations" are only a construct of product interfaces, they hurt the quality of responses from the LLM itself, and once your context is "poisoned" it will not recover, you need to start fresh with a new chat.

This matches my experience exactly. "poisoned" is a great way to put it. I find once something has gone wrong all subsequent responses are bad. This is why I am iffy on ChatGPT's memory features. I don't notice it causing any huge problems but I don't love how it pollutes my context in ways I don't fully understand.

I have very limited experience with llms but i've always thought of it as a compounding errors problem, once you get a small error early on it can compound and go completely off track later.

Re: LLMs get lost in multi-turn conversation

#254

Post by foldr »

Sharlin wrote: post #250
foldr wrote: post #246

Earlier quoted context omitted.

'Analogous in the morphology sense' is actually a more specific concept than 'similar'. But either way, we still don't know if they're analogous, or similar, or whatever term you prefer. Anyone who actually understands both LLMs and the human brain well enough to make confident claims that they basically work the same really ought to put in the effort to write up a paper and get a Nobel prize or two.

Analogous in the morphology sense means having come up with an entirely distinct solution to a common problem. Insect and bird wings have little to do with each other except that both flap to create lift. It explicitly does not imply the solutions are similar in mechanism, although that can be, and often is, a result of convergent evolution, of course. In particular, generally speaking (not claiming that LLMs a road…

It was a characterization of the position of the post I was originally responding to, not your position.

I don’t think anyone in this discussion has claimed that brains are uniquely suited to producing intelligence. The point was just that we have no idea if there is any interesting correspondence between how LLMs work and how brains work, beyond superficial and obvious analogies.

Re: LLMs get lost in multi-turn conversation

#255

Post by ben_w »

foldr wrote: post #188
ben_w wrote: post #170

Earlier quoted context omitted.

The answer is the same as how the messy bag of chemistry that is the human brain "knows" when it isn't sure: Badly, and with great difficulty, so while it can just about be done, even then only kinda.

We really don’t understand the human brain well enough to have confidence that the mechanisms that cause people to respond with “I don’t know” are at all similar to the mechanisms which cause LLMs to give such responses. And there are quite a few prima facie reasons to think that they wouldn’t be the same.

FWIW, I'm describing failure modes of a human, not mechanisms.

I also think "would" in the comment I'm replying to is closer to "could" than to "does".

Re: LLMs get lost in multi-turn conversation

#256

Post by arkh »

Sharlin wrote: post #7

Seems like this is an aspect of their well-known overconfidence and the inability to self-reflect and recognize they have to ask for more details because their priors are too low. If you look at the output of reasoning models, it’s clear that the idea of asking for clarification very rarely occurs to them – when they’re confused, it’s just endless speculation of what the user might have meant. This, of course, has ce…

> Seems like this is an aspect of their well-known overconfidence and the inability to self-reflect and recognize they have to ask for more details because their priors are too low.

When I read this I feel like I'm witnessing intelligent people get fooled by a better Emacs doctor. It is not reflecting, it is not confident. It is "just" proposing text completion. That is why once the completion starts being bad you have to start anew. It does not have any concept of anything just a huge blob of words and possible follow-up from what the texts used to train it show.

Re: LLMs get lost in multi-turn conversation

#257

Post by foldr »

ben_w wrote: post #255
foldr wrote: post #188

Earlier quoted context omitted.

We really don’t understand the human brain well enough to have confidence that the mechanisms that cause people to respond with “I don’t know” are at all similar to the mechanisms which cause LLMs to give such responses. And there are quite a few prima facie reasons to think that they wouldn’t be the same.

FWIW, I'm describing failure modes of a human, not mechanisms. I also think "would" in the comment I'm replying to is closer to "could" than to "does".

Could you expand on that? What failure modes are we talking about exactly?

Re: LLMs get lost in multi-turn conversation

#258

Post by sattard »

Why haven't AI code editors not built this at their core yet, to automatically consolidate previous conversational turns into a more structured context summary. Instead of relying solely on the model’s memory of all prior exchanges, surely these tools should take responsibility for intermittently “restating” the clarified requirements so the model doesn’t have to reconstruct context from scratch (or worse, pick up mistakes). This might mitigate compounding errors and reduce verbosity.

Re: LLMs get lost in multi-turn conversation

#259

Post by wrasee »

Retric wrote: post #212
wrasee wrote: post #148

Earlier quoted context omitted.

I could also walk everywhere, but sometimes technology can help. There’s no way I could fully read that RFC in an hour. And that’s before you even know what reading to focus your attention on, so you’re just being a worse LLM at that point.

The difference is you’d remember some of the context from reading the thing where an LLM is starting from scratch every single time it comes up.

I would remember the reply from the LLM, and cross references back to the particular parts of the RFC it identified as worth focusing time on.

I’d argue that’s a more effective capture as to what I would remember anyway.

If wanted to learn more (in a general sense) I can take the manual away with me and study it, which I can do more effectively on its own terms, in a comfy chair with a beer. But right now I have a problem to solve.

Re: LLMs get lost in multi-turn conversation

#260

Post by Retric »

wrasee wrote: post #259
Retric wrote: post #212

Earlier quoted context omitted.

The difference is you’d remember some of the context from reading the thing where an LLM is starting from scratch every single time it comes up.

I would remember the reply from the LLM, and cross references back to the particular parts of the RFC it identified as worth focusing time on. I’d argue that’s a more effective capture as to what I would remember anyway. If wanted to learn more (in a general sense) I can take the manual away with me and study it, which I can do more effectively on its own terms, in a comfy chair with a beer. But right now I have a pr…

Reading it at some later date means you also spent time with the LLM without having read the RFC. So reading it in the future means it’s going to be useful fewer times and thus less efficient overall.

IE LLM then RFC takes more time then RFC then solving the issue.

Post reply on HN