LLMs get lost in multi-turn conversation • Hacker News Board

Re: LLMs get lost in multi-turn conversation

#181

Post by root_axis » Thu, May 15, 2025, 1:35 PM UTC

jsnider3 wrote: post #158

root_axis wrote: post #67

Earlier quoted context omitted.

It could be trained to say that, but it's not exactly clear how you would reinforce the absence of certain training data in order to emit that response accurately, rather than just based on embedding proximity.

Seems easy. Have a set of vague requests and train it to ask for clarification instead of guessing.

As I said, it's possible to train it to ask for clarification, but it's not clear how to reinforce that response in a way that correctly maps on to the absence of data rather than arbitrary embedding proximity. You can't explicitly train on every possible scenario where the AI should recognize its lack of knowledge.

Re: LLMs get lost in multi-turn conversation

#182

Post by saberience » Thu, May 15, 2025, 1:35 PM UTC

dkdbejwi383 wrote: post #161

Sharlin wrote: post #20

Earlier quoted context omitted.

This is a tired semantic argument that does not bring any insight into the discussion. A token-predictor could still be trained to predict the tokens “I’m not sure what you mean because of points x, y, and z; could you elaborate?”

How would an LLM “know” when it isn’t sure? Their baseline for truth is competent text, they don’t have a baseline for truth based on observed reality. That’s why they can be “tricked” into things like “Mr Bean is the president of the USA”

Humans can just as easily be tricked. Something like 25% of the American Electorate believed Obama was the antichrist.

So saying LLMs have no "baseline for truth" doesn't really mean much one way of the other, they are much smart and accurate than 99% of humans.

Re: LLMs get lost in multi-turn conversation

#183

Post by root_axis » Thu, May 15, 2025, 1:43 PM UTC

simianwords wrote: post #103

root_axis wrote: post #67

Earlier quoted context omitted.

It could be trained to say that, but it's not exactly clear how you would reinforce the absence of certain training data in order to emit that response accurately, rather than just based on embedding proximity.

Why does it seem so hard to make training data for this? You can cook up a few thousands of training data and do an RLHF.

Yes, but all that does is locate "I don't know" near the cooked up data within the embeddings. This doesn't actually reflect an absence of data in the training.

Re: LLMs get lost in multi-turn conversation

#184

Post by diggan » Thu, May 15, 2025, 1:43 PM UTC

dr_dshiv wrote: post #180

Benjammer wrote: post #2

It's nice to see a paper that confirms what anyone who has practiced using LLM tools already knows very well, heuristically. Keeping your context clean matters, "conversations" are only a construct of product interfaces, they hurt the quality of responses from the LLM itself, and once your context is "poisoned" it will not recover, you need to start fresh with a new chat.

The #1 tip I teach is to make extensive use of the teeny-tiny mostly hidden “edit” button in ChatGPT and Claude. When you get a bad response, stop and edit to get a better one, rather than letting crap start to multiply crap.

Hear hear! Basically if the first reply isn't good/didnt understand/got something wrong, restart from the beginning with a better prompt, explaining more/better. Rinse and repeat.

Re: LLMs get lost in multi-turn conversation

#185

Post by giordanol » Thu, May 15, 2025, 1:47 PM UTC

stuffoverflow wrote: post #95

b800h wrote: post #86

Earlier quoted context omitted.

I've been saying for ages that I want to be able to fork conversations so I can experiment with the direction an exchange takes without irrevocably poisoning a promising well. I can't do this with ChatGPT, is anyone aware of a provider that offers this as a feature?

Google AI studio, ChatGPT and Claude all support this. Google AI studio is the only one that let's you branch to a separate chat though. For ChatGPT and claude you just edit the message you want to branch from.

Feels like a semi-simple UX fix could make this a lot more natural. Git-style forks but for chats.

Re: LLMs get lost in multi-turn conversation

#186

Post by yaur » Thu, May 15, 2025, 1:48 PM UTC

Benjammer wrote: post #2

It's nice to see a paper that confirms what anyone who has practiced using LLM tools already knows very well, heuristically. Keeping your context clean matters, "conversations" are only a construct of product interfaces, they hurt the quality of responses from the LLM itself, and once your context is "poisoned" it will not recover, you need to start fresh with a new chat.

One of the most frustrating features of ChatGPT is “memories” which can cause that poisoning to follow you around between chats.

Re: LLMs get lost in multi-turn conversation

#187

Post by aleksituk » Thu, May 15, 2025, 1:54 PM UTC

Benjammer wrote: post #2

It's nice to see a paper that confirms what anyone who has practiced using LLM tools already knows very well, heuristically. Keeping your context clean matters, "conversations" are only a construct of product interfaces, they hurt the quality of responses from the LLM itself, and once your context is "poisoned" it will not recover, you need to start fresh with a new chat.

Yarp! And "poisoning" can be done with "off-topic" questions and answers as well as just sort of "dilution". Have noticed this when doing content generation repeatedly, tight instructions get diluted over time.

Re: LLMs get lost in multi-turn conversation

#188

Post by foldr » Thu, May 15, 2025, 1:58 PM UTC

ben_w wrote: post #170

dkdbejwi383 wrote: post #161

Earlier quoted context omitted.

How would an LLM “know” when it isn’t sure? Their baseline for truth is competent text, they don’t have a baseline for truth based on observed reality. That’s why they can be “tricked” into things like “Mr Bean is the president of the USA”

The answer is the same as how the messy bag of chemistry that is the human brain "knows" when it isn't sure: Badly, and with great difficulty, so while it can just about be done, even then only kinda.

We really don’t understand the human brain well enough to have confidence that the mechanisms that cause people to respond with “I don’t know” are at all similar to the mechanisms which cause LLMs to give such responses. And there are quite a few prima facie reasons to think that they wouldn’t be the same.

Re: LLMs get lost in multi-turn conversation

#189

Post by alganet » Thu, May 15, 2025, 2:00 PM UTC

imtringued wrote: post #149

alganet wrote: post #3

Humans also often get lost in multi-turn conversation. I have experienced that in person many, many times. Jumps in context that seem easy for one person to follow, but very hard for others. So, assuming the paper is legit (arxiv, you never know...), its more like something that could be improved than a difference from human beings.

What you're talking about has absolutely nothing to do with the paper. It's not about jumps in context. It's about LLMs being biased towards producing a complete answer on first try, even when there isn't even enough information. When you provide them with additional information, they will stick with the originally wrong answer. This means that you need to frontload all information in the first prompt and if the LLM…

I see, thanks for the correction.

Re: LLMs get lost in multi-turn conversation

#190

Post by diggan » Thu, May 15, 2025, 2:09 PM UTC

johnisgood wrote: post #160

sixtyj wrote: post #133

Earlier quoted context omitted.

In Gemini, it is really good to have large window with 1M tokens. However, around 100,000 it starts to make mistakes and refactor its own code. Sometimes it is good to start new chat or switch to Claude. And it really helps to be very precise with wording of specification what you want to achieve. Or repeat it sometimes with some added request lines. GIGO in reality :)

Oh my, I hate it when it rewrites >1k LOC. I have to instruct it to "modify only ..., do not touch the rest" and so forth, but GPT does not listen to this often, Claude does. I dunno about Gemini.

In terms of "does useless refactors I didn't ask for nor improved anything", my own ranked list goes something like: Gemini > Claude > GPT. I don't really experience this at all with various GPT models used via the API, but overall GPTs seems to stick to the system prompt way better than the rest. Clause does OK too, but Gemini is out of control and writes soo much code and does so much you didn't ask for, really acts like a overly eager junior developer.