Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs get lost in multi-turn conversation

arxiv.org

Post reply on HN
Print view View original

231–240 of 272 posts

Re: LLMs get lost in multi-turn conversation

#231

Post by voidspark »

rrr_oh_man wrote: post #163
voidspark wrote: post #154

Earlier quoted context omitted.

> "Do you want either [A, B, or C]" That's an example of what I'm talking about. Watch the reasoning process produce multiple options. That's what it is trained to do. That is problem solving, not "engagement". It requires more compute, not less. You see that more with the expensive models. > "In [5-15] minutes I will give you the complete result" I haven't seen that before and I don't see how it's relevant.

> That's an example of what I'm talking about. Watch the reasoning process produce multiple options. That's what it is trained to do. That is problem solving, not "engagement". It requires more compute, not less. You see that more with the expensive models. Fair point. Thanks for standing your ground and arguing so matter-of-factly with me! Appreciate it.

I have never been thanked for replying here before. Thanks.

The optional choices happen when it tries to reason out a solution, but then finds it is making too many assumptions of unknown details about the user's system, preferences, goals, and so on. It's just a thought pattern that it has learned to emulate.

People here will argue that LLM's cannot truly "think", but they are good enough at emulating thinking.

Re: LLMs get lost in multi-turn conversation

#232

Post by airylizard »

thegeomaster wrote: post #209
airylizard wrote: post #207

Earlier quoted context omitted.

Hey, thanks for kicking the tires! The run you’re describing was done in mid-April, right after GPT-4.1 went live. Since then OpenAI has refreshed the weights behind the “gpt-4.1” alias a couple of times, and one of those updates fixed the em-dash miss. If you reran today you’d see the same improved pass rate I’m getting now. That’s the downside of benchmarking against latest model names; behaviour changes quietly un…

> Since then OpenAI has refreshed the weights behind the “gpt-4.1” alias a couple of times, and one of those updates fixed the em-dash miss. I don't know where you are getting this information from... The only snapshot of gpt-4.1 is gpt-4.1-2025-04-14 (mid-April), and the gpt-4.1 alias still points to it [1]. Just to be sure, I re-ran my test specifying that particular snapshot and am still getting a 100% pass rate.…

Right, the 4.1 training checkpoint hasn’t moved. What has moved is the glue on top: decoder heuristics / safety filters / logit-bias rules that OpenAI can hot-swap without re-training the model. Those “serving-layer” tweaks are what stomped the obvious em-dash miss for short, clean prompts. So the April-14 weights are unchanged, but the pipeline that samples from those weights is stricter about “don’t output X” than it was on day one. By all means, keep trying to poke holes! I’ve got nothing to sell; just sharing insights and happy to stress-test them.

Re: LLMs get lost in multi-turn conversation

#233

Post by jsnider3 »

joleyj wrote: post #228
jsnider3 wrote: post #224

Earlier quoted context omitted.

We've only had ChatGPT and the like for a few years. It took Ford longer to make automatic transmissions.

So it is hard? Not easy? I would agree with that position. I think the analogy with automatic transmissions misses though. Programming actual intelligence into a computer seems orders of magnitude more complex and difficult than building the gearbox for a car.

I'm saying it shouldn't be that hard, but it's just one of a long list of features that the people whose job it is to do are working on.

Re: LLMs get lost in multi-turn conversation

#234

Post by Terr_ »

famouswaffles wrote: post #227
Terr_ wrote: post #214

Earlier quoted context omitted.

> Dracula can't make decisions that affect the real world, so LLMs already break key aspects and assumptions of the 'Document Simulator'. Nonsense, we are already surrounded by mindless algorithms (and their outputs) that "affect the real world" because many of us have full-time jobs ensuring it happens ! " When someone uses a SimCity-esque program to generate a spreadsheet used for real-world bus schedules, does tha…

You’re talking past the point I was making. My point about Dracula isn't just that he's fictional, but that he cannot make decisions that have unscripted consequences in the real world, nor can he engage in a novel, interactive conversation. Dracula, as a character, only "acts" or "speaks" as an author (or game designer, etc.) has already written or programmed him to. He has no independent capacity to assess a new si…

> LLMs already break key aspects and assumptions of the 'Document Simulator'. [...] The “document-simulator” picture collapses that distinction, treating a dynamic decision process as if it were a block of pre-written prose. It's just nonsensical.

I feel you've erected a strawman under your this "document simulator" phrase of yours, something you've arbitrarily defined as a strictly one-shot process for creating an immutable document. Yeah, it's boring and "nonsensical" because you made it that way.

In contrast, everybody else here has been busy talking about iterative systems which do permit interaction, because the document is grown via alternate passes of (A) new content from external systems or humans and (B) new content predicted by the LLM.

Re: LLMs get lost in multi-turn conversation

#235

Post by CobrastanJorji »

lolinder wrote: post #175
dragonwriter wrote: post #106

Earlier quoted context omitted.

> This is almost certainly because someone recently adjusted its prompt to tell it what its views on white genocide are meant to be Well, someone did something to it; whether it was training, feature boosting the way Golden Gate Claude [0] was done, adjusting the system prompt, or assuring that it's internet search for contextual information would always return material about that, or some combination of those, is ne…

Source [0]. The examples look pretty clearly like they stuck it in the context window, not trained it in. It consistently seems to structure the replies as though the user they're replying to is the one who brought up white genocide in South Africa, and it responds the way that LLMs often respond to such topics: saying that it's controversial and giving both perspectives. That's not behavior I would expect if they ha…

When your boss is a crazy, drugged-up billionaire who has ADD and also runs the government, when he tells you to do something, you do it the fast way.

Re: LLMs get lost in multi-turn conversation

#236

Post by root_axis »

jsnider3 wrote: post #233
joleyj wrote: post #228

Earlier quoted context omitted.

So it is hard? Not easy? I would agree with that position. I think the analogy with automatic transmissions misses though. Programming actual intelligence into a computer seems orders of magnitude more complex and difficult than building the gearbox for a car.

I'm saying it shouldn't be that hard, but it's just one of a long list of features that the people whose job it is to do are working on.

It is hard in the sense that it's an unsolved problem that emerges due to the way LLMs work. Perhaps some clever ML PhD will come up with a technique to solve it, but right now there's no clear solution.

Re: LLMs get lost in multi-turn conversation

#237

Post by dTal »

Sharlin wrote: post #20
Terr_ wrote: post #17

Earlier quoted context omitted.

> inability to self-reflect IMO the One Weird Trick for LLMs is recognizing that there's no real entity, and that users are being tricked into a suspended-disbelief story. In most cases cases you're contributing text-lines for a User-character in a movie-script document, and the LLM algorithm is periodically triggered to autocomplete incomplete lines for a Chatbot character. You can have an interview with a vampire D…

This is a tired semantic argument that does not bring any insight into the discussion. A token-predictor could still be trained to predict the tokens “I’m not sure what you mean because of points x, y, and z; could you elaborate?”

I disagree, it's a very insightful comment.

The problem is that any information about any internal processes used to generate a particular token is lost; the LLM is stateless, apart from the generated text. If you ask an LLM-character (which I agree should be held distinct from the LLM itself and exists at a different layer of abstraction) why it said something, the best it can do is a post-hoc guess. The "character", and any internal state we might wish it to have, only exists insofar as it can be derived anew from the text.

Re: LLMs get lost in multi-turn conversation

#238

Post by famouswaffles »

Terr_ wrote: post #234
famouswaffles wrote: post #227

Earlier quoted context omitted.

You’re talking past the point I was making. My point about Dracula isn't just that he's fictional, but that he cannot make decisions that have unscripted consequences in the real world, nor can he engage in a novel, interactive conversation. Dracula, as a character, only "acts" or "speaks" as an author (or game designer, etc.) has already written or programmed him to. He has no independent capacity to assess a new si…

> LLMs already break key aspects and assumptions of the 'Document Simulator'. [...] The “document-simulator” picture collapses that distinction, treating a dynamic decision process as if it were a block of pre-written prose. It's just nonsensical. I feel you've erected a strawman under your this "document simulator" phrase of yours, something you've arbitrarily defined as a strictly one-shot process for creating an i…

I’m not arbitrarily defining it as a one-shot process. I’m pointing out how strained your “movie-script” (your words, not mine) comparison is.

>You can have an interview with a vampire DraculaBot, but that character can only "self-reflect" in the same shallow/fictional way that it can "thirst for blood" or "turn into a cloud of bats."

The "shallow/fictional way" only exists because of the limited, immutable nature of real scripts. A 'script' that does not have either of these properties would not necessarily produce characters that only reflect in a shallow manner.

Text that’s generated on-the-fly-while interrogating the user, calling tools, and updating its own working context-isn’t anything like a screenplay whose pages are fixed in advance.

There's no strawman here. You've decided that an LLM is not something you want to attribute a 'real' entity to and this is your rationalization for that.

Re: LLMs get lost in multi-turn conversation

#239

Post by rcdwealth »

oaeirjtlj wrote: post #205
Benjammer wrote: post #2

It's nice to see a paper that confirms what anyone who has practiced using LLM tools already knows very well, heuristically. Keeping your context clean matters, "conversations" are only a construct of product interfaces, they hurt the quality of responses from the LLM itself, and once your context is "poisoned" it will not recover, you need to start fresh with a new chat.

And now that chatgpt has a "memory" and can access previous conversations, it might be poisoned permanently. It gets one really bad idea, and forever after it insists on dumping that bad idea into every subsequent response ever after you repeatedly tell it "THAT'S A SHIT IDEA DON'T EVER MENTION THAT AGAIN". Sometimes it'll accidentally include some of its internal prompting, "user is very unhappy, make sure to not in…

[dead]

Re: LLMs get lost in multi-turn conversation

#240

Post by Sharlin »

dTal wrote: post #237
Sharlin wrote: post #20

Earlier quoted context omitted.

This is a tired semantic argument that does not bring any insight into the discussion. A token-predictor could still be trained to predict the tokens “I’m not sure what you mean because of points x, y, and z; could you elaborate?”

I disagree, it's a very insightful comment. The problem is that any information about any internal processes used to generate a particular token is lost; the LLM is stateless , apart from the generated text. If you ask an LLM-character (which I agree should be held distinct from the LLM itself and exists at a different layer of abstraction) why it said something, the best it can do is a post-hoc guess. The "character…

I certainly agree with the point about post-hoc justifications – but isn't it amazing that it's also something very familiar to humans who do that all the time and manage to lie to ourselves about it very convincingly?! The more you read about neuropsychology the more you're forced to assume a view where the conscious self, whatever it is, has only a very tenuous grasp of what is going on and how much it actually has control over things.

In any case, you don't need accurate understanding of how your mind works (hello humans, again!) to be able to converge on

       INSUFFICIENT DATA FOR A MEANINGFUL ANSWER
when there's no other uniquely good local optimum in the search space.
Post reply on HN