Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs get lost in multi-turn conversation

arxiv.org

Post reply on HN
Print view View original

241–250 of 272 posts

Re: LLMs get lost in multi-turn conversation

#241

Post by Sharlin »

bandrami wrote: post #139
simianwords wrote: post #105

Earlier quoted context omitted.

There are a lot of words but it feels like you have never really used LLM's (apologies for the bluntness). We see LLM's introspecting all the time[1]. >Notably, DeepSeek-AI et al. report that the average response length and downstreamperformance of DeepSeek-R1-Zero increases as training progresses. They further report an “aha moment” during training, which refers to the “emergence” of the model’s ability to reconside…

Unless they show you the Markov chain weights (and I've never seen one that does), that's confabulation, not introspection.

Unless you can show the Markov chain weights, I declare all your thoughts confabulation, not introspection.

Re: LLMs get lost in multi-turn conversation

#242

Post by Sharlin »

foldr wrote: post #188
ben_w wrote: post #170

Earlier quoted context omitted.

The answer is the same as how the messy bag of chemistry that is the human brain "knows" when it isn't sure: Badly, and with great difficulty, so while it can just about be done, even then only kinda.

We really don’t understand the human brain well enough to have confidence that the mechanisms that cause people to respond with “I don’t know” are at all similar to the mechanisms which cause LLMs to give such responses. And there are quite a few prima facie reasons to think that they wouldn’t be the same.

The mechanics don't have to be similar, only analogous, in the morphology sense.

Re: LLMs get lost in multi-turn conversation

#243

Post by Sharlin »

JustFinishedBSG wrote: post #171
dkdbejwi383 wrote: post #161

Earlier quoted context omitted.

How would an LLM “know” when it isn’t sure? Their baseline for truth is competent text, they don’t have a baseline for truth based on observed reality. That’s why they can be “tricked” into things like “Mr Bean is the president of the USA”

It would "know" the same way it "knows" anything else: The probability of the sequence "I don't know" would be higher than the probability of any other sequence.

Exactly. It's easy to imagine a component in the net that the model is steered towards when nothing else has a high enough activation.

Re: LLMs get lost in multi-turn conversation

#244

Post by Retric »

skydhash wrote: post #229
Retric wrote: post #226

Earlier quoted context omitted.

There’s plenty to learn from using LLM’s including how to interact with an LLM. However, even outside of using a LLM the temptation is always to keep the blinders on do a deep dive for a very specific bug and repeat as needed. It’s the local minima of effort and very slowly you do improve as those deep dives occasionally come up again, but what keeps it from being a global minimum is these systems aren’t suddenly goi…

Sometimes learning means understanding, aka a deep dive on the domain. Only a few domains are worth that. For the others, it's only about placing landmark so you can quickly recognize a problem and find the relevant information before solving it. I believe the best use case of LLMs is when you have recognized the problem and know the general shape of the solution, but have no time to wrangle the specifics of the impl…

> Only a few domains are worth that. For the others, it's only about placing landmark so you can quickly recognize a problem and find the relevant information before solving it.

Well said. You can only spend years digging into the intricacies a handful of systems in your lifetime, but there’s still real rewards from a few hours here and there.

Re: LLMs get lost in multi-turn conversation

#245

Post by Terr_ »

famouswaffles wrote: post #238
Terr_ wrote: post #234

Earlier quoted context omitted.

> LLMs already break key aspects and assumptions of the 'Document Simulator'. [...] The “document-simulator” picture collapses that distinction, treating a dynamic decision process as if it were a block of pre-written prose. It's just nonsensical. I feel you've erected a strawman under your this "document simulator" phrase of yours, something you've arbitrarily defined as a strictly one-shot process for creating an i…

I’m not arbitrarily defining it as a one-shot process. I’m pointing out how strained your “movie-script” (your words, not mine) comparison is. >You can have an interview with a vampire DraculaBot, but that character can only "self-reflect" in the same shallow/fictional way that it can "thirst for blood" or "turn into a cloud of bats." The "shallow/fictional way" only exists because of the limited, immutable nature of…

> I’m pointing out how strained your “movie-script” (your words, not mine) comparison is. [...] the limited, immutable nature of real scripts [...] a screenplay whose pages are fixed in advance.

You are confused and again attacking an idea nobody else has advanced.

Even in my very first comment starting the thread, I explicitly stated that the "movie-script" is mutable, with alternate phases of "contributing" and "autocompleted" content as it grows.

Re: LLMs get lost in multi-turn conversation

#246

Post by foldr »

Sharlin wrote: post #242
foldr wrote: post #188

Earlier quoted context omitted.

We really don’t understand the human brain well enough to have confidence that the mechanisms that cause people to respond with “I don’t know” are at all similar to the mechanisms which cause LLMs to give such responses. And there are quite a few prima facie reasons to think that they wouldn’t be the same.

The mechanics don't have to be similar, only analogous, in the morphology sense.

'Analogous in the morphology sense' is actually a more specific concept than 'similar'. But either way, we still don't know if they're analogous, or similar, or whatever term you prefer.

Anyone who actually understands both LLMs and the human brain well enough to make confident claims that they basically work the same really ought to put in the effort to write up a paper and get a Nobel prize or two.

Re: LLMs get lost in multi-turn conversation

#247

Post by Terr_ »

layer8 wrote: post #81
Terr_ wrote: post #17

Earlier quoted context omitted.

> inability to self-reflect IMO the One Weird Trick for LLMs is recognizing that there's no real entity, and that users are being tricked into a suspended-disbelief story. In most cases cases you're contributing text-lines for a User-character in a movie-script document, and the LLM algorithm is periodically triggered to autocomplete incomplete lines for a Chatbot character. You can have an interview with a vampire D…

Not to mention that vampires don’t reflect. ;)

Haha, true... however unlike LLMs, folklore tells us they can count! (Obsessively.)

https://carnegiemnh.org/booseum-vampires/

Re: LLMs get lost in multi-turn conversation

#248

Post by voidspark »

Workaccount2 wrote: post #192
voidspark wrote: post #44

Earlier quoted context omitted.

> inability to self-reflect and recognize they have to ask for more details because their priors are too low. Gemini 2.5 Pro and ChatGPT-o3 have often asked me to provide additional details before doing a requested task. Gemini sometimes comes up with multiple options and requests my input before doing the task.

Gemini is also the first model I have seen call me out in it's thinking. Stuff like "The user suggested we take approach ABC, but I don't think the user fully understands ABC, I will suggest XYZ as an alternative since it would be a better fit"

It is impressive when it finds subtle errors in complex reasoning.

But even the dumbest model will call you out if you ask it something like:

"Hey I'm going to fill up my petrol car with diesel to make it faster. What brand of diesel do you recommend?"

Re: LLMs get lost in multi-turn conversation

#249

Post by famouswaffles »

Terr_ wrote: post #245
famouswaffles wrote: post #238

Earlier quoted context omitted.

I’m not arbitrarily defining it as a one-shot process. I’m pointing out how strained your “movie-script” (your words, not mine) comparison is. >You can have an interview with a vampire DraculaBot, but that character can only "self-reflect" in the same shallow/fictional way that it can "thirst for blood" or "turn into a cloud of bats." The "shallow/fictional way" only exists because of the limited, immutable nature of…

> I’m pointing out how strained your “movie-script” (your words, not mine) comparison is. [...] the limited, immutable nature of real scripts [...] a screenplay whose pages are fixed in advance. You are confused and again attacking an idea nobody else has advanced. Even in my very first comment starting the thread, I explicitly stated that the "movie-script" is mutable , with alternate phases of "contributing" and "a…

Seriously what's so hard to understand that the things you are claiming are the result of a LLM that is analogous to a script are only properties of the kinds of scripts LLMs are not (and so have no leg to stand on)?

This is not a hard concept to grasp. I know what you are claiming. It doesn't automatically make your argument sound.

To call something that does not have the properties of a script a script is odd in the first place, but to realize that and still assume behaviors that are only the result of the properties you realize are not even present in your new 'script' is just bizzare.

I'm not confused. You are.

Re: LLMs get lost in multi-turn conversation

#250

Post by Sharlin »

foldr wrote: post #246
Sharlin wrote: post #242

Earlier quoted context omitted.

The mechanics don't have to be similar, only analogous, in the morphology sense.

'Analogous in the morphology sense' is actually a more specific concept than 'similar'. But either way, we still don't know if they're analogous, or similar, or whatever term you prefer. Anyone who actually understands both LLMs and the human brain well enough to make confident claims that they basically work the same really ought to put in the effort to write up a paper and get a Nobel prize or two.

Analogous in the morphology sense means having come up with an entirely distinct solution to a common problem. Insect and bird wings have little to do with each other except that both flap to create lift. It explicitly does not imply the solutions are similar in mechanism, although that can be, and often is, a result of convergent evolution, of course.

In particular, generally speaking (not claiming that LLMs a road to AGI, which is something I doubt) it's generally not a well-defensible philosophical position that the vertebrate brain (and remember that mammalian, bird and cephalopod brains are very different) is uniquely suited to produce what we call "intelligence".

> Anyone who actually understands both LLMs and the human brain well enough to make confident claims that they basically work the same

This is a strawman and not my position.

Post reply on HN