141–150 of 272 posts
Why do LLMs struggle so much with recovering from early wrong turns in multi-turn conversations — even when all prior context is available and tokenized? Is it due to the model's training distribution (mostly single-shot completions), the way context windows are encoded, or an architectural bottleneck? Feels like there's no dynamic internal state that evolves over the conversation — only a repeated re-parsing of stat…
It's nice to see a paper that confirms what anyone who has practiced using LLM tools already knows very well, heuristically. Keeping your context clean matters, "conversations" are only a construct of product interfaces, they hurt the quality of responses from the LLM itself, and once your context is "poisoned" it will not recover, you need to start fresh with a new chat.
I've been saying for ages that I want to be able to fork conversations so I can experiment with the direction an exchange takes without irrevocably poisoning a promising well. I can't do this with ChatGPT, is anyone aware of a provider that offers this as a feature?
Earlier quoted context omitted.
It's for higher quality of output. Better solutions. These are the state of the art reasoning models (subscription only, no free access) which are smarter. It also mainly happens when the context is clear that we are collaborating on work that will require multiple iterations of review and feedback, like drafting chapters of a handbook. I have seen ChatGPT ask questions immediately upfront when it relates to medical…
Close. Higher engagement means the user is more invested and values the solution more. The users are being engineered more than the models are, and this isn't the only example.
In the case of medical questions it needs to know further details to provide a relevant diagnosis. That is how it was trained.
In other cases you can observe its reasoning process to see why it would decide to request further details.
I have never seen an LLM just ask questions for the sake of asking. It is always relevant in the context. I don't use them casually. Just wrote a couple of handbooks (~100 pages in a few days). Generating tens of thousands of tokens per session with Gemini.
It's nice to see a paper that confirms what anyone who has practiced using LLM tools already knows very well, heuristically. Keeping your context clean matters, "conversations" are only a construct of product interfaces, they hurt the quality of responses from the LLM itself, and once your context is "poisoned" it will not recover, you need to start fresh with a new chat.
This matches my experience exactly. "poisoned" is a great way to put it. I find once something has gone wrong all subsequent responses are bad. This is why I am iffy on ChatGPT's memory features. I don't notice it causing any huge problems but I don't love how it pollutes my context in ways I don't fully understand.
A nice middle-ground I'm finding is to ask Claude an initial conversation starter in its "thinking" mode, and then copy/paste that conversation into LMStudio and have a weaker model like Gemma pick-up from where Claude left off.
Why do LLMs struggle so much with recovering from early wrong turns in multi-turn conversations — even when all prior context is available and tokenized? Is it due to the model's training distribution (mostly single-shot completions), the way context windows are encoded, or an architectural bottleneck? Feels like there's no dynamic internal state that evolves over the conversation — only a repeated re-parsing of stat…
When you generate future tokens, you're looking at history tokens that are happy.
So how can a model, given sad tokens, generate future happy tokens if it did not learn to do so?
The work you're looking for is already here, it's "thinking". I assume they include sad tokens in the dataset, produce "thinking", which should result in happy tokens coming after thinking tokens. If thinking is bad (by looking at following happy tokens), then it's punished, if good, then descent.
Earlier quoted context omitted.
This is a tired semantic argument that does not bring any insight into the discussion. A token-predictor could still be trained to predict the tokens “I’m not sure what you mean because of points x, y, and z; could you elaborate?”
It's not a tired argument, and not just a semantic one it's a foundational characteristic of LLM. > A token-predictor could still be trained to predict the tokens “I’m not sure what you mean because of points x, y, and z; could you elaborate?” This is entirely true, and the key insight is even right in your sentence but you don't seem to grasp it. “could still be trained”: you can train an LLM into doing whatever you…
Pre-trained LLMs will ask clarifying questions just fine. So I think this is just another consequence of post-training recipes.
Earlier quoted context omitted.
Recently, Gemini helped me fix a bug in a PPP driver (Zephyr OS) without prior knowledge of PPP or even driver development really. I would copy-paste logs of raw PPP frames in HEX and it would just decode everything and explain the meaning of each bytes. In about an hour, I knew enough about PPP to fix the bug and submit a patch. https://g.co/gemini/share/7edf8fa373fe
Or you could just read the PPP RFC [0]. I’m not saying that your approach is wrong. But most LLM workflows are either brute forcing the solution, or seeking a local minima to be stuck in. It’s like doing thousands of experiments of objects falling to figure out gravity while there’s a physics textbooks nearby. [0]: https://datatracker.ietf.org/doc/html/rfc1661
There’s no way I could fully read that RFC in an hour. And that’s before you even know what reading to focus your attention on, so you’re just being a worse LLM at that point.
Humans also often get lost in multi-turn conversation. I have experienced that in person many, many times. Jumps in context that seem easy for one person to follow, but very hard for others. So, assuming the paper is legit (arxiv, you never know...), its more like something that could be improved than a difference from human beings.