Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs get lost in multi-turn conversation

arxiv.org

Post reply on HN
Print view View original

61–70 of 272 posts

Re: LLMs get lost in multi-turn conversation

#61

Post by skydhash »

ranyume wrote: post #15

I'd like more research done on context understanding other than NIAH. I don't believe LLMs support the context length companies say they support. But I need to know this to effectively use the tools. At least for coding. Stuff like this: 1. Do: Best practice for X model is to include at max 10k lines of code + task + CONVENTIONS.md + architecture guidance. Only queue tasks for components that are fairly decoupled fro…

You could swap those recommendations for programming without LLMs. Open any software engineering books and you’ll see a lot of good recommendations for building software.

Re: LLMs get lost in multi-turn conversation

#62

Post by CobrastanJorji »

Benjammer wrote: post #2

It's nice to see a paper that confirms what anyone who has practiced using LLM tools already knows very well, heuristically. Keeping your context clean matters, "conversations" are only a construct of product interfaces, they hurt the quality of responses from the LLM itself, and once your context is "poisoned" it will not recover, you need to start fresh with a new chat.

An interesting little example of this problem is initial prompting, which is effectively just a permanent, hidden context that can't be cleared. On Twitter right now, the "Grok" bot has recently begun frequently mentioning "White Genocide," which is, y'know, odd. This is almost certainly because someone recently adjusted its prompt to tell it what its views on white genocide are meant to be, which for a perfect chatbot wouldn't matter when you ask it about other topics, but it DOES matter. It's part of the context. It's gonna talk about that now.

Re: LLMs get lost in multi-turn conversation

#65

Post by sandspar »

sitkack wrote: post #39
Terr_ wrote: post #22

Earlier quoted context omitted.

It means if you want something resembling a self-introspective theory of mind, you need to arrange the overall document to cohere to documents where such things are/appear-to-be happening. This leads us to new questions: How can we characterize and identify real-world documents which fit? How can we determine what features may be significant, and which of those can be easily transplanted to our use-case?

You are just doubling down on protecting your argument. I operate LLMs in many conversational modes where it does ask clarifying questions, probing questions, baseline determining questions. It takes at most one sentence in the prompt to get them to act this way.

Could you share your prompt to get it to ask clarifying questions? I'm wondering if it would work in custom instructions.

Re: LLMs get lost in multi-turn conversation

#67

Post by root_axis »

Sharlin wrote: post #20
Terr_ wrote: post #17

Earlier quoted context omitted.

> inability to self-reflect IMO the One Weird Trick for LLMs is recognizing that there's no real entity, and that users are being tricked into a suspended-disbelief story. In most cases cases you're contributing text-lines for a User-character in a movie-script document, and the LLM algorithm is periodically triggered to autocomplete incomplete lines for a Chatbot character. You can have an interview with a vampire D…

This is a tired semantic argument that does not bring any insight into the discussion. A token-predictor could still be trained to predict the tokens “I’m not sure what you mean because of points x, y, and z; could you elaborate?”

It could be trained to say that, but it's not exactly clear how you would reinforce the absence of certain training data in order to emit that response accurately, rather than just based on embedding proximity.

Re: LLMs get lost in multi-turn conversation

#68

Post by olalonde »

skydhash wrote: post #51
olalonde wrote: post #32

Earlier quoted context omitted.

Recently, Gemini helped me fix a bug in a PPP driver (Zephyr OS) without prior knowledge of PPP or even driver development really. I would copy-paste logs of raw PPP frames in HEX and it would just decode everything and explain the meaning of each bytes. In about an hour, I knew enough about PPP to fix the bug and submit a patch. https://g.co/gemini/share/7edf8fa373fe

Or you could just read the PPP RFC [0]. I’m not saying that your approach is wrong. But most LLM workflows are either brute forcing the solution, or seeking a local minima to be stuck in. It’s like doing thousands of experiments of objects falling to figure out gravity while there’s a physics textbooks nearby. [0]: https://datatracker.ietf.org/doc/html/rfc1661

Ironically, I could’ve read all 50 pages of that RFC and still missed the actual issue. What really helped was RFC 1331[0], specifically the "Async-Control-Character-Map" section.

That said, I’m building a product - not a PPP driver - so the quicker I can fix the problem and move on, the better.

[0] https://datatracker.ietf.org/doc/html/rfc1331

Re: LLMs get lost in multi-turn conversation

#69

Post by roywiggins »

Sharlin wrote: post #20
Terr_ wrote: post #17

Earlier quoted context omitted.

> inability to self-reflect IMO the One Weird Trick for LLMs is recognizing that there's no real entity, and that users are being tricked into a suspended-disbelief story. In most cases cases you're contributing text-lines for a User-character in a movie-script document, and the LLM algorithm is periodically triggered to autocomplete incomplete lines for a Chatbot character. You can have an interview with a vampire D…

This is a tired semantic argument that does not bring any insight into the discussion. A token-predictor could still be trained to predict the tokens “I’m not sure what you mean because of points x, y, and z; could you elaborate?”

Anthropic found that it Claude will pretend that it used the "standard" way to do addition- add the digits, carry the 1, etc- but the pattern of activations showed it using a completely different algorithm. So these things can role play as introspecting- they come up with plausible post-hoc explanations for their output- but they are still just pretending, so they will get it wrong.

So you can teach a model to sometimes ask for clarification, but will it actually have insight into when it really needs it, or will it just interject for clarification more or less at random? These models have really awful insight into their own capabilities, ChatGPT eg insists to me that it can read braille, and then cheerfully generates a pure hallucination.

Re: LLMs get lost in multi-turn conversation

#70

Post by ezst »

CobrastanJorji wrote: post #62
Benjammer wrote: post #2

It's nice to see a paper that confirms what anyone who has practiced using LLM tools already knows very well, heuristically. Keeping your context clean matters, "conversations" are only a construct of product interfaces, they hurt the quality of responses from the LLM itself, and once your context is "poisoned" it will not recover, you need to start fresh with a new chat.

An interesting little example of this problem is initial prompting, which is effectively just a permanent, hidden context that can't be cleared. On Twitter right now, the "Grok" bot has recently begun frequently mentioning "White Genocide," which is, y'know, odd. This is almost certainly because someone recently adjusted its prompt to tell it what its views on white genocide are meant to be, which for a perfect chatb…

The heck??
Post reply on HN