Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs get lost in multi-turn conversation

arxiv.org

Post reply on HN
Print view View original

161–170 of 272 posts

Re: LLMs get lost in multi-turn conversation

#161

Post by dkdbejwi383 »

Sharlin wrote: post #20
Terr_ wrote: post #17

Earlier quoted context omitted.

> inability to self-reflect IMO the One Weird Trick for LLMs is recognizing that there's no real entity, and that users are being tricked into a suspended-disbelief story. In most cases cases you're contributing text-lines for a User-character in a movie-script document, and the LLM algorithm is periodically triggered to autocomplete incomplete lines for a Chatbot character. You can have an interview with a vampire D…

This is a tired semantic argument that does not bring any insight into the discussion. A token-predictor could still be trained to predict the tokens “I’m not sure what you mean because of points x, y, and z; could you elaborate?”

How would an LLM “know” when it isn’t sure? Their baseline for truth is competent text, they don’t have a baseline for truth based on observed reality. That’s why they can be “tricked” into things like “Mr Bean is the president of the USA”

Re: LLMs get lost in multi-turn conversation

#162

Post by timdiggerm »

jsnider3 wrote: post #158
root_axis wrote: post #67

Earlier quoted context omitted.

It could be trained to say that, but it's not exactly clear how you would reinforce the absence of certain training data in order to emit that response accurately, rather than just based on embedding proximity.

Seems easy. Have a set of vague requests and train it to ask for clarification instead of guessing.

How does it identify what's vague?

Re: LLMs get lost in multi-turn conversation

#163

Post by rrr_oh_man »

voidspark wrote: post #154
rrr_oh_man wrote: post #153

Earlier quoted context omitted.

typical patterns to look out for: - "Should I now give you the complete [result], fulfilling [all your demands]?" - "Just say [go] and I will do it" - "Do you want either [A, B, or C]" - "In [5-15] minutes I will give you the complete result" ...

> "Do you want either [A, B, or C]" That's an example of what I'm talking about. Watch the reasoning process produce multiple options. That's what it is trained to do. That is problem solving, not "engagement". It requires more compute, not less. You see that more with the expensive models. > "In [5-15] minutes I will give you the complete result" I haven't seen that before and I don't see how it's relevant.

> That's an example of what I'm talking about. Watch the reasoning process produce multiple options. That's what it is trained to do. That is problem solving, not "engagement". It requires more compute, not less. You see that more with the expensive models.

Fair point. Thanks for standing your ground and arguing so matter-of-factly with me! Appreciate it.

Re: LLMs get lost in multi-turn conversation

#164

Post by aleksituk »

This is very interesting and I like the conversation about not only the technology itself, but also about the importance of thinking about the interface as a user experience and where / how it fits the paradigm.

We've been working on a lot of data processing and generation tasks. We've been doing this using an API primarily, but sometimes I end up testing creating data in a chat window and I first chat through what the requirements are for the data analysis / processing and then once I'm done I would like the whole conversation to be then summarised into basically a one-prompt process so that I can re-use it (because I can't really process new inputs via the chat).

Even when you do manage to get it down to a single prompt you can use in a chat and then ask the chat to just keep producing new data (like imagine a blog post in certain style if the base content is given as input and I'm making like 20 of them). If you produce these in the chat, there's notable benefits in that if something is wrong with the blog post the chat suggests, you can immediately edit it. The trouble is that the context window starts becoming so big that the chat starts to forget what the original instruction is and eventually you do have to just create a new chat.

One way to solve for this is having a chat with selective memory where you keep a task in memory, but you have the chat forget/not-include all the generated data in the context so that it stays clean, but only bring it to the context if the user refers to it.

Has anyone else done data processing types of tasks in chats and had issues like this? Are there some other tools to use or tricks to do in chats?

Re: LLMs get lost in multi-turn conversation

#165

Post by micromacrofoot »

lenkite wrote: post #134
CobrastanJorji wrote: post #62

Earlier quoted context omitted.

An interesting little example of this problem is initial prompting, which is effectively just a permanent, hidden context that can't be cleared. On Twitter right now, the "Grok" bot has recently begun frequently mentioning "White Genocide," which is, y'know, odd. This is almost certainly because someone recently adjusted its prompt to tell it what its views on white genocide are meant to be, which for a perfect chatb…

Probably because it is now learning from a lot of videos posted on X by misc right-wingers showing rallying cries of South African politicians like Julius Malema, Paul Mashatile etc. Not very odd. As merely 3 of over a dozen examples: https://x.com/DefiantLs/status/1922213073957327219 https://x.com/PPC4Liberty/status/1922650016579018855 https://x.com/News24/status/1920909178236776755

nah, llms don't learn like this — they specifically added it to the system prompt

Re: LLMs get lost in multi-turn conversation

#166

Post by Drunkfoowl »

tmountain wrote: post #159

I often ask the LLM for a concise summary of the discussion so far—formatted as a prompt. I then edit it appropriately and use it to start a new conversation without the baggage. I have found this to be a very effective technique, but I imagine it will be automated sometime soon.

[dead]

Re: LLMs get lost in multi-turn conversation

#167

Post by WhitneyLand »

Any reason to not call bullshit on this paper?

One of the biggest developments in language models over the last year has been test-time reasoning (aka inference scaling or “thinking”). Most vendors tested offer such a model. It’s plausible it could make a huge difference here, and they did not bother to test it or even mention it?

Things like COT and planning can really affect this and those are just a couple of things that happen automatically in more advanced models.

Seems like it wouldn’t have been hard to add this to the experiment, but they could’ve called it out in a “Limitations” or “Future Work” section. Or at least a single sentence like “We did not test chain-of-thought prompting, which may mitigate some of these issues”.

Re: LLMs get lost in multi-turn conversation

#168

Post by mdemare »

bigcat12345678 wrote: post #46
sitkack wrote: post #39

Earlier quoted context omitted.

You are just doubling down on protecting your argument. I operate LLMs in many conversational modes where it does ask clarifying questions, probing questions, baseline determining questions. It takes at most one sentence in the prompt to get them to act this way.

> It takes at most one sentence in the prompt to get them to act this way. What is this one sentence you are using? I am struggling to elicite clarification behavior form llms

"Any questions before you start coding?"

Re: LLMs get lost in multi-turn conversation

#170

Post by ben_w »

dkdbejwi383 wrote: post #161
Sharlin wrote: post #20

Earlier quoted context omitted.

This is a tired semantic argument that does not bring any insight into the discussion. A token-predictor could still be trained to predict the tokens “I’m not sure what you mean because of points x, y, and z; could you elaborate?”

How would an LLM “know” when it isn’t sure? Their baseline for truth is competent text, they don’t have a baseline for truth based on observed reality. That’s why they can be “tricked” into things like “Mr Bean is the president of the USA”

The answer is the same as how the messy bag of chemistry that is the human brain "knows" when it isn't sure:

Badly, and with great difficulty, so while it can just about be done, even then only kinda.

Post reply on HN