LLMs get lost in multi-turn conversation

271–272 of 272 posts

Re: LLMs get lost in multi-turn conversation

#271

Post by ben_w » Thu, May 22, 2025, 7:56 AM UTC

foldr wrote: post #270

ben_w wrote: post #266

Earlier quoted context omitted.

Dunning Kruger, and also everyone who thinks they know better than domain experts. Humans who are wrong are often completely oblivious to being wrong.

Ok, but what does that have to do with LLMS?

I'm saying both human brains and LLMs are bad at knowing when they're unsure — it may not even be possible to make an LLM that's superhuman in this regard.

Most random humans have nearly constant confidence in their answers regardless of how much they know about a topic.

Hmm. Stream of consciousness thoughts here:

Problem is LLMs are very broad "System 1" thinking, no "System 2". Other kinds of model that combine LLM with a more logical module? Perhaps that could work.

And if so then I guess you could use the logical module to create tokens for an LLM to learn from? Eventually, but may be inefficient.

To work, logic module needs to know what LLM gets wrong, to create tokens for LLM to learn from; if you can do that, but fine tuning is slow and expensive and still errs, so why not keep the logic separate and just have it point out mistakes and have LLM regenerate answers a few times and if that doesn't work then say "IDK"?

Re: LLMs get lost in multi-turn conversation

#272

Post by Retric » Thu, May 22, 2025, 7:33 PM UTC

halfadot wrote: post #269

Retric wrote: post #268

Earlier quoted context omitted.

And now it’s obvious why you’re working for someone else. This time it worked, but I’ve been forced for fire people with this kind of attitude before.

You produced a passive-aggressive taunt instead of addressing the argument. For clarity: nobody was asking about your business decisions, nobody is intimidated by your story. what your personal opinions about "attitude" are is irrelevant to what's being discussed (LLMs allowing optimal time use in certain cases). Also, unless your boss made the firing decision, you weren't forced to do anything.

You’re still not getting it, not having a boss means I have a very different view of businesses decisions. Most people have an overly narrow view of tasks IMO and think speed, especially for minor issues, is vastly more important than it is.

> LLMs allowing optimal time use in certain case

I never said it was slower, I’m saying what’s the tradeoff. I’ve had this same basic conversation with multiple people, and after that failed the only real option is to remove them. Ex: If you don’t quite understand why what you wrote seemingly fixes a bug don’t commit it yet, seems to work isn’t a solution.

Could be I’m not explaining very well, but ehh fuck em.

Post reply on HN