Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs Will Always Hallucinate, and We Need to Live with This

arxiv.org

Post reply on HN
Print view View original

71–80 of 274 posts

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#71

Post by giantrobot »

bicx wrote: post #3

I treat LLMs like a fallible being, the same way I treat humans. I don’t just trust output implicitly, and I accept help with tasks knowing I am taking a certain degree of risk. Mostly, my experience has been very positive with GPT-4o / ChatGPT and GitHub copilot with that in mind. I use each constantly throughout the day.

> I treat LLMs like a fallible being, the same way I treat humans.

The issue is LLMs are not marketed in this way. They're marketed as all knowing oracles to people that have been conditioned to just accept the first result Google gives them.

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#72

Post by o11c »

jampekka wrote: post #7

I'm of the opinion that the current architectures are fundamentally ridden with "hallucinations" that will severely limit their practical usage (including very much what the hype thinks they could do). But this article puts an impossible limit to what it is to "not-hallucinate". It essentially restates well known fundamental limitations of formal systems and mechanistic computation and then presents the trivial resul…

What impresses me is frankly how bad it is.

I can't claim to have tried every model out there, but most models very quickly fail when asked to do something along the lines of "describe the interaction of 3 entities." They can usually handle 2 (up to the point where they inevitably start talking in circles - often repeating entire chunks verbatim in many models), but 3 seems utterly beyond them.

LLMs might have a role in the field of "burn money to generate usually-wrong ideas that are cheap enough to check in case there's actually a good one" though.

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#74

Post by aprilthird2021 »

paulddraper wrote: post #57
leobg wrote: post #38

Isn’t hallucination just the result of speaking out loud the first possible answer to the question you’ve been asked? A human does not do this. First of all, most questions we have been asked before. We have made mistakes in answering them before, and we remember these, so we don’t repeat them. Secondly, we (at least some of us) think before we speak. We have an initial reaction to the question, and before expressing…

> A human does not do this Have you met humans?

I have never had a human give me some of the answers an LLM has given me, and I've met humans who can't tell basically any country on a map, including the one they live in

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#75

Post by aprilthird2021 »

renjimen wrote: post #63
lolinder wrote: post #17

> By establishing the mathematical certainty of hallucinations, we challenge the prevailing notion that they can be fully mitigated Having a mathematical proof is nice, but honestly this whole misunderstanding could have been avoided if we'd just picked a different name for the concept of "producing false information in the course of generating probabilistic text". "Hallucination" makes it sound like something is goi…

Maybe with vanilla LLMs, but new LLM training paradigms include post-training with the explicit goal of avoiding over-confident answers to questions the LLM should not be confident about answering. So hallucination is a malfunction, just like any overconfident incorrect prediction by a model.

I still think OP has a point. The LLMs evolved after public use to be positioned as oracles which know so much knowledge. They were always probabilistic content generators, but people use them the way they use search engines, to retrieve info they know exists but don't exactly know.

Since LLMs aren't designed for this there's a whole post process to try to make them amenable to this use case, but it will never plug that gap

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#76

Post by pkphilip »

Hallucinations in LLM will severely affect its usage in scenarios where such hallucinations are completely unacceptable - and there are many such scenarios. This is a good thing because it will mean that human intelligence and oversight will continue to be needed.

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#77

Post by [deleted] »

leobg wrote: post #38

Isn’t hallucination just the result of speaking out loud the first possible answer to the question you’ve been asked? A human does not do this. First of all, most questions we have been asked before. We have made mistakes in answering them before, and we remember these, so we don’t repeat them. Secondly, we (at least some of us) think before we speak. We have an initial reaction to the question, and before expressing…

[deleted]

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#78

Post by Der_Einzige »

TheGeminon wrote: post #16
everdrive wrote: post #10

Earlier quoted context omitted.

One big difference is that at least some people have a healthy sense for when they may be wrong. This sort of meta-cognitive introspection is currently not possible for an LLM. For instance, let's say I asked someone "do you know the first 10 elements of the periodic table of elements?" Most people would be able to accurately say "honestly I'm not sure what comes after Helium." But an LLM will just make up some bulls…

There are ways to gauge the confidence of the LLM (token probabilities over the response, generating multiple outputs and checking consistency), but yeah that’s outside the LLM itself. You could feed the info back to the LLM as a status/message I suppose

The idea of hooking LLMs back up to themselves, i.e. giving them token prob information somehow or even giving them control over the settings they use to prompt themselves is AWESOME and I cannot believe that no one has seriously done this yet.

I've done it in some jupyter notebooks and the results are really neat, especially since LLMs can be made with a tiny bit of extra code to generate a context "timer" that they wait before they prompt themselves to respond, creating a proper conversational agent system (i.e. not the walkie talkie systems of today)

I wrote a paper that mentioned doing things like this for having LLMs act as AI art directors: https://arxiv.org/abs/2311.03716

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#79

Post by JieJie »

leobg wrote: post #38

Isn’t hallucination just the result of speaking out loud the first possible answer to the question you’ve been asked? A human does not do this. First of all, most questions we have been asked before. We have made mistakes in answering them before, and we remember these, so we don’t repeat them. Secondly, we (at least some of us) think before we speak. We have an initial reaction to the question, and before expressing…

The US had a president for eight years who was re-elected on his ability to act on his “gut reaction”s.

Not saying this is ideal, just that it isn’t the showstopper you present it as. In fact, when people talk about “human values”, it might be worth reflecting on whether this a thing we’re supposed to be protecting or expunging?

"I'm not a textbook player, I'm a gut player.” —President George W. Bush.

https://www.heraldtribune.com/story/news/2003/01/12/going-to...

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#80

Post by ta8645 »

aprilthird2021 wrote: post #75
renjimen wrote: post #63

Earlier quoted context omitted.

Maybe with vanilla LLMs, but new LLM training paradigms include post-training with the explicit goal of avoiding over-confident answers to questions the LLM should not be confident about answering. So hallucination is a malfunction, just like any overconfident incorrect prediction by a model.

I still think OP has a point. The LLMs evolved after public use to be positioned as oracles which know so much knowledge. They were always probabilistic content generators, but people use them the way they use search engines, to retrieve info they know exists but don't exactly know. Since LLMs aren't designed for this there's a whole post process to try to make them amenable to this use case, but it will never plug t…

> but it will never plug that gap

They don't have to be perfect, they just have to be better than humans. And that seems very likely to be achievable eventually.

Post reply on HN