Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs Will Always Hallucinate, and We Need to Live with This

arxiv.org

Post reply on HN
Print view View original

151–160 of 274 posts

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#151

Post by MichaelZuo »

vladms wrote: post #149
aeternum wrote: post #128

Earlier quoted context omitted.

LLMs do now have a concept of truth now since much of the RLHF is focused on making them more accurate and true. I think the problem is that humanity has a poor concept of truth. We think of most things as true or not true when much of our reality is uncertain due to fundamental limitations or because we often just don't know yet. During covid for example humanity collectively hallucinated the importance of disinfect…

I think taking decisions based on different risk models is not a hallucination. To the extreme: if during covid someone would live completely off grid (no contact with anyone) would have greatly reduced infection risk, but I would have found the risk model unreasonable. The problem with LLM-s is that they don't "model" what they are not capable off (the training set is what they know). So it is harder for them to say…

Or to put it more concisely, LLMs behave similar to a superintelligent midwit.

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#152

Post by zdragnar »

TeMPOraL wrote: post #132
diputsmonro wrote: post #66

Earlier quoted context omitted.

Exactly this, I've been saying this since the beginning. Every response is a hallucination - a probabilistic string of words divorced from any concept of truth or reality. By total coincidence, some hallucinations happen to reflect the truth, but only because the training data happened to generally be truthful sentences. Therefore, creating something that imitates a truthful sentence will often happen to also be trut…

I think you're going too far here. > By total coincidence, some hallucinations happen to reflect the truth, but only because the training data happened to generally be truthful sentences. It's not a "total coincidence". It's the default. Thus, the model's responses aren't "divorced from any concept of truth or reality" - the whole distribution from which those responses are pulled is strongly aligned with reality. (W…

The problem - as defined by how end users understand it - is that the model itself doesn't know the difference, and will proclaim bullshit with the same level of confidence that it does accurate information.

That's how you end up with grocery store chatbots recommending mixing ammonia and bleach for a cocktail, or lawyers using chatbots to cite entirely fictional case law before a judge in court.

Nothing that comes out of an LLM can be implicitly trusted, so your default assumption must be that everything it gives you needs verification from another source.

Telling people "the truth is baked in" is just begging for a disaster.

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#153

Post by sholladay »

lolinder wrote: post #17

> By establishing the mathematical certainty of hallucinations, we challenge the prevailing notion that they can be fully mitigated Having a mathematical proof is nice, but honestly this whole misunderstanding could have been avoided if we'd just picked a different name for the concept of "producing false information in the course of generating probabilistic text". "Hallucination" makes it sound like something is goi…

It’s still a bug, even if it’s the only way the system can behave as currently designed. I agree that “hallucination” is a poor term for it, though. For medication, we call a bug a “side effect” even though it’s really just a chemical interaction which, given enough information, could be predicted.

Ultimately the computer only does what we tell it to do. That has always been the case and probably always will be, just as we are the result of our inputs.

As with most bugs, I think to solve hallucinations we will need to better understand the input and its interactions within the system.

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#154

Post by darby_nine »

lolinder wrote: post #17

> By establishing the mathematical certainty of hallucinations, we challenge the prevailing notion that they can be fully mitigated Having a mathematical proof is nice, but honestly this whole misunderstanding could have been avoided if we'd just picked a different name for the concept of "producing false information in the course of generating probabilistic text". "Hallucination" makes it sound like something is goi…

Cannot agree enough. Hallucination is a terrible name for this behavior.

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#155

Post by darby_nine »

sholladay wrote: post #153
lolinder wrote: post #17

> By establishing the mathematical certainty of hallucinations, we challenge the prevailing notion that they can be fully mitigated Having a mathematical proof is nice, but honestly this whole misunderstanding could have been avoided if we'd just picked a different name for the concept of "producing false information in the course of generating probabilistic text". "Hallucination" makes it sound like something is goi…

It’s still a bug, even if it’s the only way the system can behave as currently designed. I agree that “hallucination” is a poor term for it, though. For medication, we call a bug a “side effect” even though it’s really just a chemical interaction which, given enough information, could be predicted. Ultimately the computer only does what we tell it to do. That has always been the case and probably always will be, just…

> It’s still a bug

Only if it's actually possible to remove this behavior. I don't think there's any evidence of that at this point.

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#156

Post by darby_nine »

elif wrote: post #112
gwervc wrote: post #31

Earlier quoted context omitted.

This comment should be pinned at the top of any LLM-related comment section.

Nah it's quite pedantic to say that 'this neologism does not encapsulate the meaning it's meant to' This is the nature of language evolution. Everyone knows what hallucination means with respect to AI, without trying to confer to its definition the baggage of a term used for centuries as a human psychology term.

> This is the nature of language evolution.

Only if it sticks. Hallucination is such an unnatural term for the phenomenon I would be surprised to see it stick.

> Everyone knows what hallucination means with respect to AI

This is false. It took me months to realize this just meant "output incoherent with reality" rather than an issue with training—the natural place for perceptual errors to occur.

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#157

Post by PaulStatezny »

ahepp wrote: post #101
wrs wrote: post #42

Earlier quoted context omitted.

Yes, exactly, it’s a post-facto value judgment, not a precise term. If I understand the meaning of the word, “hallucination” is all the model does . If it happens to hallucinate something we think is objectively true, we just decide not to call that a “hallucination”. But there’s literally no functional difference between that case and the case of the model saying something that’s objectively false, or something whos…

maybe hallucination is all cognition is, and humans are just really good at it?

Both of those terms have precise meanings. They're not the same thing. Summarized --

Cognition: acquiring knowledge and understanding through thought and the senses.

Hallucination: An experience involving the perception of something not present.

With those definitions in mind, hallucination can be defined as false-cognition that is not based in reality. It's not cognition because cognition grants knowledge based on truth and hallucination leads the subject to believe lies.

In other words, "humans are just really good at hallucination" rejects the notion that we're able to perceive actual reality with our senses.

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#158

Post by aeternum »

vladms wrote: post #149
aeternum wrote: post #128

Earlier quoted context omitted.

LLMs do now have a concept of truth now since much of the RLHF is focused on making them more accurate and true. I think the problem is that humanity has a poor concept of truth. We think of most things as true or not true when much of our reality is uncertain due to fundamental limitations or because we often just don't know yet. During covid for example humanity collectively hallucinated the importance of disinfect…

I think taking decisions based on different risk models is not a hallucination. To the extreme: if during covid someone would live completely off grid (no contact with anyone) would have greatly reduced infection risk, but I would have found the risk model unreasonable. The problem with LLM-s is that they don't "model" what they are not capable off (the training set is what they know). So it is harder for them to say…

Exactly, we think of reasoning as knowing the answer but the real key to the enlightenment and age of reason was admitting that we don't know instead of making things up. All those myths are just human hallucinations.

Humans taught themselves not to hallucinate by changing their reward function. Experimentation and observation was valued over the experts of the time and pure philosophy, even over human-generated ideas.

I don't see any reason that wouldn't also work with LLMs. We rewarded them for next-token prediction without regard for truth, but now many variants are being trained or fine-tuned with rewards focused on truth and correct answers. Perplexity and xAI for example.

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#159

Post by mrbungie »

User23 wrote: post #134
RealityVoid wrote: post #129

Earlier quoted context omitted.

Your point of disagreement is the _medium_ of computation? The same point can be made about neurons. Do you think you could have the same kind of cognitive processes you have now if you were thinking 1000x slower than you do? Speed of processing matters, especially when you have time bounds on reaction, such in real life. Another problem with balls would be the necessity of perception, that you can't really do with b…

Are you familiar with Searle's work[1] on the subject? It's fun how topical it is here. Anyhow maybe the medium doesn't matter, but the burden of proof for that claim is on you, because it's contrary to experience, intuition, and thought experiment. [1] https://plato.stanford.edu/entries/chinese-room/

Really out-of-ignorance: Is 'proof' the right word here? A more substantial philosophical counter-argument may be needed, but proof sounds weird in these "metaphysical" (for now) discussions.

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#160

Post by throwawaymaths »

LLMs hallucinate because probs -> tokens erase confidence values and it's difficult to assign confidences to strings of tokens, especially if you don't know where to start and stop counting (one word? one sentence?)

Is there a reason to believe this is not solvable as literally an API change? The necessary data are all there.

Post reply on HN