Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs Will Always Hallucinate, and We Need to Live with This

arxiv.org

Post reply on HN
Print view View original

131–140 of 274 posts

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#131

Post by Terr_ »

calf wrote: post #117
lolinder wrote: post #17

> By establishing the mathematical certainty of hallucinations, we challenge the prevailing notion that they can be fully mitigated Having a mathematical proof is nice, but honestly this whole misunderstanding could have been avoided if we'd just picked a different name for the concept of "producing false information in the course of generating probabilistic text". "Hallucination" makes it sound like something is goi…

Your argument makes several mistakes. First, you have just punted the validation problem of what a Normal LLM Model ought to be doing. You rhetorically declared hallucinations to be part of the normal functioning (i.e., the word "Normal" is already a value judgement). But we don't even know that - we would need theoretical proof that ALL theoretical LLMs (or neural networks as a more general argument) cannot EVER att…

> You rhetorically declared hallucinations to be part of the normal functioning (i.e., the word "Normal" is already a value judgement).

No they aren't: When you flip a coin, it landing to display heads or tails is "normal". That's no value judgement, it's just a way to characterize what is common in the mechanics.

If it landed perfectly on its edge or was snatched out of the air by a hawk, that would not be "normal", but--to introduce a value judgement--it'd be pretty dang cool.

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#132

Post by TeMPOraL »

diputsmonro wrote: post #66
wrs wrote: post #42

Earlier quoted context omitted.

Yes, exactly, it’s a post-facto value judgment, not a precise term. If I understand the meaning of the word, “hallucination” is all the model does . If it happens to hallucinate something we think is objectively true, we just decide not to call that a “hallucination”. But there’s literally no functional difference between that case and the case of the model saying something that’s objectively false, or something whos…

Exactly this, I've been saying this since the beginning. Every response is a hallucination - a probabilistic string of words divorced from any concept of truth or reality. By total coincidence, some hallucinations happen to reflect the truth, but only because the training data happened to generally be truthful sentences. Therefore, creating something that imitates a truthful sentence will often happen to also be trut…

I think you're going too far here.

> By total coincidence, some hallucinations happen to reflect the truth, but only because the training data happened to generally be truthful sentences.

It's not a "total coincidence". It's the default. Thus, the model's responses aren't "divorced from any concept of truth or reality" - the whole distribution from which those responses are pulled is strongly aligned with reality.

(Which is why people started using the term "hallucinations" to describe the failure mode, instead of "fishing a coherent and true sentence out of line noise" to describe the success mode - because success mode dominates.)

Humans didn't invent language for no reason. They don't communicate to entertain themselves with meaningless noises. Most of communication - whether spoken or written - is deeply connected to reality. Language itself is deeply connected to reality. Even the most blatant lies, even all of fiction writing, they're all incorrect or fabricated only at the surface level - the whole thing, accounting for the utterance, what it is about, the meanings, the words, the grammar - is strongly correlated with truth and reality.

So there's absolutely no coincidence that LLMs get things right more often than not. Truth is thoroughly baked into the training data, simply because it's a data set of real human communication, instead of randomly generated sentences.

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#133

Post by TeMPOraL »

rerdavies wrote: post #108
lolinder wrote: post #17

> By establishing the mathematical certainty of hallucinations, we challenge the prevailing notion that they can be fully mitigated Having a mathematical proof is nice, but honestly this whole misunderstanding could have been avoided if we'd just picked a different name for the concept of "producing false information in the course of generating probabilistic text". "Hallucination" makes it sound like something is goi…

How different things would be if the phenomenon had been called "makin' stuff up" instead. Humans make stuff up all the time, and make up far more outrageous things than AIs make up. One has to ask whether humans are really intelligent /not entirely sarcasm.

I'd prefer the phenomenon be called "saying the first thing that comes to your mind" instead, because humans do that a lot as well, and that happens to produce pretty much the same failures as LLMs do.

IOW, humans "hallucinate" exactly the same way LLMs do - they just usually don't say those things out loud, but rather it's a part of the thinking process.

See also: people who are somewhat drunk, or very excited, tend to lose inhibitions around speaking, and end up frequently just blurting whatever comes to their mind verbatim (including apologizing and backtracking and "it's not what I meant" when someone points out the nonsense).

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#134

Post by User23 »

RealityVoid wrote: post #129
smokedetector1 wrote: post #90

Earlier quoted context omitted.

You can simulate a NAND gate using balls rolling down a specially designed wood board. In theory you could construct a giant wood board with billions and billions of balls that would implement the inference step of an LLM. Do you see these balls rolling down a wood board as a form of interiority/subjective experience? If not, then why do you give it to electric currents in silicon? Just because it's faster?

Your point of disagreement is the _medium_ of computation? The same point can be made about neurons. Do you think you could have the same kind of cognitive processes you have now if you were thinking 1000x slower than you do? Speed of processing matters, especially when you have time bounds on reaction, such in real life. Another problem with balls would be the necessity of perception, that you can't really do with b…

Are you familiar with Searle's work[1] on the subject? It's fun how topical it is here. Anyhow maybe the medium doesn't matter, but the burden of proof for that claim is on you, because it's contrary to experience, intuition, and thought experiment.

[1] https://plato.stanford.edu/entries/chinese-room/

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#135

Post by inglor_cz »

lolinder wrote: post #17

> By establishing the mathematical certainty of hallucinations, we challenge the prevailing notion that they can be fully mitigated Having a mathematical proof is nice, but honestly this whole misunderstanding could have been avoided if we'd just picked a different name for the concept of "producing false information in the course of generating probabilistic text". "Hallucination" makes it sound like something is goi…

If the model starts concocting nonexistent sources, like "articles from serious newspapers that just never existed", it is definitely a malfunction for me. AFAIK this is what happened in the Jonathan Turley case.

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#136

Post by aprilthird2021 »

ta8645 wrote: post #80
aprilthird2021 wrote: post #75

Earlier quoted context omitted.

I still think OP has a point. The LLMs evolved after public use to be positioned as oracles which know so much knowledge. They were always probabilistic content generators, but people use them the way they use search engines, to retrieve info they know exists but don't exactly know. Since LLMs aren't designed for this there's a whole post process to try to make them amenable to this use case, but it will never plug t…

> but it will never plug that gap They don't have to be perfect, they just have to be better than humans. And that seems very likely to be achievable eventually.

Umm, is this true? Tons of worthless technology is better than humans at something. It has to be better than humans AND better than existing technology.

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#137

Post by davesque »

yencabulator wrote: post #127
davesque wrote: post #50

The way that LLMs hallucinate now seems to have everything to do with the way in which they represent knowledge. Just look at the cost function. It's called log likelihood for a reason. The only real goal is to produce a sequence of tokens that are plausible in the most abstract sense, not consistent with concepts in a sound model of reality. Consider that when models hallucinate, they are still doing what we trained…

I posit that when someone figures out those architectural changes, the result won't be called an LLM anymore, and the paper will be correct.

Yep, could be.

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#138

Post by wordofx »

lolinder wrote: post #17

> By establishing the mathematical certainty of hallucinations, we challenge the prevailing notion that they can be fully mitigated Having a mathematical proof is nice, but honestly this whole misunderstanding could have been avoided if we'd just picked a different name for the concept of "producing false information in the course of generating probabilistic text". "Hallucination" makes it sound like something is goi…

[flagged]

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#139

Post by mlindner »

lolinder wrote: post #17

> By establishing the mathematical certainty of hallucinations, we challenge the prevailing notion that they can be fully mitigated Having a mathematical proof is nice, but honestly this whole misunderstanding could have been avoided if we'd just picked a different name for the concept of "producing false information in the course of generating probabilistic text". "Hallucination" makes it sound like something is goi…

I agreed with you until your last sentence. Solving alignment is not a necessity for solving hallucinations even though solving hallucinations is a necessity for solving alignment.

Put another way, you can have a hypothetical model that doesn't have hallucinations and still has no alignment but you can't have alignment if you have hallucinations. Alignment is about skillful lying/refusing to answer questions and is a more complex task than simply telling no lies. (My personal opinion is that trying to solve alignment is a dystopian action and should not be attempted.)

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#140

Post by rapatel0 »

Been saying this from the beginning. Let's look at comparitor of a human result.

What is the likelihood that a junior college student with access to google will generate a "hallucination" after reading a textbook and doing some basic research on a given topic. Probably pretty high.

In our culture, we're often told to fake it till you make it. How many of us are probabilistic-ly hallucinating knowledge we've regurgitate from other sources?

Post reply on HN