LLMs can neither understand nor hallucinate. All LLMs are just picking tokens based on probability. So doesn't matter how plausible the outputs look, the reasons lead to the output are absolutely NOT what we expect them to be. But such ugly fact cannot be admitted or the party would be stopped.
LLMs Will Always Hallucinate, and We Need to Live with This
201–210 of 274 posts
Re: LLMs Will Always Hallucinate, and We Need to Live with This
#202Earlier quoted context omitted.
I find it helpful to distinguish between bugs and design flaws. A bug is caused by a poorly implemented version of the design (or a literal bug in the system). Fixing a bug requires identifying where the system varies from the design and bringing it into alignment with the design. A design flaw is a case where the idealized system as conceived by the engineers is incapable of fully solving the problem statement. Fixi…
99% is way too low for production use.
It's OK for cases, when humans are worse and/or much more expensive.
Re: LLMs Will Always Hallucinate, and We Need to Live with This
#203> By establishing the mathematical certainty of hallucinations, we challenge the prevailing notion that they can be fully mitigated Having a mathematical proof is nice, but honestly this whole misunderstanding could have been avoided if we'd just picked a different name for the concept of "producing false information in the course of generating probabilistic text". "Hallucination" makes it sound like something is goi…
"Hallucinations" just means that occasionally the LLM is wrong. The same is true of people, and I still find people extremely helpful.
But unlike LLMs, a person is capable of reflecting and realizing that they made a mistake. They are also capable of correcting so that they don't make the same mistake again. This is why fallible humans are still useful, and fallible LLMs are not.
Re: LLMs Will Always Hallucinate, and We Need to Live with This
#204Earlier quoted context omitted.
Human readable, makes grammatical sense, and wrong . And no, that's often not subjective.
It's very subjective. An LLM could return statements like "Global warming is real and man-made", and it also could produce a result like "Global warming is a hoax", and it's definitely up to the reader as to whether the LLM is "hallucinating". It doesn't matter how readable or grammatically correct the LLM is, it's still up to the reader to call bullshit, or not.
If you ask about opinions, sure. Because there are no "true" opinions.
If you ask about the capital of France, any answer but "Paris" is objectively wrong, whether given by a human or LLM.
Re: LLMs Will Always Hallucinate, and We Need to Live with This
#205> By establishing the mathematical certainty of hallucinations, we challenge the prevailing notion that they can be fully mitigated Having a mathematical proof is nice, but honestly this whole misunderstanding could have been avoided if we'd just picked a different name for the concept of "producing false information in the course of generating probabilistic text". "Hallucination" makes it sound like something is goi…
Yes, exactly, it’s a post-facto value judgment, not a precise term. If I understand the meaning of the word, “hallucination” is all the model does . If it happens to hallucinate something we think is objectively true, we just decide not to call that a “hallucination”. But there’s literally no functional difference between that case and the case of the model saying something that’s objectively false, or something whos…
Excellent point. I did not think about hallucination in this manner before.
Re: LLMs Will Always Hallucinate, and We Need to Live with This
#206Oh, not that again. Didn't we see this argument about three weeks ago.
A 100% correct LLM may be impossible. A LLM checker that produces a confidence value may be possible. We sure need one. Although last week's proposal for one wasn't very good.
When someone says something practical can't be done because of the halting problem, they're probably going in the wrong direction.
The authors are all from something called "UnitedWeCare", which offers "AI-Powered Holistic Mental Health Solutions". Not sure what to make of that.
Re: LLMs Will Always Hallucinate, and We Need to Live with This
#207The way that LLMs hallucinate now seems to have everything to do with the way in which they represent knowledge. Just look at the cost function. It's called log likelihood for a reason. The only real goal is to produce a sequence of tokens that are plausible in the most abstract sense, not consistent with concepts in a sound model of reality. Consider that when models hallucinate, they are still doing what we trained…
You can never completely solve the problem because it's mathematically undecideable, which you probably didn't need this preprint to intuit. That said, a better question is whether you can get good enough performance or not.
Re: LLMs Will Always Hallucinate, and We Need to Live with This
#208Just from the way this paper is written (badly, all kinds of LaTeX errors), my belief that something meaningful was proved here, that some nice mathematical theory has been developed, is low. Example: The first 10 pages are meaningless bla
Sorry but you're just wrong. There are issues but the paper is written well enough. The content (whether this is really a novel enough idea) is debateable because anyone could have told you that LLMs aren't going to develop the halting algorithm.
Re: LLMs Will Always Hallucinate, and We Need to Live with This
#209Humans do it too. We just call it “being wrong”
Re: LLMs Will Always Hallucinate, and We Need to Live with This
#210I prefer confabulate over hallucinate.
Confabulate - To fill in gaps in one's memory with fabrications that one believes to be facts.
Hallucinate - To wander; to go astray; to err; to blunder; -- used of mental processes
Confabulation sounds a lot more like what LLMs actually do.