Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs Will Always Hallucinate, and We Need to Live with This

arxiv.org

Post reply on HN
Print view View original

111–120 of 274 posts

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#111

Post by jrflowers »

pessimizer wrote: post #100
jrflowers wrote: post #81

Earlier quoted context omitted.

> Isn’t hallucination just the result of speaking out loud the first possible answer to the question you’ve been asked? No. > In fact, if you observe your thinking… There is no reason to believe that LLMs should be compared to human minds other than our bad and irrational tendency towards anthropomorphizing everything. > So, to evaluate the intelligence of an LLM based on its first “gut reaction” to a prompt is proba…

> LLMs do not have guts Just going to ignore the scare quotes then? > do not experience time None of us experience time. Time is a way to describe cause and effect, and change. LLMs have a time when they have been invoked with a prompt, and a time when they have generated output based on that prompt. LLMs don't experience anything, they're computer programs, but we certainly experience LLMs taking time. When we run m…

> Just going to ignore the scare quotes then?

Yep. The analogy is bad even with that punctuation.

> None of us experience time.

That is not true and would only be worthy of discussion if we had agreed that comparing human experience to LLMs predicting tokens was worthwhile (which I emphatically have not done)

> You don't have to think computer programs have souls to believe that running algorithms over human languages to produce free output that is comprehensible and convincing to humans requires comparisons to humans.

This is true. You also don’t have to think that comparing this software to humans is required. That’s a belief that a person can hold, but holding it strongly does not make it an immutable truth.

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#112

Post by elif »

gwervc wrote: post #31
lolinder wrote: post #17

> By establishing the mathematical certainty of hallucinations, we challenge the prevailing notion that they can be fully mitigated Having a mathematical proof is nice, but honestly this whole misunderstanding could have been avoided if we'd just picked a different name for the concept of "producing false information in the course of generating probabilistic text". "Hallucination" makes it sound like something is goi…

This comment should be pinned at the top of any LLM-related comment section.

Nah it's quite pedantic to say that 'this neologism does not encapsulate the meaning it's meant to'

This is the nature of language evolution. Everyone knows what hallucination means with respect to AI, without trying to confer to its definition the baggage of a term used for centuries as a human psychology term.

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#113

Post by pointlessone »

lolinder wrote: post #17

> By establishing the mathematical certainty of hallucinations, we challenge the prevailing notion that they can be fully mitigated Having a mathematical proof is nice, but honestly this whole misunderstanding could have been avoided if we'd just picked a different name for the concept of "producing false information in the course of generating probabilistic text". "Hallucination" makes it sound like something is goi…

Confabulation is the term I’ve seen used a few times. I think it reflects what’s going on in LLMs better.

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#114

Post by lolinder »

BurningFrog wrote: post #87
lolinder wrote: post #17

> By establishing the mathematical certainty of hallucinations, we challenge the prevailing notion that they can be fully mitigated Having a mathematical proof is nice, but honestly this whole misunderstanding could have been avoided if we'd just picked a different name for the concept of "producing false information in the course of generating probabilistic text". "Hallucination" makes it sound like something is goi…

"Hallucinations" just means that occasionally the LLM is wrong. The same is true of people, and I still find people extremely helpful.

People constantly make this mistake, so just to clarify: absolutely nothing about what I just said implies that llms are not helpful.

Having an accurate mental model for what a tool is doing does not preclude seeing its value, but it does preclude getting caught up in unrealistic hype.

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#116

Post by imchillyb »

leptons wrote: post #107
BurningFrog wrote: post #87

Earlier quoted context omitted.

"Hallucinations" just means that occasionally the LLM is wrong. The same is true of people, and I still find people extremely helpful.

Except the LLM didn't deliver a "wrong" result, it delivered text that is human readable and makes grammatical sense. Whether or not the information contained in the text is "wrong" is subjective, and the reader gets to decide if it's factual or not. If the LLM delivered unreadable gibberish, then that could be considered "wrong", but there is no "hallicinating" going on with LLMs. That's an anthropomorphism that is…

Human beings have a tendency to prefer comforting lies over uncomfortable truths.

"The truth may set you free, but first it's really gonna piss you off." -G.S.

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#117

Post by calf »

lolinder wrote: post #17

> By establishing the mathematical certainty of hallucinations, we challenge the prevailing notion that they can be fully mitigated Having a mathematical proof is nice, but honestly this whole misunderstanding could have been avoided if we'd just picked a different name for the concept of "producing false information in the course of generating probabilistic text". "Hallucination" makes it sound like something is goi…

Your argument makes several mistakes.

First, you have just punted the validation problem of what a Normal LLM Model ought to be doing. You rhetorically declared hallucinations to be part of the normal functioning (i.e., the word "Normal" is already a value judgement). But we don't even know that - we would need theoretical proof that ALL theoretical LLMs (or neural networks as a more general argument) cannot EVER attain a certain probabilistic distribution. This is a theoretical computer science problem and remains an open problem.

So the second mistake is your probabilistic reductionism. It is true that LLMs, neural nets, and human brains alike are based on probabilistic computations. But the reasonable definition of a Hallucination is stronger than that - it needs to capture the notion that the probabilistic errors are way too extreme compared to the space of possible correct answers. An example of this is that Humans and LLMs get Right Answers and Wrong Answers in qualitatively very different ways. A concrete example of that is that Humans can demonstrate correctly the sequence of a power set (an EXP-TIME problem), but LLMs theoretically cannot ever do so. Yet both Humans and LLMs are probabilistic, we are made of chemicals and atoms.

Thirdly, the authors' thesis is that mitigation is impossible. It is not some "lens" where mitigation is equal to alignment, in fact one should use their thesis to debunk the notion that Alignmnent is an attainable problem at all. It is formally unsolvable and should be rendered as a absurd as someone claiming prima facie that the Halting Problem is solvable.

Finally, the meta issue is that the AI field is full of people who know zip about theoretical computer science. The vast majority of CS graduates have had maybe 1-2 weeks on Turing machines; an actual year-long course at the sophomore-senior level on theoretical computer science is Optional and for mathematically mature students who wish to concentrate in it. So the problem arises is a matter of a language and conceptual gap between two subdisciplines, the AI community and the TCS community. So you see lots of people believing in very simplistic arguments for or against some AI issue without a strong theoretical grounding that while CS itself has, but is not by default taught to undergraduates.

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#118

Post by lolinder »

elif wrote: post #112
gwervc wrote: post #31

Earlier quoted context omitted.

This comment should be pinned at the top of any LLM-related comment section.

Nah it's quite pedantic to say that 'this neologism does not encapsulate the meaning it's meant to' This is the nature of language evolution. Everyone knows what hallucination means with respect to AI, without trying to confer to its definition the baggage of a term used for centuries as a human psychology term.

Neologism undersells what this term is being used for. It's a technical term of art that's created its own semantic category in LLM research that separates "text generated that is factually inaccurate according to ${sources}" from "text generated that is morally repugnant to ${individuals}" or "text generated that ${governments} want to censor".

These three categories are entirely identical at a technological level, so I think it's entirely reasonable to flag that serious LLM researchers are treating them as distinct categories of problems when they're fundamentally not at all distinct. This isn't just a case of linguistic pedantry, this is a case of the language actively impeding a proper understanding of the problem by the researchers who are working on that problem.

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#119

Post by kristiandupont »

jrflowers wrote: post #81
leobg wrote: post #38

Isn’t hallucination just the result of speaking out loud the first possible answer to the question you’ve been asked? A human does not do this. First of all, most questions we have been asked before. We have made mistakes in answering them before, and we remember these, so we don’t repeat them. Secondly, we (at least some of us) think before we speak. We have an initial reaction to the question, and before expressing…

> Isn’t hallucination just the result of speaking out loud the first possible answer to the question you’ve been asked? No. > In fact, if you observe your thinking… There is no reason to believe that LLMs should be compared to human minds other than our bad and irrational tendency towards anthropomorphizing everything. > So, to evaluate the intelligence of an LLM based on its first “gut reaction” to a prompt is proba…

>> Isn’t hallucination just the result of speaking out loud the first possible answer to the question you’ve been asked?

>No.

Not literally, but it's certainly comparable.

>There is no reason to believe that LLMs should be compared to human minds

There is plenty of reason to do that. They are not the same, but that doesn't mean it's useless to look at the similarities that do exist.

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#120

Post by AnimalMuppet »

leptons wrote: post #107
BurningFrog wrote: post #87

Earlier quoted context omitted.

"Hallucinations" just means that occasionally the LLM is wrong. The same is true of people, and I still find people extremely helpful.

Except the LLM didn't deliver a "wrong" result, it delivered text that is human readable and makes grammatical sense. Whether or not the information contained in the text is "wrong" is subjective, and the reader gets to decide if it's factual or not. If the LLM delivered unreadable gibberish, then that could be considered "wrong", but there is no "hallicinating" going on with LLMs. That's an anthropomorphism that is…

Human readable, makes grammatical sense, and wrong. And no, that's often not subjective.
Post reply on HN