Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs Will Always Hallucinate, and We Need to Live with This

arxiv.org

Post reply on HN
Print view View original

91–100 of 274 posts

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#91

Post by majormajor »

durumu wrote: post #58
lolinder wrote: post #17

> By establishing the mathematical certainty of hallucinations, we challenge the prevailing notion that they can be fully mitigated Having a mathematical proof is nice, but honestly this whole misunderstanding could have been avoided if we'd just picked a different name for the concept of "producing false information in the course of generating probabilistic text". "Hallucination" makes it sound like something is goi…

I think there's a useful distinction between plausible-seeming text that is wrong in some subtle way, vs text that is completely fabricated to match a superficial output format, and the latter is what I wish people used "hallucination" to mean. A clear example of this is when you ask an LLM for some sources, with ISBNs, and it just makes up random titles and ISBNs that it knows full well do not correspond with realit…

How does a model "know full well" that it output a fake ISBN?

It's been trained that sources look like plausible-titles + random numbers.

It's been trained that when challenged it should say "oh sorry I can't do this."

Are those things actually distinct?

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#92

Post by ein0p »

leobg wrote: post #38

Isn’t hallucination just the result of speaking out loud the first possible answer to the question you’ve been asked? A human does not do this. First of all, most questions we have been asked before. We have made mistakes in answering them before, and we remember these, so we don’t repeat them. Secondly, we (at least some of us) think before we speak. We have an initial reaction to the question, and before expressing…

Humans totally do this if their prefrontal cortex shuts down due to fight or flight response. See eg stage fright or giving bullshit answers in leetcode style interviews.

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#93

Post by nativeit »

lolinder wrote: post #17

> By establishing the mathematical certainty of hallucinations, we challenge the prevailing notion that they can be fully mitigated Having a mathematical proof is nice, but honestly this whole misunderstanding could have been avoided if we'd just picked a different name for the concept of "producing false information in the course of generating probabilistic text". "Hallucination" makes it sound like something is goi…

I don’t know who/how the term was initially coined in this context, but I’m concerned that the things that make it inaccurate are also, perhaps counterintuitively, things that serve the interests of those who would overstate the capabilities of LLMs, and seek to cloud their true nature (along with inherent limitations) to investors and potential buyers. As you already pointed out, the term implies that the problems that are represented are temporary “bugs”, rather than symptoms of the underlying nature of the technology itself.

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#94

Post by Kerb_ »

SoftTalker wrote: post #64
diggan wrote: post #20

Earlier quoted context omitted.

> fundamentally ridden with "hallucinations" that will severely limit their practical usage On the other hand, a LLM that got rid of "hallucinations" is basically just a thing that copy-paste at that point. The interesting properties from LLMs comes from the fact that it can kind of make things up but still make them believable.

> can kind of make things up but still make them believable This is the definition of a bullshitter , by the way.

Or, you know, fiction writer. Some of us like little stories.

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#95

Post by GuB-42 »

leobg wrote: post #38

Isn’t hallucination just the result of speaking out loud the first possible answer to the question you’ve been asked? A human does not do this. First of all, most questions we have been asked before. We have made mistakes in answering them before, and we remember these, so we don’t repeat them. Secondly, we (at least some of us) think before we speak. We have an initial reaction to the question, and before expressing…

No, if I ask a human about something he doesn't know, the first thing he will think about is not a made up answer, it is "I don't know". It actually takes effort to make up a story, and without training we tend to be pretty bad at it. Some people do it naturally, but it is considered a disorder.

For LLMs, there is no concept of "not knowing", they will just write something that best matches their training data, and since there is not much "I don't know" in their training data, it is not a natural answer.

For example, I asked for a list of bars in a small city the LLM clearly didn't know much about, and gave me a nice list with names, addresses, phone numbers, etc... all hallucinated. Try to ask a normal human to give you a list of bars in a city he doesn't know well enough, and force him to answer something plausible, no "I don't know". Eventually, especially if he knows a lot about bars, you will get an answer, but it absolutely won't be his first thought, he will probably need to think hard about it.

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#96

Post by NemoNobody »

gus_massa wrote: post #52
leobg wrote: post #38

Isn’t hallucination just the result of speaking out loud the first possible answer to the question you’ve been asked? A human does not do this. First of all, most questions we have been asked before. We have made mistakes in answering them before, and we remember these, so we don’t repeat them. Secondly, we (at least some of us) think before we speak. We have an initial reaction to the question, and before expressing…

> A human does not do this. You obviously had never asked me anything. (Specialy tech questions while drinking a cup of cofee.) If I had a cent for every wrong answer, I'd be already a millionair.

Why?? To defend AI you used yourself as an example of how we can also be that dumb too.

I don't understand. Your example isn't true - what the OP posted is the human condition regarding this particular topic. You, as a human being obviously kno better than to blurt out the first thing that pop into your head - you even have different preset iterations of acceptable things to blurt in certain situations solely to avoid saying the wrong thing like - I'm sorry for your loss. Thoughts and prayers" and stuff like "Yes, Boss" or all the many rules of politeness, all of that is second nature to you, a prevents from blurting shit out.

Lastly, how do dumb questions in the mornings with coffee at a tech meeting in any way compare to an AI hallucination??

Did you ever reply with information that you completely made up, has seemingly little to do with the question and doesn't appear to make any logical or reasonable sense as to why that's your answer or how you even got there??

That's clearly not the behavior of an "awake" or sentient thing. That is perhaps the simplest way for normal people to "get it" is by realizing what a hallucination is and that their toddler is likely more capable of comprehending context.

You dismissed a plainly stated and correct position, with self depreciating humor - for why?

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#97

Post by tsimionescu »

renjimen wrote: post #63
lolinder wrote: post #17

> By establishing the mathematical certainty of hallucinations, we challenge the prevailing notion that they can be fully mitigated Having a mathematical proof is nice, but honestly this whole misunderstanding could have been avoided if we'd just picked a different name for the concept of "producing false information in the course of generating probabilistic text". "Hallucination" makes it sound like something is goi…

Maybe with vanilla LLMs, but new LLM training paradigms include post-training with the explicit goal of avoiding over-confident answers to questions the LLM should not be confident about answering. So hallucination is a malfunction, just like any overconfident incorrect prediction by a model.

The only time the LLM can be somewhat confident of its answer is when it is reproducing verbatim text from its training set. In any other circumstance, it has no way of knowing if the text it produced is true or not, because fundamentally it only knows if it's a likely completion of its input.

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#99

Post by Moru »

gwervc wrote: post #31
lolinder wrote: post #17

> By establishing the mathematical certainty of hallucinations, we challenge the prevailing notion that they can be fully mitigated Having a mathematical proof is nice, but honestly this whole misunderstanding could have been avoided if we'd just picked a different name for the concept of "producing false information in the course of generating probabilistic text". "Hallucination" makes it sound like something is goi…

This comment should be pinned at the top of any LLM-related comment section.

It should be part of every AI related story on the news. Just like they keep saying "X, formerly Twitter".

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#100

Post by pessimizer »

jrflowers wrote: post #81
leobg wrote: post #38

Isn’t hallucination just the result of speaking out loud the first possible answer to the question you’ve been asked? A human does not do this. First of all, most questions we have been asked before. We have made mistakes in answering them before, and we remember these, so we don’t repeat them. Secondly, we (at least some of us) think before we speak. We have an initial reaction to the question, and before expressing…

> Isn’t hallucination just the result of speaking out loud the first possible answer to the question you’ve been asked? No. > In fact, if you observe your thinking… There is no reason to believe that LLMs should be compared to human minds other than our bad and irrational tendency towards anthropomorphizing everything. > So, to evaluate the intelligence of an LLM based on its first “gut reaction” to a prompt is proba…

> LLMs do not have guts

Just going to ignore the scare quotes then?

> do not experience time

None of us experience time. Time is a way to describe cause and effect, and change. LLMs have a time when they have been invoked with a prompt, and a time when they have generated output based on that prompt. LLMs don't experience anything, they're computer programs, but we certainly experience LLMs taking time. When we run multiple stages and techniques, each depending on the output of a previous stage, those are time.

So when somebody says "gut reaction" they're trying to get you to compare the straight probabilistic generation of text to your instinctive reaction to something. They're asking you to use introspection and ask yourself if you review that first instinctive reaction i.e. have another stage afterwards that relies on the result of the instinctive reaction. If you do, then asking for LLMs to do well in one pass, rather than using the first pass to guide the next passes, is asking for superhuman performance.

I feel like this is too obvious to be explaining. Anthropomorphizing things is worth bitching about, but anthropomorphizing human languages and human language output is necessary and not wrong. You don't have to think computer programs have souls to believe that running algorithms over human languages to produce free output that is comprehensible and convincing to humans requires comparisons to humans. Otherwise, you might as well be lossy compressing music without referring to ears, or video without referring to eyes.

Post reply on HN