Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs Will Always Hallucinate, and We Need to Live with This

arxiv.org

Post reply on HN
Print view View original

181–190 of 274 posts

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#181

Post by nprateem »

wrs wrote: post #42
lolinder wrote: post #17

> By establishing the mathematical certainty of hallucinations, we challenge the prevailing notion that they can be fully mitigated Having a mathematical proof is nice, but honestly this whole misunderstanding could have been avoided if we'd just picked a different name for the concept of "producing false information in the course of generating probabilistic text". "Hallucination" makes it sound like something is goi…

Yes, exactly, it’s a post-facto value judgment, not a precise term. If I understand the meaning of the word, “hallucination” is all the model does . If it happens to hallucinate something we think is objectively true, we just decide not to call that a “hallucination”. But there’s literally no functional difference between that case and the case of the model saying something that’s objectively false, or something whos…

Exactly. Even a broken clock is right twice a day.

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#182

Post by nprateem »

theturtle32 wrote: post #130
ahepp wrote: post #101

Earlier quoted context omitted.

maybe hallucination is all cognition is, and humans are just really good at it?

In my experience, humans are at least as bad at it as GPT-4, if not far worse . In terms, specifically, of being "factually accurate" and grounded in absolute reality. Humans operate entirely in the probabilistic realm of what seems right to us based on how we were educated, the values we were raised with, our religious beliefs, etc. -- Human beings are all over the map with this.

Ok but humans aren't being hyped as this incredible new tech that's a going to lead to the singularity.

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#183

Post by [deleted] »

mewse wrote: post #166
willcipriano wrote: post #103

When will I see AI dialogue in video games? Imagine a RPG where instead of picking from a series of pre recorded dialogues, you could just talk to that villager. If it worked it would be mind blowing. The first studio to really pull it off in the AAA game would rake in the cash. That seems like the lowest hanging fruit to me, like we would do that long before we have AI going over someone's medical records. If the ma…

As a game developer, I'm absolutely not going to risk my players' engagement with the game by putting character personalities, dialog, and (therefore) plot into the hands of an AI that's going to play out differently for each player. No way, no how. That's where all my game's value is - that's what pays my rent - and I will not be handing it off to a glorified RNG.

[deleted]

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#184

Post by [deleted] »

mewse wrote: post #166
willcipriano wrote: post #103

When will I see AI dialogue in video games? Imagine a RPG where instead of picking from a series of pre recorded dialogues, you could just talk to that villager. If it worked it would be mind blowing. The first studio to really pull it off in the AAA game would rake in the cash. That seems like the lowest hanging fruit to me, like we would do that long before we have AI going over someone's medical records. If the ma…

As a game developer, I'm absolutely not going to risk my players' engagement with the game by putting character personalities, dialog, and (therefore) plot into the hands of an AI that's going to play out differently for each player. No way, no how. That's where all my game's value is - that's what pays my rent - and I will not be handing it off to a glorified RNG.

[deleted]

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#185

Post by jiggawatts »

lolinder wrote: post #17

> By establishing the mathematical certainty of hallucinations, we challenge the prevailing notion that they can be fully mitigated Having a mathematical proof is nice, but honestly this whole misunderstanding could have been avoided if we'd just picked a different name for the concept of "producing false information in the course of generating probabilistic text". "Hallucination" makes it sound like something is goi…

How sure are you that humans don’t also “simply produce plausible-sounding text”?

I’m watching my toddler learning to speak and it’s remarkably like an early LLM that outputs gibberish with nuggets of sense.

Conversely I’ve seen middle-aged professionals write formal design documents that are markedly inferior to what GPT-4 can produce. This is at every level: overall structure, semantics, and syntax. Diagrams with arrows the wrong way, labels with typos, labels on the wrong object, mixed up icons, tables with made-up content, and on and on.

Aren’t we all doing this to some degree? Half way through a sentence do you known ahead of time exactly what you’re about to say three pages later? Or are you just “completing” the next word based on the context?

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#187

Post by lolinder »

jiggawatts wrote: post #185
lolinder wrote: post #17

> By establishing the mathematical certainty of hallucinations, we challenge the prevailing notion that they can be fully mitigated Having a mathematical proof is nice, but honestly this whole misunderstanding could have been avoided if we'd just picked a different name for the concept of "producing false information in the course of generating probabilistic text". "Hallucination" makes it sound like something is goi…

How sure are you that humans don’t also “simply produce plausible-sounding text”? I’m watching my toddler learning to speak and it’s remarkably like an early LLM that outputs gibberish with nuggets of sense. Conversely I’ve seen middle-aged professionals write formal design documents that are markedly inferior to what GPT-4 can produce. This is at every level: overall structure, semantics, and syntax. Diagrams with a…

> How sure are you that humans don’t also “simply produce plausible-sounding text”?

I'm not, and I didn't say I was.

Nothing in my comment is downplaying the success and utility of LLMs. Nothing in my comment is suggesting that they won't get better or even that they won't get better than us humans. It is strictly an observation that these two things that we call different names—hallucinations and alignment—are actually equivalent and can each be expressed in terms of the other. A hallucination is a specific kind of misalignment, nothing more.

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#188

Post by NoCoooode »

lolinder wrote: post #17

> By establishing the mathematical certainty of hallucinations, we challenge the prevailing notion that they can be fully mitigated Having a mathematical proof is nice, but honestly this whole misunderstanding could have been avoided if we'd just picked a different name for the concept of "producing false information in the course of generating probabilistic text". "Hallucination" makes it sound like something is goi…

Agreed. Put simply, everything produced by an LLM today is an hallucination. Errors are simply poor hallucinations.

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#189

Post by RealityVoid »

User23 wrote: post #134
RealityVoid wrote: post #129

Earlier quoted context omitted.

Your point of disagreement is the _medium_ of computation? The same point can be made about neurons. Do you think you could have the same kind of cognitive processes you have now if you were thinking 1000x slower than you do? Speed of processing matters, especially when you have time bounds on reaction, such in real life. Another problem with balls would be the necessity of perception, that you can't really do with b…

Are you familiar with Searle's work[1] on the subject? It's fun how topical it is here. Anyhow maybe the medium doesn't matter, but the burden of proof for that claim is on you, because it's contrary to experience, intuition, and thought experiment. [1] https://plato.stanford.edu/entries/chinese-room/

My take on Searle is that he was a hack. It's possible I judge too harshly, that _I_ am a hack (the likeliest, tbh) or that I and his writing have some fundamental life outlooks different.

Regardless, I think the Chinese room experiment is bunk and proves nothing. And I fail to gather where the medium of computation steps in the Chinese room experiment. The "computer" might as well be a bunch of neurons in a petry dish.

I guess the proof will be in the pudding when we develop superhumanly intelligent AI.

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#190

Post by noobermin »

FloatArtifact wrote: post #179
lolinder wrote: post #17

> By establishing the mathematical certainty of hallucinations, we challenge the prevailing notion that they can be fully mitigated Having a mathematical proof is nice, but honestly this whole misunderstanding could have been avoided if we'd just picked a different name for the concept of "producing false information in the course of generating probabilistic text". "Hallucination" makes it sound like something is goi…

This is due to an entire field of AI /machine learning leaning into anthropomorphism shaping terminology reinforced by a narrative.

Which is totally on them. We had to suffer a decade of being told the human brain is a neural net (as if my brain was a numeric matrix, freaking ridiculous) they get to suffer when that dumb analogy comes back to bite them.
Post reply on HN