Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs Will Always Hallucinate, and We Need to Live with This

arxiv.org

Post reply on HN
Print view View original

221–230 of 274 posts

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#221

Post by kaoD »

aeternum wrote: post #128
fumeux_fume wrote: post #121

Earlier quoted context omitted.

Ok, but I think it would be more productive to educate people that LLMs have no concept of truth rather than insist they use the term "hallucinate" in an unintuitive way.

LLMs do now have a concept of truth now since much of the RLHF is focused on making them more accurate and true. I think the problem is that humanity has a poor concept of truth. We think of most things as true or not true when much of our reality is uncertain due to fundamental limitations or because we often just don't know yet. During covid for example humanity collectively hallucinated the importance of disinfect…

> LLMs do now have a concept of truth now since much of the RLHF is focused on making them more accurate and true.

Is it? I thought RLHF was mostly focused on making them (1) generate text that looks like a conversation/chat/assistant (2) ensure alignment i.e. censor it (3) make them profusely apologize to set up a facade that makes them look like they care at all.

I don't think one can RLHF the truth because there's no concept of truth/falsehood anywhere in the process.

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#222

Post by vanviegen »

SirMaster wrote: post #219
diputsmonro wrote: post #66

Earlier quoted context omitted.

Exactly this, I've been saying this since the beginning. Every response is a hallucination - a probabilistic string of words divorced from any concept of truth or reality. By total coincidence, some hallucinations happen to reflect the truth, but only because the training data happened to generally be truthful sentences. Therefore, creating something that imitates a truthful sentence will often happen to also be trut…

Maybe they shouldn’t have mixed truthful data with obviously untruthful data in the same training data set? Why not make a model only from truthful data? Like exclude all fiction for example.

That wouldn't prevent hallucination. An LLM doesn't know what it doesn't know. It will always try to come up with a response that sounds plausible, based on its knowledge or lack thereof.

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#223

Post by AStonesThrow »

elif wrote: post #112
gwervc wrote: post #31

Earlier quoted context omitted.

This comment should be pinned at the top of any LLM-related comment section.

Nah it's quite pedantic to say that 'this neologism does not encapsulate the meaning it's meant to' This is the nature of language evolution. Everyone knows what hallucination means with respect to AI, without trying to confer to its definition the baggage of a term used for centuries as a human psychology term.

"Hallucination" is derogatory and insulting when aimed at normal people who hear normal voices which don't necessarily belong to corporeal beings, or originate in the natural world. Labeling "hallucinations" and pathologizing, then medicating them, constitutes assault and bigotry.

"Hallucination" applied to inanimate and non-sentient software is insulting and presumptive on a different level. I'm fine with "confabulation".

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#224

Post by nmeagent »

ruthmarx wrote: post #213
nmeagent wrote: post #170

Earlier quoted context omitted.

> Trying to explain that different species can't procreate like that resulted in him pointing to the fact that other people believed it in the comments as proof. Those two species can't interbreed apparently, but considering the number of species that can [1] produce hybrid offspring, some even from different families, it is reasonable to forgive people for entertaining the possibility. [1] https://en.m.wikipedia.org…

I don't think it's remotely reasonable. The list you refer to, which I don't need to click on as I'm already familiar with it, is animals within the same family, e.g. bi cats. Raccoons are not any type of feline, and this should be basic knowledge for any adult in any western country who grew up there and went to school.

There are at least a couple of examples in the article that you refuse to read that describe hybrids from different families. Sorry, but your purported basic knowledge is wrong.

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#225

Post by xkcd1963 »

lolinder wrote: post #17

> By establishing the mathematical certainty of hallucinations, we challenge the prevailing notion that they can be fully mitigated Having a mathematical proof is nice, but honestly this whole misunderstanding could have been avoided if we'd just picked a different name for the concept of "producing false information in the course of generating probabilistic text". "Hallucination" makes it sound like something is goi…

Consequently we also shouldn't use intelligence or thinking when dealing with AI

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#226

Post by drzzhan »

lolinder wrote: post #17

> By establishing the mathematical certainty of hallucinations, we challenge the prevailing notion that they can be fully mitigated Having a mathematical proof is nice, but honestly this whole misunderstanding could have been avoided if we'd just picked a different name for the concept of "producing false information in the course of generating probabilistic text". "Hallucination" makes it sound like something is goi…

True. Actually, researchers know about it. In a sense, i feel "hallucination" is a way to keep the hype up, that we could fix it in the future, given enough data and compute and money. On the other hand, "generating false information by design" is too strong and negative. At least that's what I hear around in my research community.

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#227

Post by Too »

diggan wrote: post #20
jampekka wrote: post #7

I'm of the opinion that the current architectures are fundamentally ridden with "hallucinations" that will severely limit their practical usage (including very much what the hype thinks they could do). But this article puts an impossible limit to what it is to "not-hallucinate". It essentially restates well known fundamental limitations of formal systems and mechanistic computation and then presents the trivial resul…

> fundamentally ridden with "hallucinations" that will severely limit their practical usage On the other hand, a LLM that got rid of "hallucinations" is basically just a thing that copy-paste at that point. The interesting properties from LLMs comes from the fact that it can kind of make things up but still make them believable.

Training with inappropriate data will make copy-paste equally bad with hallucinating. For example, fiction or sarcasm may be taken out of context and used as a serious output.

What's missing is sanity-checking the sources and the output.

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#228

Post by Kwpolska »

irrational wrote: post #210

I prefer confabulate over hallucinate. Confabulate - To fill in gaps in one's memory with fabrications that one believes to be facts. Hallucinate - To wander; to go astray; to err; to blunder; -- used of mental processes Confabulation sounds a lot more like what LLMs actually do.

"Bullshit" is even more appropriate.

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#229

Post by ruthmarx »

nmeagent wrote: post #224
ruthmarx wrote: post #213

Earlier quoted context omitted.

I don't think it's remotely reasonable. The list you refer to, which I don't need to click on as I'm already familiar with it, is animals within the same family, e.g. bi cats. Raccoons are not any type of feline, and this should be basic knowledge for any adult in any western country who grew up there and went to school.

There are at least a couple of examples in the article that you refuse to read that describe hybrids from different families. Sorry, but your purported basic knowledge is wrong.

I'm not 'refusing to read' it, I said I'm familiar with it because I've read it numerous times in the past.

Which examples are you referring to? The only real example seems to be fish.

In any case I was using 'family' in a loose sense, not in the stricter scientific biological hierarchy sense.

My basic knowledge is not wrong at all, because my point was that animals that far apart could not reproduce. That's it. The wiki page you linked doesn't really justify your idea that because some hybrids exist people might think any hybrid could exist.

The point is, it's frankly idiotic or at least extremely ignorant for anyone 40 years of age who grew up in the US or any developed country to think that.

I also very much doubt the people who believe a racoon could rape a cat and produce offspring are even aware of that wiki page or any of the examples on it. Hell, I doubt they even know a mule is a hybrid. Your hypothesis doesn't hold water.

Additionally, most of the examples on that page are the result of human intervention and artificial insemination, not wild encounters. Context matters.

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#230

Post by IshKebab »

GuB-42 wrote: post #95
leobg wrote: post #38

Isn’t hallucination just the result of speaking out loud the first possible answer to the question you’ve been asked? A human does not do this. First of all, most questions we have been asked before. We have made mistakes in answering them before, and we remember these, so we don’t repeat them. Secondly, we (at least some of us) think before we speak. We have an initial reaction to the question, and before expressing…

No, if I ask a human about something he doesn't know, the first thing he will think about is not a made up answer, it is "I don't know". It actually takes effort to make up a story, and without training we tend to be pretty bad at it. Some people do it naturally, but it is considered a disorder. For LLMs, there is no concept of "not knowing", they will just write something that best matches their training data, and s…

Rubbish. Humans absolutely hallucinate things, including lists of bars.

Most of them do it infrequently but it absolutely happens. Sometimes they don't even realise it (see all the research about fictitious memories and eye witness accounts).

It's definitely a problem that LLMs hallucinate all the time, but let's not pretend humans are perfect and never bullshit.

Hell, I've worked with one awful guy who did bullshit just as much as LLMs. Literally never said "I don't know". Always some answer that you were never sure was true or just entirely made up. Annoyingly it worked quite well for him - lots of people couldn't see through it.

Post reply on HN