Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs Will Always Hallucinate, and We Need to Live with This

arxiv.org

Post reply on HN
Print view View original

41–50 of 274 posts

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#41

Post by rvz »

feverzsj wrote: post #14

Maybe, it's time for the bubble to burst.

But but before that we need to achieve what we call "AGI" first.

Even before that we need to define it, first and the reality is no-one knows what "AGI" even is. Thus it could be anything.

The fact that Sam doesn't believe that AGI has been "achieved" yet even after GPT-3.5, ChatGPT, GPT-4 (multi-modal), and with o1 (Strawberry) suggests that what AGI really means is to capture the creation and work of billions, raise hundreds of billions of dollars and for everyone to be on their UBI based scheme, whilst they enrich themselves as the bubble continues.

Seems like the hallucinations are an excuse to say that AGI has not yet been achieved. So it time to raise billions more money on training, energy costs on inference all for it to continue to hallucinate.

Once all the value has been captured by OpenAI and the insiders cash out, THEN they would want the bubble to burst with 95% of AI startups disappearing (Except OpenAI).

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#42

Post by wrs »

lolinder wrote: post #17

> By establishing the mathematical certainty of hallucinations, we challenge the prevailing notion that they can be fully mitigated Having a mathematical proof is nice, but honestly this whole misunderstanding could have been avoided if we'd just picked a different name for the concept of "producing false information in the course of generating probabilistic text". "Hallucination" makes it sound like something is goi…

Yes, exactly, it’s a post-facto value judgment, not a precise term. If I understand the meaning of the word, “hallucination” is all the model does. If it happens to hallucinate something we think is objectively true, we just decide not to call that a “hallucination”. But there’s literally no functional difference between that case and the case of the model saying something that’s objectively false, or something whose objective truth is unknown or undefinable.

I haven’t read the paper yet, but if they resolve this definition usefully, that would be a good contribution.

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#43

Post by User23 »

jampekka wrote: post #7

I'm of the opinion that the current architectures are fundamentally ridden with "hallucinations" that will severely limit their practical usage (including very much what the hype thinks they could do). But this article puts an impossible limit to what it is to "not-hallucinate". It essentially restates well known fundamental limitations of formal systems and mechanistic computation and then presents the trivial resul…

C.S. Peirce, who is known for characterizing abductive reasoning and had a considerable on John Sowa’s old school AI work, had an interesting take on this. I can’t fully do it justice, but essentially he held that both matter and mind are real, but aren’t dual. Rather, there is a smooth and continuous transition between the two.

However, whatever the nature of mind and matter really is, we have convincing evidence of human beings creating meaning in symbols by a process Peirce called semiosis. We lack a properly formally described semiotic, although much interesting mathematical applied philosophy has been done in the space (and frankly a ton of bullshit in the academy calls itself semiotic too). Until we can do that, we will probably have great difficulty producing an automaton that can perform semiosis. So, for now, there certainly remains a qualitative difference between the capabilities of humans and LLMs.

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#44

Post by skzv »

ninetyninenine wrote: post #32

Incomplete training data is kind of a pointless thing to measure. Isn’t incomplete data the whole point of learning in general? The reason why we have machine learning is because data was incomplete. If we had complete data we don’t need ml. We just build a function that maps the input to output based off the complete data. Machine learning is about filling in the gaps based off of a prediction. In fact this is what…

Yes, but it also makes a huge difference whether we are asking the model to interpolate or extrapolate.

Generally speaking, models perform much better on the former task, and have big problems with the latter.

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#45

Post by Terr_ »

lolinder wrote: post #17

> By establishing the mathematical certainty of hallucinations, we challenge the prevailing notion that they can be fully mitigated Having a mathematical proof is nice, but honestly this whole misunderstanding could have been avoided if we'd just picked a different name for the concept of "producing false information in the course of generating probabilistic text". "Hallucination" makes it sound like something is goi…

"So we built a blind-guessing machine, but how can we tweak it so that its blind guesses always happen to be good?"

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#46

Post by digger495 »

panarky wrote: post #6
mxwsn wrote: post #4

OK - there's always a nonzero chance of hallucination. There's also a non-zero chance that macroscale objects can do quantum tunnelling, but no one is arguing that we "need to live with this" fact. A theoretical proof of the impossibility of reaching 0% probability of some event is nice, but in practice it says little about whether we can exponentially decrease the probability of it happening or not to effectively mi…

Exactly. LLMs will sometimes be inaccurate. So are humans. When LLMs are clearly better than humans for specific use cases, we don't need 100% perfection. Autonomous cars will sometimes cause accidents. So do humans. When AVs are clearly safer than humans for specific driving scenarios, we don't need 100% perfection.

LLMs will always have some degree of inaccuracy.

FtFY.

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#48

Post by nickpsecurity »

leobg wrote: post #38

Isn’t hallucination just the result of speaking out loud the first possible answer to the question you’ve been asked? A human does not do this. First of all, most questions we have been asked before. We have made mistakes in answering them before, and we remember these, so we don’t repeat them. Secondly, we (at least some of us) think before we speak. We have an initial reaction to the question, and before expressing…

Our brains also seem to tie our thoughts to observed reality in some way. The parts that do sensing and reasoning interact with the parts that handle memory. Different types of memory exist to handle trade offs. Memory of what makes sense also grows in strength compared to random things we observed.

The LLM’s don’t seem to be doing these things. Their design is weaker than the brain on mitigating hallucinations.

For brain-inspired research, I’d look at portions of the brain that seem to be abnormal in people with hallucinations. Then, models of how they work. Then, see if we can apply that to LLM’s.

My other idea was models of things like the hippocampus applied to NN’s. That’s already being done by a number of researchers, though.

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#49

Post by anthk »

tourmalinetaco wrote: post #26
talldayo wrote: post #8

Earlier quoted context omitted.

> When AVs are clearly safer than humans for specific driving scenarios, we don't need 100% perfection. People didn't stop refining the calculator once it was fast enough to beat a human. It's reasonable to expect absolute idempotent perfection from a robot designed to manufacture text.

Maybe, down the line. The calculator went through a long period of perfecting until it became as powerful as they are today. It’s only natural LLMs will also take time. And much like calculators moving from stepped drums, to vacuum tubes, to finally transistors, the way we build LLMs are sure to change. Although I’m not quite sure idempotence is something LLMs are capable of.

A Scientific HP calculator from late 80's was powerful enough to cover most of Engineering classes.

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#50

Post by davesque »

The way that LLMs hallucinate now seems to have everything to do with the way in which they represent knowledge. Just look at the cost function. It's called log likelihood for a reason. The only real goal is to produce a sequence of tokens that are plausible in the most abstract sense, not consistent with concepts in a sound model of reality.

Consider that when models hallucinate, they are still doing what we trained them to do quite well, which is to at least produce a text that is likely. So they implicitly fall back onto more general patterns in the training data i.e. grammar and simple word choice.

I have to imagine that the right architectural changes could still completely or mostly solve the hallucination problem. But it still seems like an open question as to whether we could make those changes and still get a model that can be trained efficiently.

Update: I took out the first sentence where I said "I don't agree" because I don't feel that I've given the paper a careful enough read to determine if the authors aren't in fact agreeing with me.

Post reply on HN