Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs Will Always Hallucinate, and We Need to Live with This

arxiv.org

Post reply on HN
Print view View original

211–220 of 274 posts

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#211

Post by AceyMan »

pointlessone wrote: post #113
lolinder wrote: post #17

> By establishing the mathematical certainty of hallucinations, we challenge the prevailing notion that they can be fully mitigated Having a mathematical proof is nice, but honestly this whole misunderstanding could have been avoided if we'd just picked a different name for the concept of "producing false information in the course of generating probabilistic text". "Hallucination" makes it sound like something is goi…

Confabulation is the term I’ve seen used a few times. I think it reflects what’s going on in LLMs better.

Yes, I thought this was already the agreed upon term for those in the know. I don't know what all this HN hullabaloo is about.

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#212

Post by aptsurdist »

But humans also hallucinate.

And humans habitually stray from the “truth” too. It’s always seemed to me that getting AI to be more accurate isn’t a math problem, it’s getting AI to “care” about what is true - aka better defining what truth is- aka what sources should be cited with what weights.

We can’t even keep humans in society from believing in the stupidest conspiracy theories. When humans get their knowledge from sources indiscriminately, they also parrot stupid shit that isn’t real.

Now enter Gödel’s incompleteness Theorem: there is no perfect tie between language and reality. Super interesting. But this isn’t the issue. Or at least it’s not more of an issue for robots than it is for humans.

If/when humans deliver “accurate” results in our dialogs, it’s because we’ve been trained to care about what is “accuracy” (as defined by society’s chosen sources)

Remember that AI “doesn’t live here.” It’s swimming in a mess of noisy context without guidance for what it should care about.

IMHO, as soon as we train AI to “care” at a basic level about what we culturally agree is “true” the hallucinations will diminish to be far smaller than the hallucinations of most humans.

I’m honestly not sure if that will be a good thing or the start of something horrifying.

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#213

Post by ruthmarx »

nmeagent wrote: post #170
ruthmarx wrote: post #143

Earlier quoted context omitted.

> In my experience, humans are at least as bad at it as GPT-4, if not far worse. I had an argument with a former friend recently, because he read some comments on YouTube and was convinced a racoon raped a cat and produced some kind of hybrid offspring that was terrorizing a neighborhood. Trying to explain that different species can't procreate like that resulted in him pointing to the fact that other people believed…

> Trying to explain that different species can't procreate like that resulted in him pointing to the fact that other people believed it in the comments as proof. Those two species can't interbreed apparently, but considering the number of species that can [1] produce hybrid offspring, some even from different families, it is reasonable to forgive people for entertaining the possibility. [1] https://en.m.wikipedia.org…

I don't think it's remotely reasonable. The list you refer to, which I don't need to click on as I'm already familiar with it, is animals within the same family, e.g. bi cats.

Raccoons are not any type of feline, and this should be basic knowledge for any adult in any western country who grew up there and went to school.

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#214

Post by aptsurdist »

mrjin wrote: post #201

LLMs can neither understand nor hallucinate. All LLMs are just picking tokens based on probability. So doesn't matter how plausible the outputs look, the reasons lead to the output are absolutely NOT what we expect them to be. But such ugly fact cannot be admitted or the party would be stopped.

Human brains are also just picking tokens tho. A beautiful illusion of insight and thought in the chaos noise of information. But out of the chaos, the emergence of thought is real. It’s just not exclusive to humans.

I mean… even magicians (mentalists) can reliably hack humans into generating the next token they want you to generate.

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#215

Post by kalleboo »

leptons wrote: post #107
BurningFrog wrote: post #87

Earlier quoted context omitted.

"Hallucinations" just means that occasionally the LLM is wrong. The same is true of people, and I still find people extremely helpful.

Except the LLM didn't deliver a "wrong" result, it delivered text that is human readable and makes grammatical sense. Whether or not the information contained in the text is "wrong" is subjective, and the reader gets to decide if it's factual or not. If the LLM delivered unreadable gibberish, then that could be considered "wrong", but there is no "hallicinating" going on with LLMs. That's an anthropomorphism that is…

The term "hallucination" is in response to what the LLMs are being marketed and sold as what they are supposed to do, not how they work technologically.

It makes sense to have better terms for technical discussions, but this term is going to stick around for mainstream use.

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#216

Post by aptsurdist »

m3kw9 wrote: post #191

They did say each token is generated using probability, not certainty, given that there is a chance it produces wrong tokens

This gets at the heart of the problem. It doesn’t produce the wrong tokens. The tokens are right. It’s the data that was “wrong”. Or at least it was weighted “incorrectly” (according to the judge living outside the data with their own context they decide is true)

If you feed AI conspiracy theories and then it tells you Elvis is still alive, that’s an input problem not an algorithm problem.

Now, getting to an AI that doesn’t “hallucinate” is a little more complicated than simply filtering out “conspiracy theories” from data, but IMhO it’s not many orders of magnitudes away. Far from insurmountable in a couple Moores law cycles.

I think human brains operate on the same principal of divining next tokens. We’re just judging AI for not saying the tokens we like best even though we feed AI garbage in and don’t tell AI what it should even care about. “AI doesn’t live here” it wasn’t born “here.”

Someday soon we’ll probably give AI guard rails to respond considering the context of society (the programmers version of society), and it will probably hallucinate less than most humans.

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#217

Post by tourmalinetaco »

anthk wrote: post #49
tourmalinetaco wrote: post #26

Earlier quoted context omitted.

Maybe, down the line. The calculator went through a long period of perfecting until it became as powerful as they are today. It’s only natural LLMs will also take time. And much like calculators moving from stepped drums, to vacuum tubes, to finally transistors, the way we build LLMs are sure to change. Although I’m not quite sure idempotence is something LLMs are capable of.

A Scientific HP calculator from late 80's was powerful enough to cover most of Engineering classes.

Sure, but that doesn’t mean they haven’t improved. Try calculating 99! on a TI-59. I doubt it can do it, and I know the modern TI-30XIIS can’t do it, but my Numworks can (although doing it twice forces it to clear that section of RAM). The calculator space may be slow to improve, as most non-testing calculations have went to computers, but that doesn’t mean they’re not useful, especially with scripting languages allowing me to convert between whatever units I want or calculate anything easily.

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#218

Post by SirMaster »

tsimionescu wrote: post #195
jbm wrote: post #146

Earlier quoted context omitted.

> humanity collectively hallucinated the importance of disinfecting groceries for awhile I reject this history. I homeschooled my kids during covid due to uncertainty and even I didn't reach that level, and nor did anyone I knew in person. A very tiny number who were egged on by some YouTubers did this, including one person I knew remotely. Unsurprisingly that person was based in SV.

It's not some extremist on YouTube, disinfecting your groceries was the official recommendation of many countries worldwide, including most of Europe. I couldn't say how many people actually followed the recommendation , but I would bet it's way more than a tiny number.

This is the first I’ve even heard of people disinfecting their groceries because of Covid. Honestly that sounds rather crazy to me.

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#219

Post by SirMaster »

diputsmonro wrote: post #66
wrs wrote: post #42

Earlier quoted context omitted.

Yes, exactly, it’s a post-facto value judgment, not a precise term. If I understand the meaning of the word, “hallucination” is all the model does . If it happens to hallucinate something we think is objectively true, we just decide not to call that a “hallucination”. But there’s literally no functional difference between that case and the case of the model saying something that’s objectively false, or something whos…

Exactly this, I've been saying this since the beginning. Every response is a hallucination - a probabilistic string of words divorced from any concept of truth or reality. By total coincidence, some hallucinations happen to reflect the truth, but only because the training data happened to generally be truthful sentences. Therefore, creating something that imitates a truthful sentence will often happen to also be trut…

Maybe they shouldn’t have mixed truthful data with obviously untruthful data in the same training data set?

Why not make a model only from truthful data? Like exclude all fiction for example.

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#220

Post by leptons »

BurningFrog wrote: post #204
leptons wrote: post #172

Earlier quoted context omitted.

It's very subjective. An LLM could return statements like "Global warming is real and man-made", and it also could produce a result like "Global warming is a hoax", and it's definitely up to the reader as to whether the LLM is "hallucinating". It doesn't matter how readable or grammatically correct the LLM is, it's still up to the reader to call bullshit, or not.

If you ask about opinions, sure. Because there are no "true" opinions. If you ask about the capital of France, any answer but "Paris" is objectively wrong, whether given by a human or LLM.

Paris has not always been the capital of France. Many other cities around France have been capital.

https://en.wikipedia.org/wiki/List_of_capitals_of_France

There's practically no subject you could bring up that an LLM wouldn't "hallucinate" or give "wrong" information about given that garbage in -> garbage out, and LLMs are trained on all the garbage (as well as too many facts) they've been able to scrape. The LLM lacks the ability to reason about what century the prompt is asking about, and a guess is all it is programmed to do.

Also, if you ask 100 French citizens today what the true capital of France is, you're not always going to get "Paris" as a reply 100 times.

Post reply on HN