Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs Will Always Hallucinate, and We Need to Live with This

arxiv.org

Post reply on HN
Print view View original

101–110 of 274 posts

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#101

Post by ahepp »

wrs wrote: post #42
lolinder wrote: post #17

> By establishing the mathematical certainty of hallucinations, we challenge the prevailing notion that they can be fully mitigated Having a mathematical proof is nice, but honestly this whole misunderstanding could have been avoided if we'd just picked a different name for the concept of "producing false information in the course of generating probabilistic text". "Hallucination" makes it sound like something is goi…

Yes, exactly, it’s a post-facto value judgment, not a precise term. If I understand the meaning of the word, “hallucination” is all the model does . If it happens to hallucinate something we think is objectively true, we just decide not to call that a “hallucination”. But there’s literally no functional difference between that case and the case of the model saying something that’s objectively false, or something whos…

maybe hallucination is all cognition is, and humans are just really good at it?

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#102

Post by imoverclocked »

abernard1 wrote: post #88
ninetyninenine wrote: post #32

Incomplete training data is kind of a pointless thing to measure. Isn’t incomplete data the whole point of learning in general? The reason why we have machine learning is because data was incomplete. If we had complete data we don’t need ml. We just build a function that maps the input to output based off the complete data. Machine learning is about filling in the gaps based off of a prediction. In fact this is what…

> Machine learning is about filling in the gaps based off of a prediction. I think this is a generous interpretation of network-based ML. ML was designed to solve problems. We had lots of data, and we knew large amounts of data could derive functions (networks) as opposed to deliberate construction of algorithms with GOFAI. But "intelligence" with ML as it stands now is not how humans think. Humans do not need millio…

> Humans do not need millions of examples of cats to know what a cat is.

We have evolved over time to recognize things in our environment. We also don’t need to be told that snakes are dangerous as many humans have an innate understanding of that. Our training data is partially inherited.

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#103

Post by willcipriano »

When will I see AI dialogue in video games? Imagine a RPG where instead of picking from a series of pre recorded dialogues, you could just talk to that villager. If it worked it would be mind blowing. The first studio to really pull it off in the AAA game would rake in the cash.

That seems like the lowest hanging fruit to me, like we would do that long before we have AI going over someone's medical records.

If the major game studios aren't confident enough in the tech to have it write dialogue for a Disney character for fear of it saying the wrong thing, I'm not ready for it to anything in the real world.

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#104

Post by fsndz »

We can't get rid of hallucinations. Hallucinations are a feature not a bug. A recent study by researchers Jim Waldo and Soline Boussard highlights the risks associated with this limitation. In their analysis, they tested several prominent models, including ChatGPT-3.5, ChatGPT-4, Llama, and Google’s Gemini. The researchers found that while the models performed well on well-known topics with a large body of available data, they often struggled with subjects that had limited or contentious information, resulting in inconsistencies and errors.

This challenge is particularly concerning in fields where accuracy is critical, such as scientific research, politics, or legal matters. For instance, the study noted that LLMs could produce inaccurate citations, misattribute quotes, or provide factually wrong information that might appear convincing but lacks a solid foundation. Such errors can lead to real-world consequences, as seen in cases where professionals have relied on LLM-generated content for tasks like legal research or coding, only to discover later that the information was incorrect. https://www.lycee.ai/blog/llm-hallucinations-report

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#105

Post by pessimizer »

GuB-42 wrote: post #95
leobg wrote: post #38

Isn’t hallucination just the result of speaking out loud the first possible answer to the question you’ve been asked? A human does not do this. First of all, most questions we have been asked before. We have made mistakes in answering them before, and we remember these, so we don’t repeat them. Secondly, we (at least some of us) think before we speak. We have an initial reaction to the question, and before expressing…

No, if I ask a human about something he doesn't know, the first thing he will think about is not a made up answer, it is "I don't know". It actually takes effort to make up a story, and without training we tend to be pretty bad at it. Some people do it naturally, but it is considered a disorder. For LLMs, there is no concept of "not knowing", they will just write something that best matches their training data, and s…

> No, if I ask a human about something he doesn't know, the first thing he will think about is not a made up answer, it is "I don't know".

You've just made this up, through. It's not what happens. How would somebody even know that they didn't know without trying to come up with an answer?

But maybe more convincingly, people who have brain injuries that cause them to neglect a side (i.e. not see the left or right side of things) often don't realize (without a lot of convincing) the extent to which this is happening. If you ask them to explain their unexplainable behaviors, they'll spontaneously concoct the most convincing explanation that they can.

https://en.wikipedia.org/wiki/Hemispatial_neglect

https://en.wikipedia.org/wiki/Anosognosia

People try to make things make sense. LLMs try to minimize a loss function.

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#106

Post by [deleted] »

ahepp wrote: post #101
wrs wrote: post #42

Earlier quoted context omitted.

Yes, exactly, it’s a post-facto value judgment, not a precise term. If I understand the meaning of the word, “hallucination” is all the model does . If it happens to hallucinate something we think is objectively true, we just decide not to call that a “hallucination”. But there’s literally no functional difference between that case and the case of the model saying something that’s objectively false, or something whos…

maybe hallucination is all cognition is, and humans are just really good at it?

[deleted]

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#107

Post by leptons »

BurningFrog wrote: post #87
lolinder wrote: post #17

> By establishing the mathematical certainty of hallucinations, we challenge the prevailing notion that they can be fully mitigated Having a mathematical proof is nice, but honestly this whole misunderstanding could have been avoided if we'd just picked a different name for the concept of "producing false information in the course of generating probabilistic text". "Hallucination" makes it sound like something is goi…

"Hallucinations" just means that occasionally the LLM is wrong. The same is true of people, and I still find people extremely helpful.

Except the LLM didn't deliver a "wrong" result, it delivered text that is human readable and makes grammatical sense. Whether or not the information contained in the text is "wrong" is subjective, and the reader gets to decide if it's factual or not. If the LLM delivered unreadable gibberish, then that could be considered "wrong", but there is no "hallicinating" going on with LLMs. That's an anthropomorphism that is divorced from the reality of what an LLM does. Whoever called it "hallucinating" in the context of an LLM should have their nerd credentials revoked.

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#108

Post by rerdavies »

lolinder wrote: post #17

> By establishing the mathematical certainty of hallucinations, we challenge the prevailing notion that they can be fully mitigated Having a mathematical proof is nice, but honestly this whole misunderstanding could have been avoided if we'd just picked a different name for the concept of "producing false information in the course of generating probabilistic text". "Hallucination" makes it sound like something is goi…

How different things would be if the phenomenon had been called "makin' stuff up" instead. Humans make stuff up all the time, and make up far more outrageous things than AIs make up. One has to ask whether humans are really intelligent /not entirely sarcasm.

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#109

Post by User23 »

smokedetector1 wrote: post #90
gradus_ad wrote: post #83

Earlier quoted context omitted.

Producing text is only the visible end product. The LLM is doing a whole lot behind the scenes, which is conceivably analogous to the thought space from which our own words flow.

You can simulate a NAND gate using balls rolling down a specially designed wood board. In theory you could construct a giant wood board with billions and billions of balls that would implement the inference step of an LLM. Do you see these balls rolling down a wood board as a form of interiority/subjective experience? If not, then why do you give it to electric currents in silicon? Just because it's faster?

Tangentially, Peirce has a diagrammatic logical system that's built entirely on conjunction and negation which is isomorphic to propositional logic. He also defined extensions for what we now call predicate logic and modal logic.

John Sowa annotated Peirce's tutorial and it's quite interesting[1].

[1] https://www.jfsowa.com/pubs/egtut.pdf

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#110

Post by rerdavies »

durumu wrote: post #58
lolinder wrote: post #17

> By establishing the mathematical certainty of hallucinations, we challenge the prevailing notion that they can be fully mitigated Having a mathematical proof is nice, but honestly this whole misunderstanding could have been avoided if we'd just picked a different name for the concept of "producing false information in the course of generating probabilistic text". "Hallucination" makes it sound like something is goi…

I think there's a useful distinction between plausible-seeming text that is wrong in some subtle way, vs text that is completely fabricated to match a superficial output format, and the latter is what I wish people used "hallucination" to mean. A clear example of this is when you ask an LLM for some sources, with ISBNs, and it just makes up random titles and ISBNs that it knows full well do not correspond with realit…

In fairness, they will also admit they were wrong even if they were right.
Post reply on HN