Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI hallucinations: Why LLMs make things up (and how to fix it)

kapa.ai

Post reply on HN
Print view View original

191–200 of 257 posts

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#191

Post by ToucanLoucan »

lolinder wrote: post #184
ToucanLoucan wrote: post #181

Earlier quoted context omitted.

Except we put up with network lag because it's an understandable, if undesirable, caveat to an otherwise useful technology. No one would ever say that because a network is sometimes slow, that it is then preferable to not have computers networked. The benefits clearly outweigh the drawbacks. This is not true for many applications of LLM. Generating legal documents, for example: it is not acceptable that it hallucinat…

The benefits of networking outweigh the drawbacks in many situations , but not all, and good engineers avoid the network in cases where the lag would be unacceptable (i.e., real-time computing applications such as assembly line software). The same applies to LLMs—even if we're never able to get the rate of failure down below 5%, there are some applications that that would be fine for. The important thing isn't that t…

I mean, agree. Now tell me which applications of LLM that are currently trending and being sold so hard by Silicon Valley meet that standard? It's not none, certainly, but it's a hell of a lot less than exist.

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#192

Post by throwawaymaths »

TZubiri wrote: post #71
throwawaymaths wrote: post #28

Completely misses the fact that a big part of the reason why llms hallucinate sp much is because there's a huge innate bias towards producing more tokens over just stopping.

The less tokens produced at inference the lower the quality of the response will be. The process of thinking for an LLM involves the use of words, which is why prompts that ask the LLM to only return the answer will cause lower quality.

We're not talking about quality, we're talking about accuracy.

In general, a model has to learn to positively say "I don't know" instead of "I don't know" being in the negative space of tokens falling into a weak distribution. The softmax selector also normalizes the token logits, so if no options are any good (all next tokens suck) it could pick randomly from a bunch of bad choices, which then locks the model into a continuation based off of that first bad choice.

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#193

Post by zmgsabst »

mort96 wrote: post #126
zmgsabst wrote: post #44

Earlier quoted context omitted.

A challenge is that it’s not easy to limit hallucinations without also limiting imagination and synthesis. In humans. But also apparently in LLMs.

Healthy humans generally have some internal model of the world against which they can judge what they're about to say. They can introspect and determine whether what they say is a guess or a statement of fact. LLMs can't.

Humans routinely misremember facts but are relatively certain those remembrances are correct.

That’s a form of minor, everyday hallucination.

If you engage in such thorough criticism and checking of every recalled fact as to eliminate that, you’ll crush your ability to synthesize or compose new content.

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#194

Post by zmgsabst »

namaria wrote: post #113
zmgsabst wrote: post #44

Earlier quoted context omitted.

A challenge is that it’s not easy to limit hallucinations without also limiting imagination and synthesis. In humans. But also apparently in LLMs.

> A challenge is that it’s not easy to limit hallucinations without also limiting imagination and synthesis. > In humans. True, but distinguishing reality from imagination is a cornerstone of mental health. And it's becoming apparent that the average person will take the confident spurious affirmations of LLMs as facts, which should call their mental health into question.

Misremembering facts isn’t a negative mental health event, yet is an example of imagination rather than recall — similar to LLMs hallucinating.

Humans imagine events all the time, without the ability to know that happened. Part of why eye-witness testimony is so unreliable.

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#195

Post by [deleted] »

GuB-42 wrote: post #157
lolinder wrote: post #8

> While the hallucination problem in LLMs is inevitable [0], they can be significantly reduced... Every article on hallucinations needs to start with this fact until we've hammered that into every "AI Engineer"'s head. Hallucinations are not a bug—they're not a different mode of operation, they're not a logic error. They're not even really a distinct kind of output. What they are is a value judgement we assign to the…

Of course they are a bug. Just that hallucination emerge from the normal function of a LLM doesn't make it "not a bug". No programmer in their right mind will call the lack of bound checking resulting in garbled output "not a bug", even though it is a totally normal thing to do from the point of view of a CPU. It is a bug and you need additional code to fix it, for example by checking for out-of-bounds condition and…

[deleted]

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#196

Post by fnordpiglet »

skydhash wrote: post #146
IWeldMelons wrote: post #142

LLM hallucinations in fact has a positive side effect too, if you are using them for learning some subject; makes you verify their claims, and finding errors in them is very rewarding.

Why not just read a book where the author is sincerely trying to teach you?

When trying to learn a subject I find being able to ask my specific questions and getting a specific answer back is helpful. I find books tend to be laborious and filled with frankly filler, often poorly indexed, and when my question isn’t covered in the book I’m left with no recourse other than googling through SEO wastelands or on topic forum questions with off topic replies. At least with LLMs they always have an answer that’s got enough of the truth in it to give me a direction, or often when I’ve gone into an area with genuinely no known answers or the thing doesn’t exist the answer is easily verified as wrong - but that process, as was pointed out above, teaches me a lot too. I actually prefer the mistakes it makes because it forces me to really learn - even to the point of giving me things to look up in the index of a book.

Treating LLMs as a single source of truth and a monolithic resource is as bad an idea as excluding them as a tool in learning.

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#197

Post by scott_w »

butlike wrote: post #188
ToucanLoucan wrote: post #181

Earlier quoted context omitted.

Except we put up with network lag because it's an understandable, if undesirable, caveat to an otherwise useful technology. No one would ever say that because a network is sometimes slow, that it is then preferable to not have computers networked. The benefits clearly outweigh the drawbacks. This is not true for many applications of LLM. Generating legal documents, for example: it is not acceptable that it hallucinat…

If it's not acceptable to hallucinate laws for writing legal documents, then writing legal documents is probably an unacceptable use case. Also, how do you mitigate a lawyer writing whatever they want (aka: hallucinating) when writing legal documents? Double-checking??

Lawyers can already be sanctioned for this: https://www.youtube.com/watch?v=oqSYljRYDEM&pp=ygUObGVnYWwgZ...

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#199

Post by joesb »

lolinder wrote: post #175
GuB-42 wrote: post #157

Earlier quoted context omitted.

Of course they are a bug. Just that hallucination emerge from the normal function of a LLM doesn't make it "not a bug". No programmer in their right mind will call the lack of bound checking resulting in garbled output "not a bug", even though it is a totally normal thing to do from the point of view of a CPU. It is a bug and you need additional code to fix it, for example by checking for out-of-bounds condition and…

A bug is defined as an unexpected defect. You can fix an unexpected defect by correcting the error in the code that led to the defect. In your example of lack of bounds checking there's a very concrete answer that will instantly fix the defect—add bounds checking. Hallucinations are not unexpected in LLMs and cannot be fixed by correcting an error in the code. Instead they are fundamental property of the computing pa…

[flagged]

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#200

Post by lurker919 »

lolinder wrote: post #190
tsujamin wrote: post #182

Earlier quoted context omitted.

I’d argue hallucinations are unexpected in LLMs by the large (non technical) number of users who use them directly, or indirectly though other services. It all depends on whose specification you’re assessing the “bugginess” against, the inference code as written, the research paper, colloquial understanding in technical circles, or how the product is pitched and presents to users.

> how the product is pitched and presents to users. And this is why I feel it's so important to fix the way we talk about hallucinations. Engineers need to be extremely clear with product owners, salespeople, and other business folks about the inherent limitations of LLMs—about the fact that certain things, like factual accuracy, may asymptotically approach 100% accuracy but will never reach it. About the fact that e…

> individual 'hallucinations' can't be treated as bugs to troubleshoot

You are wrong here - my company can fix individual responses by adding specific targeted data for the RAG prompt. So a JIRA ticket for a wrong response can be fixed in 2 days.

Post reply on HN