Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI hallucinations: Why LLMs make things up (and how to fix it)

kapa.ai

Post reply on HN
Print view View original

181–190 of 257 posts

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#181

Post by ToucanLoucan »

lolinder wrote: post #175
GuB-42 wrote: post #157

Earlier quoted context omitted.

Of course they are a bug. Just that hallucination emerge from the normal function of a LLM doesn't make it "not a bug". No programmer in their right mind will call the lack of bound checking resulting in garbled output "not a bug", even though it is a totally normal thing to do from the point of view of a CPU. It is a bug and you need additional code to fix it, for example by checking for out-of-bounds condition and…

A bug is defined as an unexpected defect. You can fix an unexpected defect by correcting the error in the code that led to the defect. In your example of lack of bounds checking there's a very concrete answer that will instantly fix the defect—add bounds checking. Hallucinations are not unexpected in LLMs and cannot be fixed by correcting an error in the code. Instead they are fundamental property of the computing pa…

Except we put up with network lag because it's an understandable, if undesirable, caveat to an otherwise useful technology. No one would ever say that because a network is sometimes slow, that it is then preferable to not have computers networked. The benefits clearly outweigh the drawbacks.

This is not true for many applications of LLM. Generating legal documents, for example: it is not acceptable that it hallucinate laws that do not exist. Recipes: it is not acceptable that it would tell people to make pizza with glue, or mustard gas to remove stains. Or, in my case: it is not acceptable for a developer assisting AI to hallucinate into existence libraries that are not real and not only will not solve my problem, but that will cause me to lose hours of my day trying to figure out where to get said library.

If pneumatic tires failed to hold air as often as LLM's hallucinate, we wouldn't use them. That's not to say a tire can't blow out, sure they can, happens all the time. It's about the rate of failure. Or hell, to bring it back to your metaphor, if your network experienced high latency at the rate most LLM's hallucinate, I might actually suggest you not network computers, or at the very least, I'd say you should be replaced at whatever company you work for since you're clearly unqualified to manage a network.

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#182

Post by tsujamin »

lolinder wrote: post #175
GuB-42 wrote: post #157

Earlier quoted context omitted.

Of course they are a bug. Just that hallucination emerge from the normal function of a LLM doesn't make it "not a bug". No programmer in their right mind will call the lack of bound checking resulting in garbled output "not a bug", even though it is a totally normal thing to do from the point of view of a CPU. It is a bug and you need additional code to fix it, for example by checking for out-of-bounds condition and…

A bug is defined as an unexpected defect. You can fix an unexpected defect by correcting the error in the code that led to the defect. In your example of lack of bounds checking there's a very concrete answer that will instantly fix the defect—add bounds checking. Hallucinations are not unexpected in LLMs and cannot be fixed by correcting an error in the code. Instead they are fundamental property of the computing pa…

I’d argue hallucinations are unexpected in LLMs by the large (non technical) number of users who use them directly, or indirectly though other services.

It all depends on whose specification you’re assessing the “bugginess” against, the inference code as written, the research paper, colloquial understanding in technical circles, or how the product is pitched and presents to users.

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#184

Post by lolinder »

ToucanLoucan wrote: post #181
lolinder wrote: post #175

Earlier quoted context omitted.

A bug is defined as an unexpected defect. You can fix an unexpected defect by correcting the error in the code that led to the defect. In your example of lack of bounds checking there's a very concrete answer that will instantly fix the defect—add bounds checking. Hallucinations are not unexpected in LLMs and cannot be fixed by correcting an error in the code. Instead they are fundamental property of the computing pa…

Except we put up with network lag because it's an understandable, if undesirable, caveat to an otherwise useful technology. No one would ever say that because a network is sometimes slow, that it is then preferable to not have computers networked. The benefits clearly outweigh the drawbacks. This is not true for many applications of LLM. Generating legal documents, for example: it is not acceptable that it hallucinat…

The benefits of networking outweigh the drawbacks in many situations, but not all, and good engineers avoid the network in cases where the lag would be unacceptable (i.e., real-time computing applications such as assembly line software). The same applies to LLMs—even if we're never able to get the rate of failure down below 5%, there are some applications that that would be fine for.

The important thing isn't that the rate of failure be below a specific threshold before the technology is adopted anywhere, the important thing is that engineers working on this technology have an understanding of the fundamental limitations of the computing paradigm and design accordingly—up to and including telling leadership that LLMs are a fundamentally inappropriate tool for the job.

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#186

Post by d0mine »

GuB-42 wrote: post #157
lolinder wrote: post #8

> While the hallucination problem in LLMs is inevitable [0], they can be significantly reduced... Every article on hallucinations needs to start with this fact until we've hammered that into every "AI Engineer"'s head. Hallucinations are not a bug—they're not a different mode of operation, they're not a logic error. They're not even really a distinct kind of output. What they are is a value judgement we assign to the…

Of course they are a bug. Just that hallucination emerge from the normal function of a LLM doesn't make it "not a bug". No programmer in their right mind will call the lack of bound checking resulting in garbled output "not a bug", even though it is a totally normal thing to do from the point of view of a CPU. It is a bug and you need additional code to fix it, for example by checking for out-of-bounds condition and…

The difference is that you can fix IndexError by modifying your code but no amount of prompt manipulation may fix hallucinations. For that you need solutions outside LLMs.

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#187

Post by PittleyDunkin »

GuB-42 wrote: post #157
lolinder wrote: post #8

> While the hallucination problem in LLMs is inevitable [0], they can be significantly reduced... Every article on hallucinations needs to start with this fact until we've hammered that into every "AI Engineer"'s head. Hallucinations are not a bug—they're not a different mode of operation, they're not a logic error. They're not even really a distinct kind of output. What they are is a value judgement we assign to the…

Of course they are a bug. Just that hallucination emerge from the normal function of a LLM doesn't make it "not a bug". No programmer in their right mind will call the lack of bound checking resulting in garbled output "not a bug", even though it is a totally normal thing to do from the point of view of a CPU. It is a bug and you need additional code to fix it, for example by checking for out-of-bounds condition and…

> Of course they are a bug.

A bug implies fixable behavior rather than expected behavior. An LLM making shit up is expected behavior.

> LLMs naturally hallucinate, but it is not what we want, so it is a bug.

Maybe you just don't want an LLM! This is what LLMs do. Maybe you want a decision tree or a scripted chatbot?

> And to fix it, we need to engineer solutions that prevent the hallucinations from happening, maybe resulting in an "I don't know" response that would be analogous to an error message.

I'm sure we'll figure out how to do this when we can fix the same bug in humans, too. Given that humans can't even agree when we're right or wrong—much less sense the incoherency of their own worldviews—I doubt we're going to see a solution to this in our lifetimes.

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#188

Post by butlike »

ToucanLoucan wrote: post #181
lolinder wrote: post #175

Earlier quoted context omitted.

A bug is defined as an unexpected defect. You can fix an unexpected defect by correcting the error in the code that led to the defect. In your example of lack of bounds checking there's a very concrete answer that will instantly fix the defect—add bounds checking. Hallucinations are not unexpected in LLMs and cannot be fixed by correcting an error in the code. Instead they are fundamental property of the computing pa…

Except we put up with network lag because it's an understandable, if undesirable, caveat to an otherwise useful technology. No one would ever say that because a network is sometimes slow, that it is then preferable to not have computers networked. The benefits clearly outweigh the drawbacks. This is not true for many applications of LLM. Generating legal documents, for example: it is not acceptable that it hallucinat…

If it's not acceptable to hallucinate laws for writing legal documents, then writing legal documents is probably an unacceptable use case.

Also, how do you mitigate a lawyer writing whatever they want (aka: hallucinating) when writing legal documents? Double-checking??

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#189

Post by ToucanLoucan »

butlike wrote: post #188
ToucanLoucan wrote: post #181

Earlier quoted context omitted.

Except we put up with network lag because it's an understandable, if undesirable, caveat to an otherwise useful technology. No one would ever say that because a network is sometimes slow, that it is then preferable to not have computers networked. The benefits clearly outweigh the drawbacks. This is not true for many applications of LLM. Generating legal documents, for example: it is not acceptable that it hallucinat…

If it's not acceptable to hallucinate laws for writing legal documents, then writing legal documents is probably an unacceptable use case. Also, how do you mitigate a lawyer writing whatever they want (aka: hallucinating) when writing legal documents? Double-checking??

A licensing body, and consequences for the failure to practice law correctly.

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#190

Post by lolinder »

tsujamin wrote: post #182
lolinder wrote: post #175

Earlier quoted context omitted.

A bug is defined as an unexpected defect. You can fix an unexpected defect by correcting the error in the code that led to the defect. In your example of lack of bounds checking there's a very concrete answer that will instantly fix the defect—add bounds checking. Hallucinations are not unexpected in LLMs and cannot be fixed by correcting an error in the code. Instead they are fundamental property of the computing pa…

I’d argue hallucinations are unexpected in LLMs by the large (non technical) number of users who use them directly, or indirectly though other services. It all depends on whose specification you’re assessing the “bugginess” against, the inference code as written, the research paper, colloquial understanding in technical circles, or how the product is pitched and presents to users.

> how the product is pitched and presents to users.

And this is why I feel it's so important to fix the way we talk about hallucinations. Engineers need to be extremely clear with product owners, salespeople, and other business folks about the inherent limitations of LLMs—about the fact that certain things, like factual accuracy, may asymptotically approach 100% accuracy but will never reach it. About the fact that even getting asymptotically close to 100% is extremely (most likely prohibitively) expensive. And once they've chosen a non-zero failure rate, they have to be clear about what the consequences of the chosen failure rate are.

Before engineers can communicate that to the business side, they have to have that straight in their own heads. Then they can communicate expectations with the business and ensure that they understand that once you've chosen a failure rate, individual 'hallucinations' can't be treated as bugs to troubleshoot—you need instead to have an industrial-style QC process that measures trends and reacts only when your process produces results outside of a set of well-defined tolerances.

(Yes, I'm aware that many organizations are so thoroughly broken that engineering has no influence over what business tells customers. But those businesses are hopeless anyway, and many businesses do listen to their engineers.)

Post reply on HN