Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI hallucinations: Why LLMs make things up (and how to fix it)

kapa.ai

Post reply on HN
Print view View original

211–220 of 257 posts

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#211

Post by Workaccount2 »

swatcoder wrote: post #21
lolinder wrote: post #8

> While the hallucination problem in LLMs is inevitable [0], they can be significantly reduced... Every article on hallucinations needs to start with this fact until we've hammered that into every "AI Engineer"'s head. Hallucinations are not a bug—they're not a different mode of operation, they're not a logic error. They're not even really a distinct kind of output. What they are is a value judgement we assign to the…

> The trouble is that we software engineers have spent so long working in an artificially deterministic world that we're not used to designing and evaluating probabilistic quality control systems for computer output. I think that's a mischaracterization and not really accurate. As a trade, we're familiar with probabilistic/non-deterministic components and how to approach them. You were closer when you used quotes aro…

Take all computers and make it so all memory has a 0.1-5% chance of bit flipping any second (depending on cost and temperature). That this just became a fundamental truth of reality. Any bit, anywhere in memory. It would completely turn SWE work on it's head.

This is kind of how traditional engineering is, since reality is analog and everything is on a spectrum interacting with everything else all the time.

There is no simple function where you put in 1 and get out 0. Everything in reality is put in 1 +/- .25 and get out 0 +/- .25. It's the reason why the complexity of hardware is trivial compared to the complexity of software.

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#212

Post by mort96 »

zmgsabst wrote: post #193
mort96 wrote: post #126

Earlier quoted context omitted.

Healthy humans generally have some internal model of the world against which they can judge what they're about to say. They can introspect and determine whether what they say is a guess or a statement of fact. LLMs can't.

Humans routinely misremember facts but are relatively certain those remembrances are correct. That’s a form of minor, everyday hallucination. If you engage in such thorough criticism and checking of every recalled fact as to eliminate that, you’ll crush your ability to synthesize or compose new content.

No, that's not hallucination.

In a human, there is a distinction between "this is information I truly think I know, my intention is to state a true fact about the world" and "this is something I don't know so I made something up". That distinction doesn't exist in LLMs. The fact that humans can be mistaken is a completely different issue.

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#213

Post by mdaniel »

lurker919 wrote: post #200
lolinder wrote: post #190

Earlier quoted context omitted.

> how the product is pitched and presents to users. And this is why I feel it's so important to fix the way we talk about hallucinations. Engineers need to be extremely clear with product owners, salespeople, and other business folks about the inherent limitations of LLMs—about the fact that certain things, like factual accuracy, may asymptotically approach 100% accuracy but will never reach it. About the fact that e…

> individual 'hallucinations' can't be treated as bugs to troubleshoot You are wrong here - my company can fix individual responses by adding specific targeted data for the RAG prompt. So a JIRA ticket for a wrong response can be fixed in 2 days.

Wow, that sounds great: just have every customer who interacts with your LLM come back to the site in 2 days to get the real answer to their question. How can I invest?

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#214

Post by snowwrestler »

tsujamin wrote: post #182
lolinder wrote: post #175

Earlier quoted context omitted.

A bug is defined as an unexpected defect. You can fix an unexpected defect by correcting the error in the code that led to the defect. In your example of lack of bounds checking there's a very concrete answer that will instantly fix the defect—add bounds checking. Hallucinations are not unexpected in LLMs and cannot be fixed by correcting an error in the code. Instead they are fundamental property of the computing pa…

I’d argue hallucinations are unexpected in LLMs by the large (non technical) number of users who use them directly, or indirectly though other services. It all depends on whose specification you’re assessing the “bugginess” against, the inference code as written, the research paper, colloquial understanding in technical circles, or how the product is pitched and presents to users.

This is why “fit for purpose” is such a useful idea.

Because it gives you two ends from which to approach the business challenge. You can improve the fitness—the functionality itself. But you can also adjust the purpose—what people expect it to do.

I think a lot of the concerns about LLMs come down to unrealistic expectations: oracles, Google killers, etc.

Google has problems finding and surfacing good info. LLMs are way better at that… but they err in the opposite direction. They are great at surfacing fake info too! So they need to be thought of (marketed) in a different way.

Their promise needs to be better aligned with how the technology actually works. Which is why it’s helpful to emphasize that “hallucinations” are a fundamental attribute, not an easily fixed mistake.

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#215

Post by vunderba »

HeatrayEnjoyer wrote: post #18
lawlessone wrote: post #9

Earlier quoted context omitted.

They probably have a letter counting tool added to it now. that it just knows to call when asked to do this. you ask it the number of letters and it sends those words off to another tool to count instances of L, but they didn't add a placement one so it's still guessing those. edit: corrected some typos and phrasing. Maybe we'll reach a point where the LLM's are just tool calling models and not really giver their own…

There are only 5 tools it has available to call, and that isn't one of them. A GitHub (forgot the url) stays up to date with the latest dumped system instructions.

I can't speak to all LLMs, but OpenAI has a built-in python interpreter. Assuming it recognizes the problem as "tokenization counting", it doesn't need a dedicated tool.

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#216

Post by snowwrestler »

lurker919 wrote: post #200
lolinder wrote: post #190

Earlier quoted context omitted.

> how the product is pitched and presents to users. And this is why I feel it's so important to fix the way we talk about hallucinations. Engineers need to be extremely clear with product owners, salespeople, and other business folks about the inherent limitations of LLMs—about the fact that certain things, like factual accuracy, may asymptotically approach 100% accuracy but will never reach it. About the fact that e…

> individual 'hallucinations' can't be treated as bugs to troubleshoot You are wrong here - my company can fix individual responses by adding specific targeted data for the RAG prompt. So a JIRA ticket for a wrong response can be fixed in 2 days.

It's important to understand that you're addressing the problem by adding a layer on top of the core technology, to mitigate or mask how it actually works.

At scale, your solution looks like bolting an expert system on top of the LLM. Which is something that some researchers and companies are actually working on.

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#218

Post by js8 »

mike_hearn wrote: post #136
emil_sorensen wrote: post #130

Earlier quoted context omitted.

That's a great point. Reminds me of the "feature, not a bug" Karpathy tweet [0]. [0]: https://x.com/karpathy/status/1733299213503787018?lang=en

... which is linked to from the article ;) He's right but do people really misunderstand this? I think it's pretty clear that the issue is one of over-creativity. The hallucination problem is IMHO at heart two things that the fine article itself doesn't touch on: 1. The training sets contain few examples of people expressing uncertainty because the social convention on the internet is that if you don't know the answe…

I like your analogy with the child. There are different types of human discourse. There is a "helpful free man" discourse where you try to reach the truth. There is a "creative child" discourse where you are play with the world and trying out weird things. There is also a "slave mindset" discourse where you blindly follow orders to satisfy the master, regardless of your own actual opinion on the matter.

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#219

Post by mdaniel »

madiator wrote: post #97

For the specific form of hallucination, which is called grounded factuality, we have trained a pretty good model that can detect if a claim is supported by a context. This is super useful for RAG. More info at https://bespokelabs.ai/bespoke-minicheck .

Your playground pre-populated example isn't doing you any favors, and the "examples" folder linked to on curator's GitHub would be better served by showing areas where your model shines, not "generate a poem" which hardly has any factuality to it. I don't have any earthly idea what camel.py is trying to showcase with respect to your model's capabilities

I am open to the fact that maybe the value your service provides is in spitting out a percentage, even if it is - itself - hallucinated. But, hey, it's a metric that can be monitored

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#220

Post by SilasX »

lolinder wrote: post #175
GuB-42 wrote: post #157

Earlier quoted context omitted.

Of course they are a bug. Just that hallucination emerge from the normal function of a LLM doesn't make it "not a bug". No programmer in their right mind will call the lack of bound checking resulting in garbled output "not a bug", even though it is a totally normal thing to do from the point of view of a CPU. It is a bug and you need additional code to fix it, for example by checking for out-of-bounds condition and…

A bug is defined as an unexpected defect. You can fix an unexpected defect by correcting the error in the code that led to the defect. In your example of lack of bounds checking there's a very concrete answer that will instantly fix the defect—add bounds checking. Hallucinations are not unexpected in LLMs and cannot be fixed by correcting an error in the code. Instead they are fundamental property of the computing pa…

I've found it very helpful to make the following distinction:

Spec: Do X in situation Y.

Correctness bug: It doesn't do X in situation Y.

Fitness-for-purpose (FFP) bug: It does X in situation Y, but, knowing this, you decide you don't actually want it to do X in situation Y.

Hallucination is an FFP bug.

Post reply on HN