Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI hallucinations: Why LLMs make things up (and how to fix it)

kapa.ai

Post reply on HN
Print view View original

201–210 of 257 posts

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#201

Post by watwut »

lolinder wrote: post #175
GuB-42 wrote: post #157

Earlier quoted context omitted.

Of course they are a bug. Just that hallucination emerge from the normal function of a LLM doesn't make it "not a bug". No programmer in their right mind will call the lack of bound checking resulting in garbled output "not a bug", even though it is a totally normal thing to do from the point of view of a CPU. It is a bug and you need additional code to fix it, for example by checking for out-of-bounds condition and…

A bug is defined as an unexpected defect. You can fix an unexpected defect by correcting the error in the code that led to the defect. In your example of lack of bounds checking there's a very concrete answer that will instantly fix the defect—add bounds checking. Hallucinations are not unexpected in LLMs and cannot be fixed by correcting an error in the code. Instead they are fundamental property of the computing pa…

Expected defects are bugs too. I totally expect half the problems in the software my company is developing. They are still bugs.

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#203

Post by empath75 »

namaria wrote: post #118
empath75 wrote: post #25

Earlier quoted context omitted.

I refuse to believe that you did any of this with any of the latest models. Gemini and Chat GPT with search are both perfectly capable of producing decent essays with accurate citations. And the 4o model is extremely good at writing python code that can accurately solve math and logic problems. I asked 4o with search to write an essay about the dangers of smoking, along with citations and quotes from the relevant sou…

You exemplify well a big problem with LLMs: When people see accurate enough output on some test question and take it as evidence that they can trust the output to any extent on areas they don't dominate.

No, that's not really the case. I don't think you should trust LLM output at all, but I think in general it's closer to the level of reliability of wikipedia than it is to producing useless bullshit.

Which is to say that it's useful, but you shouldn't trust it without double checking.

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#204

Post by PittleyDunkin »

tsujamin wrote: post #182
lolinder wrote: post #175

Earlier quoted context omitted.

A bug is defined as an unexpected defect. You can fix an unexpected defect by correcting the error in the code that led to the defect. In your example of lack of bounds checking there's a very concrete answer that will instantly fix the defect—add bounds checking. Hallucinations are not unexpected in LLMs and cannot be fixed by correcting an error in the code. Instead they are fundamental property of the computing pa…

I’d argue hallucinations are unexpected in LLMs by the large (non technical) number of users who use them directly, or indirectly though other services. It all depends on whose specification you’re assessing the “bugginess” against, the inference code as written, the research paper, colloquial understanding in technical circles, or how the product is pitched and presents to users.

> I’d argue hallucinations are unexpected in LLMs by the large (non technical) number of users who use them directly, or indirectly though other services.

People also blithely trust other humans even against all evidence that they're trustworthy. Some things just aren't fixable.

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#205

Post by PittleyDunkin »

watwut wrote: post #201
lolinder wrote: post #175

Earlier quoted context omitted.

A bug is defined as an unexpected defect. You can fix an unexpected defect by correcting the error in the code that led to the defect. In your example of lack of bounds checking there's a very concrete answer that will instantly fix the defect—add bounds checking. Hallucinations are not unexpected in LLMs and cannot be fixed by correcting an error in the code. Instead they are fundamental property of the computing pa…

Expected defects are bugs too. I totally expect half the problems in the software my company is developing. They are still bugs.

What is the utility of this sense of "bug"? If not all bugs can be fixed it seems better to toss the entire concept of a "bug" out the window as no longer useful for describing the behavior of software.

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#206

Post by PittleyDunkin »

butlike wrote: post #188
ToucanLoucan wrote: post #181

Earlier quoted context omitted.

Except we put up with network lag because it's an understandable, if undesirable, caveat to an otherwise useful technology. No one would ever say that because a network is sometimes slow, that it is then preferable to not have computers networked. The benefits clearly outweigh the drawbacks. This is not true for many applications of LLM. Generating legal documents, for example: it is not acceptable that it hallucinat…

If it's not acceptable to hallucinate laws for writing legal documents, then writing legal documents is probably an unacceptable use case. Also, how do you mitigate a lawyer writing whatever they want (aka: hallucinating) when writing legal documents? Double-checking??

> If it's not acceptable to hallucinate laws for writing legal documents

Legislators pass incoherent legislation every day. "hallucination" is the de-facto standard for human behavior (and for law).

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#207

Post by stonemetal12 »

GuB-42 wrote: post #157
lolinder wrote: post #8

> While the hallucination problem in LLMs is inevitable [0], they can be significantly reduced... Every article on hallucinations needs to start with this fact until we've hammered that into every "AI Engineer"'s head. Hallucinations are not a bug—they're not a different mode of operation, they're not a logic error. They're not even really a distinct kind of output. What they are is a value judgement we assign to the…

Of course they are a bug. Just that hallucination emerge from the normal function of a LLM doesn't make it "not a bug". No programmer in their right mind will call the lack of bound checking resulting in garbled output "not a bug", even though it is a totally normal thing to do from the point of view of a CPU. It is a bug and you need additional code to fix it, for example by checking for out-of-bounds condition and…

> LLMs naturally hallucinate, but it is not what we want, so it is a bug.

I rolled a one in D&D, it is not what I wanted, so it is a bug. Remove it from all my dice.

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#208

Post by Workaccount2 »

watwut wrote: post #201
lolinder wrote: post #175

Earlier quoted context omitted.

A bug is defined as an unexpected defect. You can fix an unexpected defect by correcting the error in the code that led to the defect. In your example of lack of bounds checking there's a very concrete answer that will instantly fix the defect—add bounds checking. Hallucinations are not unexpected in LLMs and cannot be fixed by correcting an error in the code. Instead they are fundamental property of the computing pa…

Expected defects are bugs too. I totally expect half the problems in the software my company is developing. They are still bugs.

In real world engineering, defects are part of the design and not bugs. Really they aren't even called defects, because they are inherent in the design.

Maybe you bump your car because you stopped an inch too far. Perhaps it's because the tires on your car were from a lower performing but still in spec batch. Those tires weren't defective or bugged, but instead the product of a system with statistical outputs (manufacturing variation) rather than software-like deterministic ones (binary yes/no output).

Which goes back to OP's initial point: SWE types aren't used to working in fully statistical output environments.

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#209

Post by jrflowers »

I like that none of the suggestions address probabilistic output generation (aside from the first bullet point of section 3C, which essentially suggests that you just use a search engine instead of a language model).

TLDR: Hallucinations are inherent to the whole thing but as humans we can apply bubble gum, bandaids and prayers

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#210

Post by jrm4 »

GuB-42 wrote: post #157
lolinder wrote: post #8

> While the hallucination problem in LLMs is inevitable [0], they can be significantly reduced... Every article on hallucinations needs to start with this fact until we've hammered that into every "AI Engineer"'s head. Hallucinations are not a bug—they're not a different mode of operation, they're not a logic error. They're not even really a distinct kind of output. What they are is a value judgement we assign to the…

Of course they are a bug. Just that hallucination emerge from the normal function of a LLM doesn't make it "not a bug". No programmer in their right mind will call the lack of bound checking resulting in garbled output "not a bug", even though it is a totally normal thing to do from the point of view of a CPU. It is a bug and you need additional code to fix it, for example by checking for out-of-bounds condition and…

Not a bug at all, IMHO.

If someone puts the wrong address for their business; Google picks it up, and someone Googles and gets the wrong address, it says nothing about "bugs in software."

Post reply on HN