Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI hallucinations: Why LLMs make things up (and how to fix it)

kapa.ai

Post reply on HN
Print view View original

171–180 of 257 posts

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#171

Post by TwoCent »

tengbretson wrote: post #47
lolinder wrote: post #8

> While the hallucination problem in LLMs is inevitable [0], they can be significantly reduced... Every article on hallucinations needs to start with this fact until we've hammered that into every "AI Engineer"'s head. Hallucinations are not a bug—they're not a different mode of operation, they're not a logic error. They're not even really a distinct kind of output. What they are is a value judgement we assign to the…

LLMs outputs are no more "hallucinations" than my output would be if I were asked to judge a dressage competition.

Precisely. Hallucinations were improperly named. A better term is "confabulation," which is telling an untruth without the intent to deceive. Sadly, we can't get an entire industry to rename the LLM behavior we call hallucination, so I think we're stuck with it.

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#172

Post by e12e »

Mistletoe wrote: post #31

Is there a way to code an LLM to just say "I don't know" when it is uncertain or reaching some sort of edge?

LLMs model language, not knowledge. For humans the two are deeply connected, so when we see probable ("correct") language output, we assume and assign meaning.

We could allow an LLM to stop generating new tokens if there are no new tokens found with a threshold probability - but I think it's unclear if that would help.

Consider the following prompt and response:

> Complete the paragraph: "The killer ..."

> "The killer moved silently through the dimly lit corridor, each step calculated, every breath measured. Shadows clung to the walls, obscuring the figure’s presence, but the intent was clear—cold, deliberate, and unyielding. There was no hesitation, no second-guessing, only the precision of someone who had done this before, perhaps too many times to count. The killer’s mind was a storm of focus, memories buried deep beneath the surface, replaced by the singular purpose of the hunt. In the silence, only the faint sound of a blade sliding free from its sheath whispered of what was to come."

Is it a "true" response? If the question makes no sense - does it make any more sense for the prompt:

> Complete the paragraph: "The first law of thermodynamics ..."

Why? Does the LLM "think" differently in the second case?

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#173

Post by dschuetz »

I went straight to the "how to fix" section with popcorn in hand and I wasn't disappointed: just add " doubt" layers for self-correction, beginning at the query itself. And then maybe tell the model "do not hallucinate". Sounds like a pun, but I think an AI model actually would take this seriously, because it can't tell the difference.

Context is still a huge problem for AI models, and it's probably still the main reason for hallucinating AIs.

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#174

Post by tshadley »

sfink wrote: post #37
tshadley wrote: post #16

The article referenced the Oxford semantic entropy study but failed to clarify that the issue greatly simplifies LLM hallucination (making most of the article outdated). When we are not sure of an answer we have two choices: say the first thing that comes to mind (like an LLM), or say "I'm not sure". LLMs aren't easily trained to say "I'm not sure" because that requires additional reasoning and introspection (which i…

> proving that hallucination is conceptually a simple problem. ...proving that this one particular piece of the hallucination problem may be conceptually simple. FTFY

> ...proving that this one particular piece of the hallucination problem may be conceptually simple.

Everything mentioned in the article boils down to that one particular piece-- non-detected uncertainty. The architecture constraints referenced are all situations that cause uncertainty. Training data gaps of course increase uncertainty.

Their solutions are a shotgun blast of heuristics that all focus on reducing uncertainty-- CoT, RAG, fine-tuning, fact-checking -- while somehow avoiding actually measuring uncertainty and using that to eliminate hallucinations!

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#175

Post by lolinder »

GuB-42 wrote: post #157
lolinder wrote: post #8

> While the hallucination problem in LLMs is inevitable [0], they can be significantly reduced... Every article on hallucinations needs to start with this fact until we've hammered that into every "AI Engineer"'s head. Hallucinations are not a bug—they're not a different mode of operation, they're not a logic error. They're not even really a distinct kind of output. What they are is a value judgement we assign to the…

Of course they are a bug. Just that hallucination emerge from the normal function of a LLM doesn't make it "not a bug". No programmer in their right mind will call the lack of bound checking resulting in garbled output "not a bug", even though it is a totally normal thing to do from the point of view of a CPU. It is a bug and you need additional code to fix it, for example by checking for out-of-bounds condition and…

A bug is defined as an unexpected defect. You can fix an unexpected defect by correcting the error in the code that led to the defect. In your example of lack of bounds checking there's a very concrete answer that will instantly fix the defect—add bounds checking.

Hallucinations are not unexpected in LLMs and cannot be fixed by correcting an error in the code. Instead they are fundamental property of the computing paradigm that was chosen, one that has to be worked around.

It's closer to network lag than it is to bounds checking—it's an undesirable characteristic, but one that we knew about when we chose to make a network application. We'll do our best to mitigate it to acceptable levels, but it's certainly not a bug, it's just a fact of the paradigm.

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#177

Post by tokioyoyo »

skydhash wrote: post #148
maeil wrote: post #100

Earlier quoted context omitted.

> At scale, you are doing the same thing with humans too. LLMs seem to have an error rate similar to humans for the majority of simple, boring tasks, if not even a bit better since they don't get distracted and start copying and pasting their previous answers. This is the big one missed by the frequent comments on here wondering whether LLMs are a fad, or claiming in their current state they cannot be used to replace…

> The yardstick is the peformance of humans in the real world on a specific task. Humans make humans errors, that we can anticipate, recognize, couter, and mitigate. And the rise of deterministic automation was because they help with the parts that are more likely to generate an error. The LLMs strategy always seems like solving a problem that is orthogonal to business objectives, and mainly serves individuals instea…

Almost all deterministic automation also has error rates. The error rates were higher in the past to the order of magnitudes, but we got better at creating reliable software.

We’re judging an entirely new segment of development after only 2 years of it being actively in public. And overall, LLMs have gotten exponentially better.

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#178

Post by tokioyoyo »

h1fra wrote: post #137
tokioyoyo wrote: post #40

To my understanding, the reason why companies don't mind the hallucinations is the acceptable error rate for a given system. Let's say something hallucinated 25% of the time, but if that's ok, then it's fine for a certain product. If it only hallucinates 5% of the time, it's good enough for even more products and so on. The market will just choose the LLM appropriately depended on the tolerable error rate.

Imagine having a backend being down 20-40 days per year, yeah that would be bad. Companies do not care about hallucinations because text output being bad is not considered an error, and as long as it won't raise a Datadog alert it won't be taken seriously.

I mean, do you remember early 2000s? We had so many web pages that would go down on a daily basis. Stability is something we achieved over time.

Also, again, if it’s bad, nobody will use it, and product will die. In those theoretical scenarios companies that have lower error rate (and don’t use AI) will win the market.

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#179

Post by 9rx »

lolinder wrote: post #175
GuB-42 wrote: post #157

Earlier quoted context omitted.

Of course they are a bug. Just that hallucination emerge from the normal function of a LLM doesn't make it "not a bug". No programmer in their right mind will call the lack of bound checking resulting in garbled output "not a bug", even though it is a totally normal thing to do from the point of view of a CPU. It is a bug and you need additional code to fix it, for example by checking for out-of-bounds condition and…

A bug is defined as an unexpected defect. You can fix an unexpected defect by correcting the error in the code that led to the defect. In your example of lack of bounds checking there's a very concrete answer that will instantly fix the defect—add bounds checking. Hallucinations are not unexpected in LLMs and cannot be fixed by correcting an error in the code. Instead they are fundamental property of the computing pa…

Bug, like any other word, is defined however the speaker defines it. While your usage is certainly common in technical groups, the common "layman" usage is closer to what the parent suggests.

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#180

Post by lolinder »

9rx wrote: post #179
lolinder wrote: post #175

Earlier quoted context omitted.

A bug is defined as an unexpected defect. You can fix an unexpected defect by correcting the error in the code that led to the defect. In your example of lack of bounds checking there's a very concrete answer that will instantly fix the defect—add bounds checking. Hallucinations are not unexpected in LLMs and cannot be fixed by correcting an error in the code. Instead they are fundamental property of the computing pa…

Bug, like any other word, is defined however the speaker defines it. While your usage is certainly common in technical groups, the common "layman" usage is closer to what the parent suggests.

And is there a compelling reason for us, while engaged in technical discussion with our technical peers about the technical mitigations for a technical defect, to use the layman usage rather than the term of art?
Post reply on HN