Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI hallucinations: Why LLMs make things up (and how to fix it)

kapa.ai

Post reply on HN
Print view View original

221–230 of 257 posts

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#221

Post by jhbadger »

mike_hearn wrote: post #163
Gormo wrote: post #162

Earlier quoted context omitted.

Nope. They don't know any specific facts. The training data produces a probability matrix that reflects what words are likely to be found in relation other words, allowing it to generate novel combinations of words that are coherent and understandable. But there is no mechanism involved for determining whether those novel expressions are actually factual representations of reality.

Again, read the papers. They absolutely do know facts, and that can be seen in the activations. Your description is oversimplified. It's easy to get models to emit statistically improbable but correct sequences of words. They are not just looking at what words are near by each other, that doesn't lead to the kind of output LLMs are capable of.

Exactly. People forget that we did make systems that were just Markov chains long before LLMs, like the famous Usenet Poster "Mark V. Shaney" (created by Rob Pike of Plan 9 and Golang fame) that was trained on Usenet posts in the 1980s. You didn't need deep learning or any sort of neural nets for that. It could come up with sentences that sometimes made some sort of sense, but that was it. The oversimplified way LLMs are sometimes explained makes it sound like they are no different from Mark V. Shaney, but they obviously are.

https://en.wikipedia.org/wiki/Mark_V._Shaney

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#223

Post by SirMaster »

stonemetal12 wrote: post #207
GuB-42 wrote: post #157

Earlier quoted context omitted.

Of course they are a bug. Just that hallucination emerge from the normal function of a LLM doesn't make it "not a bug". No programmer in their right mind will call the lack of bound checking resulting in garbled output "not a bug", even though it is a totally normal thing to do from the point of view of a CPU. It is a bug and you need additional code to fix it, for example by checking for out-of-bounds condition and…

> LLMs naturally hallucinate, but it is not what we want, so it is a bug. I rolled a one in D&D, it is not what I wanted, so it is a bug. Remove it from all my dice.

What? You are telling me that when you roll a 6 sided dice you are not expecting any of the 1-6 as a result?

If a 6-sided dice produced a 7 that would be a bug.

When you rolled a dice, I would argue that you knew you wanted a random number from 1-6, not that you wanted a specific number or not a specific number. If you wanted that you wouldn't have used a dice.

When I ask an LLM to write code for me and it references a completely made up library that doesn't exist and has never existed, is this really analogous to your dice example?

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#224

Post by swatcoder »

Workaccount2 wrote: post #211
swatcoder wrote: post #21

Earlier quoted context omitted.

> The trouble is that we software engineers have spent so long working in an artificially deterministic world that we're not used to designing and evaluating probabilistic quality control systems for computer output. I think that's a mischaracterization and not really accurate. As a trade, we're familiar with probabilistic/non-deterministic components and how to approach them. You were closer when you used quotes aro…

Take all computers and make it so all memory has a 0.1-5% chance of bit flipping any second (depending on cost and temperature). That this just became a fundamental truth of reality. Any bit, anywhere in memory. It would completely turn SWE work on it's head. This is kind of how traditional engineering is, since reality is analog and everything is on a spectrum interacting with everything else all the time. There is…

That's not really engaging with the point because you're suggesting turning all of our tools into something grossly unreliable. Of course that's a radical shift from what anybody's used to and undermines every practice in the trade.

But your mistake is just reinforcing what I wrote, because its the same mistake that the "loud people" are make when they think about generative AI. They imagine it as being a wholesale replacement for how projects are implemented and even how they're built in the first place.

But the many experienced engineers looking at generative AI recognize it as one of many tools that they can turn to while building a project that fulfills their requirements. And like all their tools, it has capabilities, costs, and limitations that need to be considered. That its sometimes non-deterministic is not a new kind of cost or limitation. It's a challenging one, but not a novel one, and one just mindfully (or analytically) considers whether and how that non-determinism can be leveraged, minimized, etc. That is engineering, and it's what many of us have been doing with all sorts of tools for decades.

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#225

Post by sfink »

tshadley wrote: post #174
sfink wrote: post #37

Earlier quoted context omitted.

> proving that hallucination is conceptually a simple problem. ...proving that this one particular piece of the hallucination problem may be conceptually simple. FTFY

> ...proving that this one particular piece of the hallucination problem may be conceptually simple. Everything mentioned in the article boils down to that one particular piece-- non-detected uncertainty. The architecture constraints referenced are all situations that cause uncertainty. Training data gaps of course increase uncertainty. Their solutions are a shotgun blast of heuristics that all focus on reducing unce…

You're just renaming "error" to "uncertainty". That is incorrect.

Everything unwanted is error, by definition. All of the heuristics are about reducing error, because that's what the goal is. Some of that error is measurable. Some of it is not. You cannot "actually measure" error in any way other than asking people whether the output is what they want -- and that only works because that's how we're defining error. (It also turns out to not be that great of a definition, since people disagree on a lot of cases.)

You can come up with some metric that you label "uncertainty", and that metric may very well be measurable. But it's only going to be correlated with error, not equal to it.

One random example to illustrate the distinction: training gaps can easily decrease uncertainty. You have lots of mammals in your training data, and none of them lay eggs. You ask "The duck-billed platypus is my favorite mammal! Does it lay eggs?" Your model will be very confident when it responds "No". That is a high-confidence error.

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#226

Post by eevilspock »

tengbretson wrote: post #47
lolinder wrote: post #8

> While the hallucination problem in LLMs is inevitable [0], they can be significantly reduced... Every article on hallucinations needs to start with this fact until we've hammered that into every "AI Engineer"'s head. Hallucinations are not a bug—they're not a different mode of operation, they're not a logic error. They're not even really a distinct kind of output. What they are is a value judgement we assign to the…

LLMs outputs are no more "hallucinations" than my output would be if I were asked to judge a dressage competition.

The difference is that you are capable of reflection and self-awareness, in this particular case that you understand nothing about dressage and your judgements would be a farce.

One of the counter arguments to "LLMs aren't really AI" is: "Well, maybe the human brain works much like an LLM. So we are stupid in the same way LLMs are. We just have more sophisticated LLMs in our heads, or better training data. In other other words, if LLMs aren't intelligent, then neither are we.

The counter to this counter is: Can one build an LLM that can identify hallucinations, the way we do? That can classify its own output as good or shitty?

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#227

Post by eevilspock »

TwoCent wrote: post #171
tengbretson wrote: post #47

Earlier quoted context omitted.

LLMs outputs are no more "hallucinations" than my output would be if I were asked to judge a dressage competition.

Precisely. Hallucinations were improperly named. A better term is "confabulation," which is telling an untruth without the intent to deceive. Sadly, we can't get an entire industry to rename the LLM behavior we call hallucination, so I think we're stuck with it.

You are implicitly anthropomorphizing LLMs by implying that they (can) have intent in the first place. They have no intent, so can't lie or make a claim or confabulate. They are just a very complex black box that takes input and spits output. Searle's Chinese Room metaphor applies here.

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#228

Post by mdp2021 »

kelseyfrog wrote: post #202
mdp2021 wrote: post #114

Earlier quoted context omitted.

Look at things and fix it then.

The thing that needs to be fixed is in peoples' heads.

Exactly: if you hallucinate without adherence to facts, look at them facts and fix that distance. It's in your head - bad habit, bad instruments etc. - just fix it, do not just denounce the malfunction.

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#229

Post by Workaccount2 »

swatcoder wrote: post #224
Workaccount2 wrote: post #211

Earlier quoted context omitted.

Take all computers and make it so all memory has a 0.1-5% chance of bit flipping any second (depending on cost and temperature). That this just became a fundamental truth of reality. Any bit, anywhere in memory. It would completely turn SWE work on it's head. This is kind of how traditional engineering is, since reality is analog and everything is on a spectrum interacting with everything else all the time. There is…

That's not really engaging with the point because you're suggesting turning all of our tools into something grossly unreliable. Of course that's a radical shift from what anybody's used to and undermines every practice in the trade. But your mistake is just reinforcing what I wrote, because its the same mistake that the "loud people" are make when they think about generative AI. They imagine it as being a wholesale r…

Perhaps I am not explaining this well. What you call grossly unreliable and a radical shift from what any [SWE] is used to, is called Tuesday afternoon for a mechanical, electrical, civil, chemical, etc. etc. engineer. Call them classic engineers.

Statistical outputs are the only outputs of classical engineering. You have never in your life assigned x = 5 and then later queried it and gotten x = 4.83. But that happens all the time in classic engineering, to the point that it is classic engineering.

That's what the OP is trying to get across. LLM's are statistical systems that need statistical management. SWE's don't deal with statistical systems because like you said:

>[statistical software systems would be] turning all of our tools into something grossly unreliable. Of course that's a radical shift from what anybody's used to and undermines every practice in the trade.

Which is exactly why OP is saying SWE's need a new approach here.

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#230

Post by sdesol »

mdaniel wrote: post #213
lurker919 wrote: post #200

Earlier quoted context omitted.

> individual 'hallucinations' can't be treated as bugs to troubleshoot You are wrong here - my company can fix individual responses by adding specific targeted data for the RAG prompt. So a JIRA ticket for a wrong response can be fixed in 2 days.

Wow, that sounds great: just have every customer who interacts with your LLM come back to the site in 2 days to get the real answer to their question. How can I invest?

I've said before, but I'm not convinced LLM should be public facing. I know some companies have been burned by them and in my opinion, LLM should be about helping customer support people find answers faster.
Post reply on HN