AI hallucinations: Why LLMs make things up (and how to fix it) • Hacker News Board

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#51

Post by gerdesj » Thu, Dec 05, 2024, 1:24 AM UTC

Mistletoe wrote: post #31

Is there a way to code an LLM to just say "I don't know" when it is uncertain or reaching some sort of edge?

"It" does not know when it does not know. A LLM is a funny old beast that basically outputs words one after another based on probabilities. There is no reasoning as we would know it involved.

However, I'll tentatively allow that you do get a sort of "emergent behaviour" from them. You do seem to get some form of intelligent output from a prompt but correctness is not built in, nor is any sort of reasoning.

The examples around here of how to trip up a LLM are cool. There's: "How many letter "m"s in the word minimum" howler which is probably optimised for by now and hence held up as a counterpoint by a fan. The one about boxes adding up to 1000 will leave a relative of mine for lost for ever but they can still walk and catch a ball, negotiate stairs and recall facts from 50 years ago with clarity.

Intelligence is a slippery concept to even define, let alone ask what an artificial one might look like. LLMs are a part of the puzzle and certainly not a solution.

You mention the word "edge" and I suppose you might be riffing on how neurons seem to work. LLMs don't have a sort of trigger threshold, they simply output the most likely answers based on their input.

If you keep your model tightly ie domain focussed and curate all of the input then you have more chance of avoiding "hallucinations" than if you don't. Trying to cover the entirety of everything is Quixotic nonsense.

Garbage in; garbage out.

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#52

Post by threeseed » Thu, Dec 05, 2024, 1:25 AM UTC

leprechaun1066 wrote: post #30

lolinder wrote: post #8

> While the hallucination problem in LLMs is inevitable [0], they can be significantly reduced... Every article on hallucinations needs to start with this fact until we've hammered that into every "AI Engineer"'s head. Hallucinations are not a bug—they're not a different mode of operation, they're not a logic error. They're not even really a distinct kind of output. What they are is a value judgement we assign to the…

Calling them hallucinations was a huge mistake.

I see two types of faults with LLMs.

a) They output incorrect results given a constrained set of allowable outputs.

b) When unconstrained they invent new outputs unrelated to what is being asked.

So for me the term hallucination accurately describes b) e.g. you ask for code to solve a problem and it invents new APIs that don't exist. Technically it is all just tokens and probabilities but it's a reasonably term to describe end user behaviour.

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#53

Post by int_19h » Thu, Dec 05, 2024, 1:27 AM UTC

tshadley wrote: post #16

The article referenced the Oxford semantic entropy study but failed to clarify that the issue greatly simplifies LLM hallucination (making most of the article outdated). When we are not sure of an answer we have two choices: say the first thing that comes to mind (like an LLM), or say "I'm not sure". LLMs aren't easily trained to say "I'm not sure" because that requires additional reasoning and introspection (which i…

QwQ is really good at saying "I'm not sure", to the point where it will sometimes check the correct and obviously trivial answer a dozen times before concluding that it is, indeed, correct. And it does punch way above its weight for its size.

So, basically, the answer seems to be to give models extreme anxiety and doubt in their own abilities.

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#54

Post by wpietri » Thu, Dec 05, 2024, 1:28 AM UTC

lolinder wrote: post #8

> While the hallucination problem in LLMs is inevitable [0], they can be significantly reduced... Every article on hallucinations needs to start with this fact until we've hammered that into every "AI Engineer"'s head. Hallucinations are not a bug—they're not a different mode of operation, they're not a logic error. They're not even really a distinct kind of output. What they are is a value judgement we assign to the…

Excellent point:

> just output from an LLM-based workflow that is not fit for purpose

And I think this is just one aspect of what I think of as the stone soup [1] problem. Outside of rigorous test conditions, humans just have a hard time telling how much work they're doing when they interpret something. It's the same sort of thing you see with "psychics" doing things like cold reading. People make meaning out of vaguery and nonsense and then credit the nonsense-producer with the work.

[1] https://en.wikipedia.org/wiki/Stone_Soup

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#55

Post by int_19h » Thu, Dec 05, 2024, 1:28 AM UTC

Mistletoe wrote: post #31

Is there a way to code an LLM to just say "I don't know" when it is uncertain or reaching some sort of edge?

It is. The new crop of models specifically trained to do CoT (as opposed to just forcing them into it via prompting) is partly about forcing them to continue to think and not just hastily conclude that they have the answer.

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#56

Post by threeseed » Thu, Dec 05, 2024, 1:29 AM UTC

jebarker wrote: post #50

sfink wrote: post #34

Earlier quoted context omitted.

The term is actually fine. The problem is when it's divorced from the reality of: > in some sense, hallucination is all LLMs do. They are dream machines. If you understand that, then the term "hallucination" makes perfect sense. Note that this in no way invalidates your point, because the term is constantly used and understood without this context. We would have avoided a lot of confusion if we had based it on the ph…

I agree with your point, but I don't think anthropomorphizing LLMs is helpful. They're statistical estimators trained by curve fitting. All generations are equally valid for the training data, objective and architecture. To me it's much clearer to think about it that way versus crude analogies to human brains.

We can't expect end users to understand what "statistical estimators trained by curve fitting" means.

That's why we use high level terms like hallucination. Because it's something everyone can understand even if it's not completely accurate.

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#57

Post by kelseyfrog » Thu, Dec 05, 2024, 1:30 AM UTC

PLenz wrote: post #11

Everything an LLM returns is an hallucination, it's just that some of those hallucinations line up with reality

Subjectively, we operate the same way.

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#58

Post by threeseed » Thu, Dec 05, 2024, 1:31 AM UTC

ml_more wrote: post #6

Terr_ wrote: post #3

When people talk about stopping an LLM from "seeing hallucinations instead of the truth", that's like stopping an Ouija-board from "channeling the wrong spirits instead of the right spirits." It suggests a qualitative difference between desirable and undesirable operation that isn't really there. They're all hallucinations, we just happen to like some of them more than others.

The problem is that LLMs are just convincing enough that people DO trust them which is sort of a problem since AI slop is creeping into everything. What can be done to solve it (while not perfect) is pretty powerful. You can force feed them the facts (RAG) and then verify the result. Which is way better than trusting LLMs while doing neither of those things (which is what a lot of people do today anyway). See the rec…

> Clearly investors think there's something there

Of course. Because enterprise companies take a long time to evaluate new technologies. And so there is plenty of money to be made selling them tools over the next few years. As well as selling tools to those who are making tools.

But from my experience in rolling out these technologies only a handful of these companies will exist in 5-10 years. Because LLMs are "garbage in, garbage out" and we've never figured out how to keep the "garbage in" to a minimum.

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#59

Post by jebarker » Thu, Dec 05, 2024, 1:34 AM UTC

threeseed wrote: post #56

jebarker wrote: post #50

Earlier quoted context omitted.

I agree with your point, but I don't think anthropomorphizing LLMs is helpful. They're statistical estimators trained by curve fitting. All generations are equally valid for the training data, objective and architecture. To me it's much clearer to think about it that way versus crude analogies to human brains.

We can't expect end users to understand what "statistical estimators trained by curve fitting" means. That's why we use high level terms like hallucination. Because it's something everyone can understand even if it's not completely accurate.

That's a good point. But re: not anthropomorphizing, what's wrong with errors, mistakes or inaccuracies? That's something everybody is familiar with and is more accurate. I'd guess most people have never actually experienced a hallucination anyway, so we're appealing to some vague notion of what that is.

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#60

Post by Terr_ » Thu, Dec 05, 2024, 1:35 AM UTC

latexr wrote: post #22

ml_more wrote: post #6

Earlier quoted context omitted.

The problem is that LLMs are just convincing enough that people DO trust them which is sort of a problem since AI slop is creeping into everything. What can be done to solve it (while not perfect) is pretty powerful. You can force feed them the facts (RAG) and then verify the result. Which is way better than trusting LLMs while doing neither of those things (which is what a lot of people do today anyway). See the rec…

> and that unlocks a bit of value out in the world. > Don't take my word for it look at the proposed valuations of AI companies. Clearly investors think there's something there. Investors back whatever they think will make them money. They couldn’t give less of a crap if something is valuable to the world, or works well, of is in any way positive to others. All they care is if they can profit from it and they’ll chas…

> Investors back whatever they think will make them money.

A not-flagrantly-illegal example of this might be casinos, where IMO it is basically impossible to argue the fleeting entertainment they offer offsets the financial ruin inflicted on certain vulnerable types of patron.

> All they care is if they can profit from it

Notably that isn't the same as the business itself being profitable: Some investors may be hoping they can dump their stake at a higher price onto a Greater Fool [0] and exit before the collapse.

[0] https://en.wikipedia.org/wiki/Greater_fool_theory