Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI hallucinations: Why LLMs make things up (and how to fix it)

kapa.ai

Post reply on HN
Print view View original

81–90 of 257 posts

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#81

Post by LPisGood »

JKCalhoun wrote: post #76
tengbretson wrote: post #73

Earlier quoted context omitted.

LLMs don't know things, they just string together responses that are a best fit for what follows from their prompt. I suspect its so hard to get them to say "I don't know" because if they were biased towards responding that way then I would assume thats almost all they would ever say, since "I don't know" is an appropriate answer to every question imaginable.

I get that, but since it is all probabilities, you might imagine even the LLM knows when it is skating on thin ice. If I'm beginning with "Once / upon / a" I think the data will show a very high confidence in the word to follow with. So too I would imagine it would know when the trail of breadcrumbs it has been following is of the trashier and low probability kind. So just tell me. (Or perhaps speak to me and when yo…

[dead]

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#82

Post by Gormo »

JKCalhoun wrote: post #76
tengbretson wrote: post #73

Earlier quoted context omitted.

LLMs don't know things, they just string together responses that are a best fit for what follows from their prompt. I suspect its so hard to get them to say "I don't know" because if they were biased towards responding that way then I would assume thats almost all they would ever say, since "I don't know" is an appropriate answer to every question imaginable.

I get that, but since it is all probabilities, you might imagine even the LLM knows when it is skating on thin ice. If I'm beginning with "Once / upon / a" I think the data will show a very high confidence in the word to follow with. So too I would imagine it would know when the trail of breadcrumbs it has been following is of the trashier and low probability kind. So just tell me. (Or perhaps speak to me and when yo…

Maybe just having a confidence weight assigned to each sentence the LLM generates, reflected in tooltips or text coloring, would be a big improvement.

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#83

Post by Gormo »

ml_more wrote: post #6
Terr_ wrote: post #3

When people talk about stopping an LLM from "seeing hallucinations instead of the truth", that's like stopping an Ouija-board from "channeling the wrong spirits instead of the right spirits." It suggests a qualitative difference between desirable and undesirable operation that isn't really there. They're all hallucinations, we just happen to like some of them more than others.

The problem is that LLMs are just convincing enough that people DO trust them which is sort of a problem since AI slop is creeping into everything. What can be done to solve it (while not perfect) is pretty powerful. You can force feed them the facts (RAG) and then verify the result. Which is way better than trusting LLMs while doing neither of those things (which is what a lot of people do today anyway). See the rec…

Sure, using RAG is great, but it limits the LLM to functioning as a natural-language search engine. That's a pretty useful thing in its own right, and will revolutionize a lot of activities, but it still falls far short of the expectations people have for generative AI.

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#84

Post by Gormo »

latexr wrote: post #22
ml_more wrote: post #6

Earlier quoted context omitted.

The problem is that LLMs are just convincing enough that people DO trust them which is sort of a problem since AI slop is creeping into everything. What can be done to solve it (while not perfect) is pretty powerful. You can force feed them the facts (RAG) and then verify the result. Which is way better than trusting LLMs while doing neither of those things (which is what a lot of people do today anyway). See the rec…

> and that unlocks a bit of value out in the world. > Don't take my word for it look at the proposed valuations of AI companies. Clearly investors think there's something there. Investors back whatever they think will make them money. They couldn’t give less of a crap if something is valuable to the world, or works well, of is in any way positive to others. All they care is if they can profit from it and they’ll chas…

> They couldn’t give less of a crap if something is valuable to the world

"The world" is an abstraction: concretely, every bit of value that is generated within that abstraction accrues to someone in particular -- investors in AI projects, for example.

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#85

Post by lmm »

xienze wrote: post #62
tengbretson wrote: post #47

Earlier quoted context omitted.

LLMs outputs are no more "hallucinations" than my output would be if I were asked to judge a dressage competition.

I’ve had multiple occasions where I’ve asked an LLM how to do in Java and it’ll very confidently answer to use . It would be far more helpful to me to receive an answer like “I don’t think there’s a third party library that does this, you’ll have to write it yourself” than to waste my time telling me a lie. If anything, calling these outputs “hallucinations” is a very polite way of saying that the LLM is bullshitting…

The LLM is always bullshitting the user. It's just sometimes the things it talks about happen to be real and sometimes they don't.

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#86

Post by Terr_ »

TZubiri wrote: post #69
Terr_ wrote: post #3

When people talk about stopping an LLM from "seeing hallucinations instead of the truth", that's like stopping an Ouija-board from "channeling the wrong spirits instead of the right spirits." It suggests a qualitative difference between desirable and undesirable operation that isn't really there. They're all hallucinations, we just happen to like some of them more than others.

I disagree with this take, Stallman has expressed it recently by linking some "scientific article". While I get that LLMs generate text in some way that does not guarantee correctness. There is a correlation between generated text and correctness, which is why millions of people use it... You can judge the correctness of a sentence generated by an LLM. In the same way you can judge the correctness of a human generate…

> I disagree with this take, Stallman has expressed it recently by linking some "scientific article".

I don't know how to parse this. What article did Stallman "link", and what are you saying Stallman "expressed" by linking/using it?

> whether the truthness or correlation with reality of an LLM sentence can be judged on its own or whether it requires a human to interpret it is not very relevant

It's incredibly relevant. We wouldn't even be having these debates if complex LLM judgements could always be verified without a human checking the logic.

> sentences produced by the LLM are still correct most of the time

At least half the problem here is that humans are accustomed to using certain cues as an indirect sign of time-investment, attentiveness, intelligence, truth, etc... and now those cues can be cheaply and quickly counterfeited. It breaks all those old correlations faster than we are adapting.

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#87

Post by Gormo »

__MatrixMan__ wrote: post #24
PLenz wrote: post #11

Everything an LLM returns is an hallucination, it's just that some of those hallucinations line up with reality

There's room for splitting hairs in there though. Even fiction, for instance, can succeed or fail at being internally consistent, is or is not grammatically correct... Calling everything an AI does a hallucination isn't incorrect, but it reduces the term to meaninglessness. I'm not sure that's most useful thing we can be doing. Atoms are not indivisible, yet we use the term because it works. I anticipate hallucinatio…

> Calling everything an AI does a hallucination isn't incorrect, but it reduces the term to meaninglessness.

I don't think it does. In this case, "hallucination" refers to claims generated entirely within a closed system, but which pertain to a reality external to it.

That's not meaningless, and makes "hallucinations" distinguishable from claims verified against direct observation of the reality they are meant to represent.

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#88

Post by jrmg »

HeatrayEnjoyer wrote: post #18
lawlessone wrote: post #9

Earlier quoted context omitted.

They probably have a letter counting tool added to it now. that it just knows to call when asked to do this. you ask it the number of letters and it sends those words off to another tool to count instances of L, but they didn't add a placement one so it's still guessing those. edit: corrected some typos and phrasing. Maybe we'll reach a point where the LLM's are just tool calling models and not really giver their own…

There are only 5 tools it has available to call, and that isn't one of them. A GitHub (forgot the url) stays up to date with the latest dumped system instructions.

How do we know they’re the real system instructions? If they’re determined by interrogating the LLM hallucination is a very real possibility.

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#89

Post by lmm »

jebarker wrote: post #59
threeseed wrote: post #56

Earlier quoted context omitted.

We can't expect end users to understand what "statistical estimators trained by curve fitting" means. That's why we use high level terms like hallucination. Because it's something everyone can understand even if it's not completely accurate.

That's a good point. But re: not anthropomorphizing, what's wrong with errors, mistakes or inaccuracies? That's something everybody is familiar with and is more accurate. I'd guess most people have never actually experienced a hallucination anyway, so we're appealing to some vague notion of what that is.

> what's wrong with errors, mistakes or inaccuracies?

They're not specific enough terms for what we're talking about. Saying a lion has stripes is an error, mistake, or inaccuracy. Describing a species of striped lions in detail is probably all those things, but it's a distinctive kind of error/mistake/inaccuracy that's worth having a term for.

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#90

Post by Gormo »

TZubiri wrote: post #70
PLenz wrote: post #11

Everything an LLM returns is an hallucination, it's just that some of those hallucinations line up with reality

How are you defining hallucination then? In some pretty useless way inevitably. Hallucinations are precisely the generated expressions that don't correlate with reality or are not truthful.

I don't think that definition works: it's attempting to categorize statements according to criteria completely external to them rather than according to any inherent property of the statement.

A better definition is that a hallucination is an expression that is generated within a closed system without direct input from the reality it is meant to represent. The point is that an expression about reality that doesn't come from observing reality can only be true coincidentally.

By way of analogy, if I have a dream about a future event, and then that event actually happens, it was still just a dream and not a clairvoyant vision of the future. Sure, my dreams are influenced by past experiences I've had (in the same way that verified facts are included in the training data for LLMs), which makes them likely to include things that frequently do happen in real life and might be likely to happen again -- but the dream an the LLM alike are effectively just "remixing" prior input, and not generating any new observations of reality.

Post reply on HN