Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI hallucinations: Why LLMs make things up (and how to fix it)

kapa.ai

Post reply on HN
Print view View original

121–130 of 257 posts

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#121

Post by Freak_NL »

int_19h wrote: post #45
ryanmcbride wrote: post #5

Earlier quoted context omitted.

My go-to to show people who don't understand its limitations used to be the old "how many Ms are there in the word 'minimum' or something along those lines, but looks like it's gotten a bit better at that. I just tried it with GPT4o and it gave me the right number, but the wrong placement. In the past it's given it completely wrong: >how many instances of the letter L are in the word parallel The word parallel contai…

Here's QwQ-32 answering the same question. I'm including the entire answer here including the CoT: > Let's see. The word is "parallel." I need to count how many times the letter "L" appears in it. Okay, spelling it out: P-A-R-A-L-L-E-L. So, starting from the beginning, first letter is P, not L. Then A, not L. R, not L. Another A, still not L. Then L, that's one L. Next letter is L again, so that's two L's. Then E, no…

This AI is talking as if it is in the final stages of a multi-day interrogation by a very hush-hush and deniable branch of government. I give it another five minutes before it breaks down and will sell out its mother just to be allowed to sleep with the lights off.

I hope that this is not something I ever have to deal with. It's exhausting.

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#122

Post by mort96 »

Animats wrote: post #103
lolinder wrote: post #8

> While the hallucination problem in LLMs is inevitable [0], they can be significantly reduced... Every article on hallucinations needs to start with this fact until we've hammered that into every "AI Engineer"'s head. Hallucinations are not a bug—they're not a different mode of operation, they're not a logic error. They're not even really a distinct kind of output. What they are is a value judgement we assign to the…

> While the hallucination problem in LLMs is inevitable Oh, please. That's the same old computability argument used to claim that program verification is impossible. Computability isn't the problem. LLMs are forced to a reply, regardless of the quality of the reply. If "Confidence level is too low for a reply" is an option, the argument in that paper becomes invalid. The trouble is that we don't know how to get a con…

Confidence levels aren't necessarily low for incorrect replies, that's the problem. The LLM doesn't "know" that what it's outputting is incorrect. It just knows that the words it's writing are probable given the inputs; "this is how answers tend to look like".

You can make improvements, as your parent comment already said, but it's not a problem which can be solved, only to some degree be reduced.

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#123

Post by russnes »

lolinder wrote: post #8

> While the hallucination problem in LLMs is inevitable [0], they can be significantly reduced... Every article on hallucinations needs to start with this fact until we've hammered that into every "AI Engineer"'s head. Hallucinations are not a bug—they're not a different mode of operation, they're not a logic error. They're not even really a distinct kind of output. What they are is a value judgement we assign to the…

so what you're saying is that LLMs are like middle aged men, just throwing things out there seeing if they'll stick?

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#124

Post by mdp2021 »

fsckboy wrote: post #92

it's superficially counterintuitive to people that an AI that will sometimes spit out verbatim copies of written texts, also will just make other things up. It's like "choose one, please". MetaAI makes up stuff reliably. You'd think it would be an ace at baseball stats for example, but "what teams did so-and-so play for", you absolutely must check the results yourself.

> "counterintuitive"

It is consistent with the topic that the reply would be "Tell them that sequences of words that were verbatim in a past input have high probability, and gaps in sequences compete in probability". Which fixes intuition, as duly. In fact, things are not supposed to reply through intuition, but through vetted intuition (and "vetted mature intuition", in a loop).

> you absolutely must check the results yourself

So, consistently with the above, things are supposed to reply through a sort of """RAG""" of the vetted (dynamically built through iterations of checks).

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#125

Post by namaria »

mdp2021 wrote: post #119
namaria wrote: post #111

Earlier quoted context omitted.

Ancient Egypt chronology is a poor example of determined knowledge. We do not know in fact exactly when (which?) Pyramids were built, there are large margin of errors in the estimates.

That was my point: answering that question is a more complex evaluation than others. In lower percentiles you may have "what is in front of you" and in upper percentiles you may have "how to fix the balance of power in the Pacific" - all more or less complex evaluations. I said, "Not even factual notions are trivial, e.g. "When has this event happened" - all have some foundational ground of higher or lower solidity".

Right, I misread your comment. Sorry!

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#126

Post by mort96 »

zmgsabst wrote: post #44
lolinder wrote: post #8

> While the hallucination problem in LLMs is inevitable [0], they can be significantly reduced... Every article on hallucinations needs to start with this fact until we've hammered that into every "AI Engineer"'s head. Hallucinations are not a bug—they're not a different mode of operation, they're not a logic error. They're not even really a distinct kind of output. What they are is a value judgement we assign to the…

A challenge is that it’s not easy to limit hallucinations without also limiting imagination and synthesis. In humans. But also apparently in LLMs.

Healthy humans generally have some internal model of the world against which they can judge what they're about to say. They can introspect and determine whether what they say is a guess or a statement of fact. LLMs can't.

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#127

Post by emil_sorensen »

int_19h wrote: post #43

I've been playing with Qwen's QwQ-32b, and watching this thing's chain of thought is really interesting. In particular, it's pretty good at catching its own mistakes, and at the same time, gives off a "feeling" of someone very uncertain about themselves, trying to verify their answer again and again. Which seems to be the main reason why it can correctly solve puzzles that some much larger models fail. You can still…

I find it so interesting that it's possible to develop a "feeling" of a new model.

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#128

Post by TZubiri »

ausbah wrote: post #104
TZubiri wrote: post #71

Earlier quoted context omitted.

The less tokens produced at inference the lower the quality of the response will be. The process of thinking for an LLM involves the use of words, which is why prompts that ask the LLM to only return the answer will cause lower quality.

do you know if prompting without regards for length then asking for a summarization of the previous out out works?

It does. I think this was used in a gpt4 version, they called it Chain of Thought.

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#129

Post by Timwi »

namaria wrote: post #120
__MatrixMan__ wrote: post #24

Earlier quoted context omitted.

There's room for splitting hairs in there though. Even fiction, for instance, can succeed or fail at being internally consistent, is or is not grammatically correct... Calling everything an AI does a hallucination isn't incorrect, but it reduces the term to meaninglessness. I'm not sure that's most useful thing we can be doing. Atoms are not indivisible, yet we use the term because it works. I anticipate hallucinatio…

> Atoms are not indivisible They are the smallest unit of a substance that cannot be broken down into smaller units of the same substance. They are, in a sense, indivisible.

Of course they can. Carbon dioxide consists of quarks and electrons. I can divide it into units smaller than atoms and it's still quarks and electrons. All you did was a word trick by assuming a specific meaning of “substance”.

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#130

Post by emil_sorensen »

lolinder wrote: post #8

> While the hallucination problem in LLMs is inevitable [0], they can be significantly reduced... Every article on hallucinations needs to start with this fact until we've hammered that into every "AI Engineer"'s head. Hallucinations are not a bug—they're not a different mode of operation, they're not a logic error. They're not even really a distinct kind of output. What they are is a value judgement we assign to the…

That's a great point. Reminds me of the "feature, not a bug" Karpathy tweet [0].

[0]: https://x.com/karpathy/status/1733299213503787018?lang=en

Post reply on HN