Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI hallucinations: Why LLMs make things up (and how to fix it)

kapa.ai

Post reply on HN
Print view View original

161–170 of 257 posts

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#161

Post by imchillyb »

Toddlers don't understand truth either, until it's taught.

This crayon is red. This crayon is blue.

The adult asks: "is this crayon red?" The child responds: "no that crayon is blue." The adult then affirms or corrects the response.

This occurs over and over and over until that child understands the difference between red and blue, orange and green, yellow and black etcetera.

We then move on to more complex items and comparisons. How could we expect AI to understand these truths without training them to understand?

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#162

Post by Gormo »

mike_hearn wrote: post #138
Gormo wrote: post #80

Earlier quoted context omitted.

Of course the LLM is bullshitting the user. That's precisely its purpose: LLMs are tools that generate comprehensible sounding language based on probability models that describe what words/tokens tend to be found in proximity to each other. An LLM doesn't actually know anything by reference to verifiable, external facts. Sure, LLMs can be used as fancy search engines that index documents and then answer questions by…

Models do know things. Facts are encoded in their parameters. Look at the some of the interpretability research to see that. They aren't just Markov chains.

Nope. They don't know any specific facts. The training data produces a probability matrix that reflects what words are likely to be found in relation other words, allowing it to generate novel combinations of words that are coherent and understandable. But there is no mechanism involved for determining whether those novel expressions are actually factual representations of reality.

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#163

Post by mike_hearn »

Gormo wrote: post #162
mike_hearn wrote: post #138

Earlier quoted context omitted.

Models do know things. Facts are encoded in their parameters. Look at the some of the interpretability research to see that. They aren't just Markov chains.

Nope. They don't know any specific facts. The training data produces a probability matrix that reflects what words are likely to be found in relation other words, allowing it to generate novel combinations of words that are coherent and understandable. But there is no mechanism involved for determining whether those novel expressions are actually factual representations of reality.

Again, read the papers. They absolutely do know facts, and that can be seen in the activations. Your description is oversimplified. It's easy to get models to emit statistically improbable but correct sequences of words. They are not just looking at what words are near by each other, that doesn't lead to the kind of output LLMs are capable of.

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#164

Post by smartmic »

lolinder wrote: post #8

> While the hallucination problem in LLMs is inevitable [0], they can be significantly reduced... Every article on hallucinations needs to start with this fact until we've hammered that into every "AI Engineer"'s head. Hallucinations are not a bug—they're not a different mode of operation, they're not a logic error. They're not even really a distinct kind of output. What they are is a value judgement we assign to the…

A machine with probabilistic output generation cannot tell what is a hallucination and what is not. It does not know the difference between truth and everything else. It is us humans on the receiving end who have to classify the content - and that is the problem. We have little patience, time, or energy to do this verification work for every piece of information. That's why we have the human trait of trust, which has been at the core of human progress from the beginning.

Now the question can be rephrased. Is it possible to trust AI information generators - what's to be done to build trust? And here is the difficulty - I do not know why I should ever trust a probabilistic system as long as it has this property and does not turn into a deterministic version of itself. I won't lower my standards for trusting people, for good reasons. But I cannot even raise the bar for trust in a machine above zero as long as it is driven by randomness at its core.

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#165

Post by mycall »

lolinder wrote: post #8

> While the hallucination problem in LLMs is inevitable [0], they can be significantly reduced... Every article on hallucinations needs to start with this fact until we've hammered that into every "AI Engineer"'s head. Hallucinations are not a bug—they're not a different mode of operation, they're not a logic error. They're not even really a distinct kind of output. What they are is a value judgement we assign to the…

Does it help discourse at all by instead calling hallucinations a negative perceived form of imagination?

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#166

Post by mycall »

JKCalhoun wrote: post #74
swatcoder wrote: post #21

Earlier quoted context omitted.

> The trouble is that we software engineers have spent so long working in an artificially deterministic world that we're not used to designing and evaluating probabilistic quality control systems for computer output. I think that's a mischaracterization and not really accurate. As a trade, we're familiar with probabilistic/non-deterministic components and how to approach them. You were closer when you used quotes aro…

> As a trade, we're familiar with probabilistic/non-deterministic components and how to approach them. Yes, users.

..and bugs, especially with analog computers.

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#167

Post by devmor »

GuB-42 wrote: post #157
lolinder wrote: post #8

> While the hallucination problem in LLMs is inevitable [0], they can be significantly reduced... Every article on hallucinations needs to start with this fact until we've hammered that into every "AI Engineer"'s head. Hallucinations are not a bug—they're not a different mode of operation, they're not a logic error. They're not even really a distinct kind of output. What they are is a value judgement we assign to the…

Of course they are a bug. Just that hallucination emerge from the normal function of a LLM doesn't make it "not a bug". No programmer in their right mind will call the lack of bound checking resulting in garbled output "not a bug", even though it is a totally normal thing to do from the point of view of a CPU. It is a bug and you need additional code to fix it, for example by checking for out-of-bounds condition and…

A bug is generally treated as undefined and undesirable side effects of a program.

Hallucinations are undesirable but not undefined. We know that the process creates them and expect them.

It’d be like using floats to calculate dollars and cents and calling the resulting math a bug - it’s not, you just used the technology wrong.

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#168

Post by jimmySixDOF »

tshadley wrote: post #16

The article referenced the Oxford semantic entropy study but failed to clarify that the issue greatly simplifies LLM hallucination (making most of the article outdated). When we are not sure of an answer we have two choices: say the first thing that comes to mind (like an LLM), or say "I'm not sure". LLMs aren't easily trained to say "I'm not sure" because that requires additional reasoning and introspection (which i…

funny I was just looking at this approach and see what's new in the wild (a work in progress) see Entropix [1] by [2]

[1] https://github.com/xjdr-alt/entropix

[2] https://x.com/_xjdr

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#169

Post by cameronh90 »

maeil wrote: post #100
cameronh90 wrote: post #46

Earlier quoted context omitted.

At scale, you are doing the same thing with humans too. LLMs seem to have an error rate similar to humans for the majority of simple, boring tasks, if not even a bit better since they don't get distracted and start copying and pasting their previous answers. The difference with LLMs is they simply cannot (currently) do the most complex tasks that some humans can, and when they do produce erroneous output, the errors…

> At scale, you are doing the same thing with humans too. LLMs seem to have an error rate similar to humans for the majority of simple, boring tasks, if not even a bit better since they don't get distracted and start copying and pasting their previous answers. This is the big one missed by the frequent comments on here wondering whether LLMs are a fad, or claiming in their current state they cannot be used to replace…

Related to that, I once had a CT scan for a potentially fatal brain concern, and the note that the radiologist sent back to my consultant was for a completely different patient, and the notes for my scan were attached to someone else's report. The only reason it was caught was because it referred to me as "she".

If we were both the same gender, I probably would have had my skull opened up for no reason, and she would have been discharged and later died.

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#170

Post by objclxt »

empath75 wrote: post #25
Loughla wrote: post #4

I just recently showed a group of college students how and why using AI in school is a bad idea. Telling them it's plagiarism doesn't have an impact, but showing them how it gets even simple things wrong had a HUGE impact. The first problem was a simple numbers problem. It's 2 digit numbers in a series of boxes. You have to add numbers together to make a trail to get from left to right moving only horizontally or ver…

I refuse to believe that you did any of this with any of the latest models. Gemini and Chat GPT with search are both perfectly capable of producing decent essays with accurate citations. And the 4o model is extremely good at writing python code that can accurately solve math and logic problems. I asked 4o with search to write an essay about the dangers of smoking, along with citations and quotes from the relevant sou…

> Gemini and Chat GPT with search are both perfectly capable of producing decent essays with accurate citations

OK, but this quote from your essay:

> The National Cancer Institute (NCI) reports that "cigarette smoking and exposure to tobacco smoke cause about 480,000 premature deaths each year in the United States."

...that citation is wrong. It's not from the NCI at all, the NCI cited that figure which came from another paper by the U.S. Department of Health and Human Services.

The essay doesn't have accurate citations, the model has regurgitated content and doesn't understand when that content is from a primary source or when it in turn has come from a different citation.

Post reply on HN