On the Existence, Impact, and Origin of Hallucination-Associated Neurons in LLMs • Hacker News Board

Re: On the Existence, Impact, and Origin of Hallucination-Associated Neurons in LLMs

#21

Post by lambdaone » Mon, Dec 22, 2025, 3:35 PM UTC

airhangerf15 wrote: post #3

dang wrote: post #41

[stub for offtopicness] [submitters: one reason for not editorializing titles is it makes the threads be about that!]

LLMs don't "hallucinate" or "lie." They have no intent. They're Weighted Random Word Generator Machines. They're train mathematically to create series of tokens. Whenever they get something "right," it's literally by accident. If you get that rate of accidental rightness up to 80%, and people suddenly thing the random word generator is some kind of oracle. It's not. It's a large model with an embedded space, tokens a…

"Hallucinate" is a term of art, and does not imply a philosophical commitment to whether LLMs have minds. "Confabulation" might be a more appropriate term.

What is indisputable is that LLMs, even though they are 'just' word generators, are remarkably good at generating factual statements and accurate answers to problems, yet also regrettably inclined to generating apparenly equally confident counterfactual statements and bogus answers. That's all that 'hallucination' means in this context.

If this work can be replicated, it may offer a way to greatly improve the signal-to-bullshit ratio of LLMs, and that will be both impressive and very useful if true.

Re: On the Existence, Impact, and Origin of Hallucination-Associated Neurons in LLMs

#22

Post by tsumnia » Mon, Dec 22, 2025, 3:36 PM UTC

crazygringo wrote: post #11

airhangerf15 wrote: post #3

Earlier quoted context omitted.

LLMs don't "hallucinate" or "lie." They have no intent. They're Weighted Random Word Generator Machines. They're train mathematically to create series of tokens. Whenever they get something "right," it's literally by accident. If you get that rate of accidental rightness up to 80%, and people suddenly thing the random word generator is some kind of oracle. It's not. It's a large model with an embedded space, tokens a…

> LLMs don't "hallucinate" or "lie." They have no intent. You're just arguing about semantics. It doesn't matter in any substantial way. Ultimately, we need a word to distinguish factual output from confidently asserted erroneous output. We use the word "hallucinate". If we used a different word, it wouldn't make any difference -- the observable difference remains the same. "Hallucinate" is the word that has emerged,…

> You're just arguing about semantics. It doesn't matter in any substantial way.

While I agree for many general aspects of LLMs, I do disagree in terms of some of the meta-terms used when describing LLM behavior. For example, the idea that AI has "bias" is problematic because neural networks literally have a variable called "bias", thus of course AI will always have "bias". Plus, a biases AI is literally the purpose behind classification algorithms.

But these terms, "bias" and "hallucinations", are co-opted to spin a narrative of no longer trusting AI.

How in the world did creating an overly confident chatbot completely 180 years of AI progress and sentiment?

Re: On the Existence, Impact, and Origin of Hallucination-Associated Neurons in LLMs

#23

Post by 2026iknewit » Mon, Dec 22, 2025, 3:37 PM UTC

airhangerf15 wrote: post #3

dang wrote: post #41

[stub for offtopicness] [submitters: one reason for not editorializing titles is it makes the threads be about that!]

LLMs don't "hallucinate" or "lie." They have no intent. They're Weighted Random Word Generator Machines. They're train mathematically to create series of tokens. Whenever they get something "right," it's literally by accident. If you get that rate of accidental rightness up to 80%, and people suddenly thing the random word generator is some kind of oracle. It's not. It's a large model with an embedded space, tokens a…

This is still not true.

"Whenever they get something "right," it's literally by accident." "the random word generator"

First of, the input is not random at all which allows the question how random the output is.

Second, it compresses data which has an impact on that data. Probably cleaning or adjustment which should reduce 'random' even more. It compresses data from us into concepts. A high level concept is more robust than 'random'.

Thinking or reasoning models are also finetuning the response by walking the hyperspace and basically collecting and strengthening data.

We as humans do very similiar things and no one is calling us just random word predictors...

And because of this, "hallucinations -- plausible but factually incorrect outputs" is an absolut accurate description of what an LLM does when it response with a low probability output.

Humans also do this often enough btw.

Please stop saying an LLM is just a random word predictor.

Re: On the Existence, Impact, and Origin of Hallucination-Associated Neurons in LLMs

#24

Post by dpweb » Mon, Dec 22, 2025, 3:37 PM UTC

airhangerf15 wrote: post #3

dang wrote: post #41

[stub for offtopicness] [submitters: one reason for not editorializing titles is it makes the threads be about that!]

LLMs don't "hallucinate" or "lie." They have no intent. They're Weighted Random Word Generator Machines. They're train mathematically to create series of tokens. Whenever they get something "right," it's literally by accident. If you get that rate of accidental rightness up to 80%, and people suddenly thing the random word generator is some kind of oracle. It's not. It's a large model with an embedded space, tokens a…

We don't understand the brain. We fully understand what LLM are doing, humans built them. The idea we don't understand what LLMs are doing is magical. Magical is good for clicks and fundraising.

Re: On the Existence, Impact, and Origin of Hallucination-Associated Neurons in LLMs

#25

Post by lukeinator42 » Mon, Dec 22, 2025, 3:38 PM UTC

It is fairly well established that neurons in these artificial neural networks are polysemantic and information is represented in directions in the activation embedding space rather than neurons independently representing information (which is why anthropic is doing things like training sparse autoencoders). I haven't read the paper in depth but it seems like it is based on a fundamental misunderstanding about neurons in ANNs vs the brain.

Re: On the Existence, Impact, and Origin of Hallucination-Associated Neurons in LLMs

#26

Post by eurekin » Mon, Dec 22, 2025, 3:39 PM UTC

jrm4 wrote: post #5

dang wrote: post #41

[stub for offtopicness] [submitters: one reason for not editorializing titles is it makes the threads be about that!]

There is no such thing as a "hallucination" that could be isolated from "not a hallucination" in a provable systematic way because all they do is hallucinate. I'm extremely comfortable calling this paper complete and utter bullshit (or, I suppose if I'm being charitable, extremely poorly titled) from the title alone.

We are in the vibe science era, it seems

Re: On the Existence, Impact, and Origin of Hallucination-Associated Neurons in LLMs

#27

Post by 2026iknewit » Mon, Dec 22, 2025, 3:39 PM UTC

jrm4 wrote: post #5

dang wrote: post #41

[stub for offtopicness] [submitters: one reason for not editorializing titles is it makes the threads be about that!]

There is no such thing as a "hallucination" that could be isolated from "not a hallucination" in a provable systematic way because all they do is hallucinate. I'm extremely comfortable calling this paper complete and utter bullshit (or, I suppose if I'm being charitable, extremely poorly titled) from the title alone.

The Input of an LLM is real data. The n-dimensional space an LLM works in is a reflection of this. Statistical probably speaking there should be a way of knowing when an LLM is confident vs. when not.

This type of research is absolut valid.

An LLM is not just hallucinate.

Re: On the Existence, Impact, and Origin of Hallucination-Associated Neurons in LLMs

#28

Post by lambdaone » Mon, Dec 22, 2025, 3:42 PM UTC

jrm4 wrote: post #5

dang wrote: post #41

[stub for offtopicness] [submitters: one reason for not editorializing titles is it makes the threads be about that!]

There is no such thing as a "hallucination" that could be isolated from "not a hallucination" in a provable systematic way because all they do is hallucinate. I'm extremely comfortable calling this paper complete and utter bullshit (or, I suppose if I'm being charitable, extremely poorly titled) from the title alone.

Arguably, all we do is something similar to hallucination; it's just that hundreds of millions of years have selected against brains that generate internal states that lead to counter-survival behavior.

I recently almost fell on a tram as it accelerated suddenly; my arm reached out for a stanchion that was out of my vision, so rapidly I wasn't aware of what I was doing before it had happened. All of this occurred using subconscious processes, based on a non-physical internal mental model of something I literally couldn't see at the moment it happened. Consciousness is over-rated; I believe Thomas Metzinger's work on consciousness (specifically, the illusion of consciousness) captures something really important about the nature of how our minds really work.

Re: On the Existence, Impact, and Origin of Hallucination-Associated Neurons in LLMs

#29

Post by dudu24 » Mon, Dec 22, 2025, 3:46 PM UTC

airhangerf15 wrote: post #3

dang wrote: post #41

[stub for offtopicness] [submitters: one reason for not editorializing titles is it makes the threads be about that!]

LLMs don't "hallucinate" or "lie." They have no intent. They're Weighted Random Word Generator Machines. They're train mathematically to create series of tokens. Whenever they get something "right," it's literally by accident. If you get that rate of accidental rightness up to 80%, and people suddenly thing the random word generator is some kind of oracle. It's not. It's a large model with an embedded space, tokens a…

I'm tired of this pseudointellectual reductionist response. It's not "literally by accident" when they're trained to do something, as if we are not also machines that generate next actions based on learned neural weights and abstract (embedded) representations. Your issue is with semantics rather than content.

Obviously "hallucinate" and "lie" are metaphors. Get over it. These are still emergent structures that we have a lot to learn from by studying. But I suppose any attempt by researchers to do so should be disregarded because Person On The Internet has watched the 3blue1brown series on Neural Nets and knows better. We know the basic laws of physics, but spend lifetimes studying their emergent behaviors. This is really no different.

Re: On the Existence, Impact, and Origin of Hallucination-Associated Neurons in LLMs

#30

Post by cyost » Mon, Dec 22, 2025, 3:48 PM UTC

airhangerf15 wrote: post #3

dang wrote: post #41

[stub for offtopicness] [submitters: one reason for not editorializing titles is it makes the threads be about that!]

LLMs don't "hallucinate" or "lie." They have no intent. They're Weighted Random Word Generator Machines. They're train mathematically to create series of tokens. Whenever they get something "right," it's literally by accident. If you get that rate of accidental rightness up to 80%, and people suddenly thing the random word generator is some kind of oracle. It's not. It's a large model with an embedded space, tokens a…

Somewhat on the subject: here's a neuroscientist reflecting on our failure to model a worm's brain, a "mere" 302 neurons (3 parts, this one is the first). https://ccli.substack.com/p/the-biggest-mystery-in-neuroscie...

Biological systems are hard.