On the Existence, Impact, and Origin of Hallucination-Associated Neurons in LLMs • Hacker News Board

Re: On the Existence, Impact, and Origin of Hallucination-Associated Neurons in LLMs

#31

Post by spwa4 » Mon, Dec 22, 2025, 3:49 PM UTC

tsumnia wrote: post #22

crazygringo wrote: post #11

Earlier quoted context omitted.

> LLMs don't "hallucinate" or "lie." They have no intent. You're just arguing about semantics. It doesn't matter in any substantial way. Ultimately, we need a word to distinguish factual output from confidently asserted erroneous output. We use the word "hallucinate". If we used a different word, it wouldn't make any difference -- the observable difference remains the same. "Hallucinate" is the word that has emerged,…

> You're just arguing about semantics. It doesn't matter in any substantial way. While I agree for many general aspects of LLMs, I do disagree in terms of some of the meta-terms used when describing LLM behavior. For example, the idea that AI has "bias" is problematic because neural networks literally have a variable called "bias", thus of course AI will always have "bias". Plus, a biases AI is literally the purpose…

Terminology sucks. There is an ML technique called "hallucinating", that can really improve results. It works, for example, on Alphafold, and allows you to reverse the function of Alphafold (instead of finding the fold that matches a given protein or protein complex, find a protein complex that has a specific shape, or fits on a specific shape).

It's called hallucination because it works by imagining you have the solution and then learning what the input needs to be to get that solution. Treat the input or the output as weights and learn an input that fits an output or vice-versa instead of the network. Fix what the network sees as the "real world" to match what "what you already knew", just like a hallucinating human does.

You can imagine how hard it is to find papers on this technique nowadays.

Re: On the Existence, Impact, and Origin of Hallucination-Associated Neurons in LLMs

#32

Post by TomasBM » Mon, Dec 22, 2025, 3:50 PM UTC

crazygringo wrote: post #11

airhangerf15 wrote: post #3

Earlier quoted context omitted.

LLMs don't "hallucinate" or "lie." They have no intent. They're Weighted Random Word Generator Machines. They're train mathematically to create series of tokens. Whenever they get something "right," it's literally by accident. If you get that rate of accidental rightness up to 80%, and people suddenly thing the random word generator is some kind of oracle. It's not. It's a large model with an embedded space, tokens a…

> LLMs don't "hallucinate" or "lie." They have no intent. You're just arguing about semantics. It doesn't matter in any substantial way. Ultimately, we need a word to distinguish factual output from confidently asserted erroneous output. We use the word "hallucinate". If we used a different word, it wouldn't make any difference -- the observable difference remains the same. "Hallucinate" is the word that has emerged,…

While I agree that we need a word for this type of behavior, hallucinate is a wrong choice IMO.

Hallucinations are already associated with a type of behavior, which is (roughly defined) "subjectively seeing/hearing things which aren't there". This is an input-level error, not the right umbrella term for the majority of errors happening with LLMs, many if which are at output-level.

I don't know what would be a better term, but we should distinguish between different semantic errors, such as:

- confabulating, i.e., recalling distorted or misinterpreted memories;

- lying, i.e., intentionally misrepresenting an event or memory;

- bullshitting, i.e., presenting a version without regard for the truth or provenance; etc.

I'm sure someone already made a better taxonomy, and hallucination is OK for normal public discussions, but I'm not sure why the distinctions aren't made in supposedly more serious works.

Re: On the Existence, Impact, and Origin of Hallucination-Associated Neurons in LLMs

#33

Post by GaggiX » Mon, Dec 22, 2025, 3:52 PM UTC

The people talking about semantics in the comment section seems to completely ignore the positive correlation of LLMs between accuracy and stated confidence, this is called calibration and this "old" blog post from a year ago already showed it, LLMs can know what they know: https://openai.com/index/introducing-simpleqa/

Re: On the Existence, Impact, and Origin of Hallucination-Associated Neurons in LLMs

#34

Post by allears » Mon, Dec 22, 2025, 3:58 PM UTC

dpweb wrote: post #24

airhangerf15 wrote: post #3

Earlier quoted context omitted.

LLMs don't "hallucinate" or "lie." They have no intent. They're Weighted Random Word Generator Machines. They're train mathematically to create series of tokens. Whenever they get something "right," it's literally by accident. If you get that rate of accidental rightness up to 80%, and people suddenly thing the random word generator is some kind of oracle. It's not. It's a large model with an embedded space, tokens a…

We don't understand the brain. We fully understand what LLM are doing, humans built them. The idea we don't understand what LLMs are doing is magical. Magical is good for clicks and fundraising.

We know how we built the machines, but their complexity produces emergent behavior that we don't completely understand.

Re: On the Existence, Impact, and Origin of Hallucination-Associated Neurons in LLMs

#35

Post by tracker1 » Mon, Dec 22, 2025, 4:11 PM UTC

dudu24 wrote: post #29

airhangerf15 wrote: post #3

Earlier quoted context omitted.

LLMs don't "hallucinate" or "lie." They have no intent. They're Weighted Random Word Generator Machines. They're train mathematically to create series of tokens. Whenever they get something "right," it's literally by accident. If you get that rate of accidental rightness up to 80%, and people suddenly thing the random word generator is some kind of oracle. It's not. It's a large model with an embedded space, tokens a…

I'm tired of this pseudointellectual reductionist response. It's not "literally by accident" when they're trained to do something, as if we are not also machines that generate next actions based on learned neural weights and abstract (embedded) representations. Your issue is with semantics rather than content. Obviously "hallucinate" and "lie" are metaphors. Get over it. These are still emergent structures that we ha…

I just kind of wish the behavior for "hallucinations" just didn't have such confident language in the context... actual people will generally be relatively forthcoming at the edge of their knowledge or at least not show as much confidence. I know LLMs are a bit different, but that's about the best comparison I can come up with.

Re: On the Existence, Impact, and Origin of Hallucination-Associated Neurons in LLMs

#36

Post by crazygringo » Mon, Dec 22, 2025, 4:12 PM UTC

TomasBM wrote: post #32

crazygringo wrote: post #11

Earlier quoted context omitted.

> LLMs don't "hallucinate" or "lie." They have no intent. You're just arguing about semantics. It doesn't matter in any substantial way. Ultimately, we need a word to distinguish factual output from confidently asserted erroneous output. We use the word "hallucinate". If we used a different word, it wouldn't make any difference -- the observable difference remains the same. "Hallucinate" is the word that has emerged,…

While I agree that we need a word for this type of behavior, hallucinate is a wrong choice IMO. Hallucinations are already associated with a type of behavior, which is (roughly defined) "subjectively seeing/hearing things which aren't there". This is an input-level error, not the right umbrella term for the majority of errors happening with LLMs, many if which are at output-level. I don't know what would be a better…

I mean, I think you're right that confabulation is probably a more correct technical term, but we all use hallucinate now, so it doesn't really matter. It might have been useful to argue about it 4 or 5 years ago, but that ship has long since sailed. [1]

And I think we already distinguish between types of errors -- LLM's effectively don't lie, AFAIK, unless you're asking them to engage in role-play or something. They mostly either hallucinate/confabulate in terms of inventing knowledge they don't have, or they just make "mistakes" e.g. in arithmetic, or in attempting to copy large amounts of code verbatim.

And when you're interested in mistakes, you're generally interested in a specific category of mistakes, like arithmetic, or logic, or copying mistakes, and we refer to them as such -- arithmetic errors, logic errors, etc.

So I don't think hallucination is taking away from any kind of specificity. To the contrary, it is providing specificity, because we don't call arithmetic errors hallucinations. And we use the word hallucination precisely to distinguish it from these run-of-the-mill mistakes.

[1] https://trends.google.com/explore?q=hallucination&date=all&g...

Re: On the Existence, Impact, and Origin of Hallucination-Associated Neurons in LLMs

#37

Post by jurystillout26 » Mon, Dec 22, 2025, 4:19 PM UTC

Many people seem to be claiming that "LLMs do what humans do / humans also hallucinate", as if the process of human knowledge is identical to the purely semantic knowledge of LLMs.

No. Human beings have experiential, embodied, temporal knowledge of the world through our senses. That is why we can, say, empirically know something, which is vastly different than semantically or logically knowing something. Yes, human beings also have probabalistic ways of understanding the world and interacting with others. We have many other forms of knowledge as well and the LLM way of interpreting data is by no means the primary way in which we feel confident that something is true or false.

That said, I don't get up in arms about the term "hallucination", although I prefer the term confabulation per neuroscientist Anil Seth. Many clunky metaphors are now mainstream, and as long as the engineers and researchers who study these kinds of things are ok with that, that's the most important thing.

But what I think all these people who dismiss objections to the term as "arguing semantics" are missing is the fundamental point: LLMs have no intent, and they have no way of distinguishing what data is empirically true or not. This is why the framing, not just the semantics, of this piece is flawed. "Hallucinations" is a feature of LLMs that exists at the very conceptual level, not as a design flaw of current models. They have pattern recognition, which gets us very far in terms of knowing things, but people who only rely on such methods of knowing are most often referred to as conspiracy theorists.

Re: On the Existence, Impact, and Origin of Hallucination-Associated Neurons in LLMs

#38

Post by jaredcwhite » Mon, Dec 22, 2025, 5:01 PM UTC

2026iknewit wrote: post #23

airhangerf15 wrote: post #3

Earlier quoted context omitted.

LLMs don't "hallucinate" or "lie." They have no intent. They're Weighted Random Word Generator Machines. They're train mathematically to create series of tokens. Whenever they get something "right," it's literally by accident. If you get that rate of accidental rightness up to 80%, and people suddenly thing the random word generator is some kind of oracle. It's not. It's a large model with an embedded space, tokens a…

This is still not true. "Whenever they get something "right," it's literally by accident." "the random word generator" First of, the input is not random at all which allows the question how random the output is. Second, it compresses data which has an impact on that data. Probably cleaning or adjustment which should reduce 'random' even more. It compresses data from us into concepts. A high level concept is more robu…

OK, it's a semi-random word predictor.

Re: On the Existence, Impact, and Origin of Hallucination-Associated Neurons in LLMs

#39

Post by directorscut82 » Mon, Dec 22, 2025, 5:10 PM UTC

dudu24 wrote: post #29

airhangerf15 wrote: post #3

Earlier quoted context omitted.

LLMs don't "hallucinate" or "lie." They have no intent. They're Weighted Random Word Generator Machines. They're train mathematically to create series of tokens. Whenever they get something "right," it's literally by accident. If you get that rate of accidental rightness up to 80%, and people suddenly thing the random word generator is some kind of oracle. It's not. It's a large model with an embedded space, tokens a…

I'm tired of this pseudointellectual reductionist response. It's not "literally by accident" when they're trained to do something, as if we are not also machines that generate next actions based on learned neural weights and abstract (embedded) representations. Your issue is with semantics rather than content. Obviously "hallucinate" and "lie" are metaphors. Get over it. These are still emergent structures that we ha…

Of course they hallucinate because we are training on random mode. +Since you mentioned 3blue1brown there is an excellent video on ANN interpretation based on the works of famous researchers who attempt to provide plausible explanations about how these (transformers based) archs store and retrieve information. Randomness and stochasticity is literally the most basic components which allow all these billions of parameters to represent better embedding spaces almost hilbertian in nature and barely orthogonal as training progresses.

The "emergent structures" you are mentioning are just the outcome of randomness guided by "gradiently" descending to data landscapes. There is nothing to learn by studying these frankemonsters. All these experiments have been conducted in the past (decades past) multiple times but not at this scale.

We are still missing basic theorems, not stupid papers about which tech bro payed the highest electricity bill to "train" on extremely inefficient gaming hardware.

Re: On the Existence, Impact, and Origin of Hallucination-Associated Neurons in LLMs

#40

Post by dang » Mon, Dec 22, 2025, 6:02 PM UTC

dijksterhuis wrote: post #19

dang wrote: post #41

[stub for offtopicness] [submitters: one reason for not editorializing titles is it makes the threads be about that!]

> please use the original title, unless it is misleading or linkbait; don't editorialize. https://news.ycombinator.com/newsguidelines.html

Yes, changed above now. (Submitted title was "Origin of Hallucination in LLMs, The physical source of hallucinations has found")

Submitters: If you want to say what you think is important about an article, that's fine, but do it by adding a comment to the thread. Then your view will be on a level playing field with everyone else's: https://hn.algolia.com/?dateRange=all&page=0&prefix=false&so...