On the Existence, Impact, and Origin of Hallucination-Associated Neurons in LLMs
1–10 of 47 posts
Re: On the Existence, Impact, and Origin of Hallucination-Associated Neurons in LLMs
#2[stub for offtopicness] [submitters: one reason for not editorializing titles is it makes the threads be about that!]
“Physical”? I don’t think that’s the right word to use
Re: On the Existence, Impact, and Origin of Hallucination-Associated Neurons in LLMs
#3[stub for offtopicness] [submitters: one reason for not editorializing titles is it makes the threads be about that!]
LLMs don't "hallucinate" or "lie." They have no intent. They're Weighted Random Word Generator Machines. They're train mathematically to create series of tokens. Whenever they get something "right," it's literally by accident. If you get that rate of accidental rightness up to 80%, and people suddenly thing the random word generator is some kind of oracle. It's not. It's a large model with an embedded space, tokens and a whole series of computationally expensive perceptron and attention blocks that generate output.
The title/introduction is very baited, because it implies some "physical" connection to hallucinations in biological organism, but it's focused on trying to single out certain parts of the model. LLMs are absolutely nothing at all like a biological system, of which our brains are orders of magnitudes more complex than the machines we've built that we no longer fully understand. Believing in these LLMs as being some next stage in understanding intelligence is hubris.
Re: On the Existence, Impact, and Origin of Hallucination-Associated Neurons in LLMs
#4[stub for offtopicness] [submitters: one reason for not editorializing titles is it makes the threads be about that!]
“Physical”? I don’t think that’s the right word to use
Agreed, the original title is better (and typo free) although a bit long.
H-Neurons: On the Existence, Impact, and Origin of Hallucination-Associated Neurons in LLMs
Re: On the Existence, Impact, and Origin of Hallucination-Associated Neurons in LLMs
#5[stub for offtopicness] [submitters: one reason for not editorializing titles is it makes the threads be about that!]
There is no such thing as a "hallucination" that could be isolated from "not a hallucination" in a provable systematic way because all they do is hallucinate.
I'm extremely comfortable calling this paper complete and utter bullshit (or, I suppose if I'm being charitable, extremely poorly titled) from the title alone.
Re: On the Existence, Impact, and Origin of Hallucination-Associated Neurons in LLMs
#6[stub for offtopicness] [submitters: one reason for not editorializing titles is it makes the threads be about that!]
“Physical”? I don’t think that’s the right word to use
Yeah, actual title is "H-Neurons: On the Existence, Impact, and Origin of Hallucination-Associated Neurons in LLMs"
But regardless of title this is all highly dubious...
Re: On the Existence, Impact, and Origin of Hallucination-Associated Neurons in LLMs
#7[stub for offtopicness] [submitters: one reason for not editorializing titles is it makes the threads be about that!]
LLMs don't "hallucinate" or "lie." They have no intent. They're Weighted Random Word Generator Machines. They're train mathematically to create series of tokens. Whenever they get something "right," it's literally by accident. If you get that rate of accidental rightness up to 80%, and people suddenly thing the random word generator is some kind of oracle. It's not. It's a large model with an embedded space, tokens a…
It’s been more than 3 years and people still can’t understand this.
Re: On the Existence, Impact, and Origin of Hallucination-Associated Neurons in LLMs
#8Earlier quoted context omitted.
LLMs don't "hallucinate" or "lie." They have no intent. They're Weighted Random Word Generator Machines. They're train mathematically to create series of tokens. Whenever they get something "right," it's literally by accident. If you get that rate of accidental rightness up to 80%, and people suddenly thing the random word generator is some kind of oracle. It's not. It's a large model with an embedded space, tokens a…
It’s been more than 3 years and people still can’t understand this.
People don't want to understand this.
Re: On the Existence, Impact, and Origin of Hallucination-Associated Neurons in LLMs
#9[stub for offtopicness] [submitters: one reason for not editorializing titles is it makes the threads be about that!]
LLMs don't "hallucinate" or "lie." They have no intent. They're Weighted Random Word Generator Machines. They're train mathematically to create series of tokens. Whenever they get something "right," it's literally by accident. If you get that rate of accidental rightness up to 80%, and people suddenly thing the random word generator is some kind of oracle. It's not. It's a large model with an embedded space, tokens a…
You're reasoning at the wrong abstraction level. This will not get you very far.
Re: On the Existence, Impact, and Origin of Hallucination-Associated Neurons in LLMs
#10From a quick read, this is cool but maybe a little overstated. From Figure 3, completely suppressing these neurons only reduces hallucinations by like ~5% compared to their normal state.
Table 1 is even more odd, H-neurons predicts hallucination ~75% of the time but a similar % of random neurons predict hallucinations ~60% of the time, which doesn't seem like a huge difference to me.