[stub for offtopicness]
[submitters: one reason for not editorializing titles is it makes the threads be about that!]
On the Existence, Impact, and Origin of Hallucination-Associated Neurons in LLMs
41–47 of 47 posts
Re: On the Existence, Impact, and Origin of Hallucination-Associated Neurons in LLMs
#42It is fairly well established that neurons in these artificial neural networks are polysemantic and information is represented in directions in the activation embedding space rather than neurons independently representing information (which is why anthropic is doing things like training sparse autoencoders). I haven't read the paper in depth but it seems like it is based on a fundamental misunderstanding about neuron…
It may be that there are just a few misfirings that dramatically degrade the results. Strokes in the human brain causing a few neurons to die can have outsized affects in the body...
Re: On the Existence, Impact, and Origin of Hallucination-Associated Neurons in LLMs
#43Earlier quoted context omitted.
LLMs don't "hallucinate" or "lie." They have no intent. They're Weighted Random Word Generator Machines. They're train mathematically to create series of tokens. Whenever they get something "right," it's literally by accident. If you get that rate of accidental rightness up to 80%, and people suddenly thing the random word generator is some kind of oracle. It's not. It's a large model with an embedded space, tokens a…
This is still not true. "Whenever they get something "right," it's literally by accident." "the random word generator" First of, the input is not random at all which allows the question how random the output is. Second, it compresses data which has an impact on that data. Probably cleaning or adjustment which should reduce 'random' even more. It compresses data from us into concepts. A high level concept is more robu…
LLMs work by “walking” “hyperspace” ? Don’t try to correct people if you don’t know what you’re talking about. LLMs are quite literally non-deterministic language models, and that’s all they are.
Re: On the Existence, Impact, and Origin of Hallucination-Associated Neurons in LLMs
#44Earlier quoted context omitted.
We don't understand the brain. We fully understand what LLM are doing, humans built them. The idea we don't understand what LLMs are doing is magical. Magical is good for clicks and fundraising.
We know how we built the machines, but their complexity produces emergent behavior that we don't completely understand.
This isn’t settled science, by the way. There’s evidence that the “emergent” behaviors are just a mirage.
Re: On the Existence, Impact, and Origin of Hallucination-Associated Neurons in LLMs
#45Many people seem to be claiming that "LLMs do what humans do / humans also hallucinate", as if the process of human knowledge is identical to the purely semantic knowledge of LLMs. No. Human beings have experiential, embodied, temporal knowledge of the world through our senses. That is why we can, say, empirically know something, which is vastly different than semantically or logically knowing something. Yes, human b…
On one hand this is true. On the other hand it does seem possible, in theory, that through enough post-training and other measures they could become a bit closer to human minds and not just be token guessers.
The human brain may at its fundamental level operate on the principles of predictive processing (https://slatestarcodex.com/2017/09/05/book-review-surfing-un...). It might be that it has many layers surrounding that raw predictive core which develop us into epistemological beings. The LLMs we see today may be in the very early stages of a similar sort of (artificial) evolution.
The reflexiveness with which even top models like Opus 4.5 will sometimes seamlessly confabulate things definitely does make it seem like it is a very deep problem, but I don't think it's necessarily unsolvable. I used to be among the vast majority of people who thought LLMs were not sufficient to get us to AGI/ASI, but I'm increasingly starting to feel that piling enough hacks atop LLMs might really be what gets there before anything else.
Re: On the Existence, Impact, and Origin of Hallucination-Associated Neurons in LLMs
#46Earlier quoted context omitted.
This is still not true. "Whenever they get something "right," it's literally by accident." "the random word generator" First of, the input is not random at all which allows the question how random the output is. Second, it compresses data which has an impact on that data. Probably cleaning or adjustment which should reduce 'random' even more. It compresses data from us into concepts. A high level concept is more robu…
LLMs work by “walking” “hyperspace” ? Don’t try to correct people if you don’t know what you’re talking about. LLMs are quite literally non-deterministic language models, and that’s all they are.
Yes hyperspace: "Hyperspace is a concept from physics and science fiction referring to a higher-dimensional space"
I prefer to write hyperspace instead of n-dimensional.
Feel free to explain to me why you think my description is wrong.
LLMs are not non-deterministic in their nature. We add noise/randomess into specific layers to make them more 'creative'/'engaging' instead of always getting the exact same response we get variations.
Your whole sentence doesn't even contain a real description of what a LLM is. You say 'LLM are random LLMs'.
I explained that in a different comment already, feel free to check them out.
Re: On the Existence, Impact, and Origin of Hallucination-Associated Neurons in LLMs
#47Earlier quoted context omitted.
This is still not true. "Whenever they get something "right," it's literally by accident." "the random word generator" First of, the input is not random at all which allows the question how random the output is. Second, it compresses data which has an impact on that data. Probably cleaning or adjustment which should reduce 'random' even more. It compresses data from us into concepts. A high level concept is more robu…
OK, it's a semi-random word predictor.
I'm lost on your comment.
Its like you have an agenda against LLMs by now marking them as 'semi-random' and undermining the complexity and the results we get from current LLMs.