Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Language models can explain neurons in language models

openai.com

Post reply on HN
Print view View original

401–410 of 497 posts

Re: Language models can explain neurons in language models

#401

Post by stevenhuang »

Aerbil313 wrote: post #360
stevenhuang wrote: post #253

Earlier quoted context omitted.

Perhaps there is no purpose to consciousness. Perhaps it's a phenomenon that somehow arises independently ex nihilo from sufficiently complex systems, only ever able to observe, unable to act. Weird to think about.

Can conscious experience ever arise from matter? Even if the said matter is neural networks? This seems utterly nonsensical to me.

Do you consist of matter? Are you conscious? Are you aware the brain is a neural network?

Let's assume the premise that a form of neural network is necessary but insufficient to give rise to conscious experience. Then might it not matter whether the medium is physical or digital?

If you answer this with anything other than "we don't yet know", then you'll be wrong, because you'll be asserting a position beyond what science is able to currently provide; all of this is an open question. But hint: the evidence is mounting that yes, the medium might not matter.

Once you take on an information theoretic view of consciousness, a lot of possibilities and avenues of research open up.

https://en.m.wikipedia.org/wiki/Neural_correlates_of_conscio...

Re: Language models can explain neurons in language models

#402

Post by liamconnell »

srajabi wrote: post #3

"This work is part of the third pillar of our approach to alignment research: we want to automate the alignment research work itself. A promising aspect of this approach is that it scales with the pace of AI development. As future models become increasingly intelligent and helpful as assistants, we will find better explanations." On first look this is genius but it seems pretty tautological in a way. How do we know i…

Gpt2 answers to gpt3. Gpt3 answers to gpt4. Gpt4 answers to God.

Re: Language models can explain neurons in language models

#403

Post by ethanbond »

westurner wrote: post #392
ethanbond wrote: post #125

Earlier quoted context omitted.

The split brain experiments very very clearly indicate that different parts of the brain can independently conduct behavior and gain knowledge independently of other parts. How or if this generalizes to healthy brains is not super clear, but it does actually provide a good explanatory model for all sorts of self-contradictory behavior (like addiction): the brain has many semi-independent “interests” that are jockeyin…

Given that functional localization varies widely from subject to subject per modern neuroimaging, how are split brain experiments more than crude attempts to confirm functional specialization (which is already confirmed without traumatically severing a corpus callosum) "hemispheric" or "lateral"? Neuroimaging indicates high levels of redundancy and variance in spatiotemporal activation. Studies of cortices and other…

Because there’s a version of specialization that is, “different regions are specialized but they all seem to build consensus” and there’s a version that is “different regions are specialized and consensus does not seem to be necessary or potentially even usual or possible.”

These offer very different interpretations of cognition and behavior, and the split brain experiments point toward the latter.

Re: Language models can explain neurons in language models

#404

Post by westurner »

ethanbond wrote: post #403
westurner wrote: post #392

Earlier quoted context omitted.

Given that functional localization varies widely from subject to subject per modern neuroimaging, how are split brain experiments more than crude attempts to confirm functional specialization (which is already confirmed without traumatically severing a corpus callosum) "hemispheric" or "lateral"? Neuroimaging indicates high levels of redundancy and variance in spatiotemporal activation. Studies of cortices and other…

Because there’s a version of specialization that is, “different regions are specialized but they all seem to build consensus” and there’s a version that is “different regions are specialized and consensus does not seem to be necessary or potentially even usual or possible.” These offer very different interpretations of cognition and behavior, and the split brain experiments point toward the latter.

Functional specialization > Major theories of the brain> Modularity or/and Distributive processing: https://en.wikipedia.org/wiki/Functional_specialization_(bra... :

> Modularity: [...] The difficulty with this theory is that in typical non-lesioned subjects, locations within the brain anatomy are similar but not completely identical. There is a strong defense for this inherent deficit in our ability to generalize when using functional localizing techniques (fMRI, PET etc.). To account for this problem, the coordinate-based Talairach and Tournoux stereotaxic system is widely used to compare subjects' results to a standard brain using an algorithm. Another solution using coordinates involves comparing brains using sulcal reference points. A slightly newer technique is to use functional landmarks, which combines sulcal and gyral landmarks (the groves and folds of the cortex) and then finding an area well known for its modularity such as the fusiform face area. This landmark area then serves to orient the researcher to the neighboring cortex. [7]

Is there a way to address the brain with space-filling curves around ~loci/landmarks? For brain2brain etc

FWIU, Markham's lab found that the brain is at max 11D in some places; But an electron wave model (in the time domain) may or must be sufficient according to psychoenergetics (Bearden)

> Distributive processing: [...] McIntosh's research suggests that human cognition involves interactions between the brain regions responsible for processes sensory information, such as vision, audition, and other mediating areas like the prefrontal cortex. McIntosh explains that modularity is mainly observed in sensory and motor systems, however, beyond these very receptors, modularity becomes "fuzzier" and you see the cross connections between systems increase.[33] He also illustrates that there is an overlapping of functional characteristics between the sensory and motor systems, where these regions are close to one another. These different neural interactions influence each other, where activity changes in one area influence other connected areas. With this, McIntosh suggest that if you only focus on activity in one area, you may miss the changes in other integrative areas.[33] Neural interactions can be measured using analysis of covariance in neuroimaging [...]

FWIU electrons are most appropriately modeled with Minkowski 4-space in the time-domain; (L^3)t

Neuroplasticity: https://en.wikipedia.org/wiki/Neuroplasticity :

> The adult brain is not entirely "hard-wired" with fixed neuronal circuits. There are many instances of cortical and subcortical rewiring of neuronal circuits in response to training as well as in response to injury.

> There is ample evidence [53] for the active, experience-dependent re-organization of the synaptic networks of the brain involving multiple inter-related structures including the cerebral cortex.[54] The specific details of how this process occurs at the molecular and ultrastructural levels are topics of active neuroscience research. The way experience can influence the synaptic organization of the brain is also the basis for a number of theories of brain function

Re: Language models can explain neurons in language models

#405

Post by westurner »

westurner wrote: post #404
ethanbond wrote: post #403

Earlier quoted context omitted.

Because there’s a version of specialization that is, “different regions are specialized but they all seem to build consensus” and there’s a version that is “different regions are specialized and consensus does not seem to be necessary or potentially even usual or possible.” These offer very different interpretations of cognition and behavior, and the split brain experiments point toward the latter.

Functional specialization > Major theories of the brain> Modularity or/and Distributive processing: https://en.wikipedia.org/wiki/Functional_specialization_(bra... : > Modularity: [...] The difficulty with this theory is that in typical non-lesioned subjects, locations within the brain anatomy are similar but not completely identical. There is a strong defense for this inherent deficit in our ability to generalize wh…

"EM Wave Polarization Transductions" Lt. Col. T.E Bearden (1999) :

> Physical observation (via the transverse photon interaction) is the process given by applying the operator ∂/∂t to (L^3)t, yielding an L3 output.

Re: Language models can explain neurons in language models

#406

Post by westurner »

westurner wrote: post #404
ethanbond wrote: post #403

Earlier quoted context omitted.

Because there’s a version of specialization that is, “different regions are specialized but they all seem to build consensus” and there’s a version that is “different regions are specialized and consensus does not seem to be necessary or potentially even usual or possible.” These offer very different interpretations of cognition and behavior, and the split brain experiments point toward the latter.

Functional specialization > Major theories of the brain> Modularity or/and Distributive processing: https://en.wikipedia.org/wiki/Functional_specialization_(bra... : > Modularity: [...] The difficulty with this theory is that in typical non-lesioned subjects, locations within the brain anatomy are similar but not completely identical. There is a strong defense for this inherent deficit in our ability to generalize wh…

"Representational drift: Emerging theories for continual learning and experimental future directions" (2022) https://www.sciencedirect.com/science/article/pii/S095943882... :

> Recent work has revealed that the neural activity patterns correlated with sensation, cognition, and action often are not stable and instead undergo large scale changes over days and weeks—a phenomenon called representational drift. Here, we highlight recent observations of drift, how drift is unlikely to be explained by experimental confounds, and how the brain can likely compensate for drift to allow stable computation. We propose that drift might have important roles in neural computation to allow continual learning, both for separating and relating memories that occur at distinct times. Finally, we present an outlook on future experimental directions that are needed to further characterize drift and to test emerging theories for drift's role in computation.

So, to run the same [fMRI, NIRS,] stimulus response activation observation/burn-in again weeks or months later with the same subjects is likely necessary given Representational drift.

Re: Language models can explain neurons in language models

#407

Post by p1esk »

chimpanzee wrote: post #372
p1esk wrote: post #342

Earlier quoted context omitted.

What would be an example of “non-deductive” reasoning, which requires embodied perceptual experiences?

“God, that felt great!” As detailed as possible, describe what happened.

I have no idea what happened. I don’t even know what you expect me to describe. Someone feels great about something? And I don’t know what it has to do with reasoning.

Re: Language models can explain neurons in language models

#408

Post by YawningAngel »

TeMPOraL wrote: post #224
YawningAngel wrote: post #76

Earlier quoted context omitted.

What if you do? LLMs don't have reflexive output or internal streams of thought, they are simply (complex) processes that produce streams of tokens based on an inputted stream of tokens. They don't have a special response to tokens that indicate higher-level thinking to humans.

LLMs seem to me to be the "internal streams of thought". I.e. it's not LLMs that are missing an internal process that humans have, but rather it's humans that have an entire process of conscious thinking built on top of something akin to LLM.

That's possible I guess but is there positive evidence for that being the case?

Re: Language models can explain neurons in language models

#409

Post by TeMPOraL »

jxf wrote: post #235
TeMPOraL wrote: post #179

Earlier quoted context omitted.

To be honest, this description is leaning heavily on the associations we have with individual words used. Ant "architecture" isn't like our architecture. Ant "plumbing" and "ventilation" have little in common with the kind of plumbing and ventilation we use in buildings. "Nurseries", "rearing the young", that's just stretching the analogy to the point of breaking. "Agriculture", "animal husbandry" - I don't even know…

You could say the same in reverse. Humans can't lift fifty times their own weight. Humans can't communicate in real-time with pheromones alone. Most humans do not know how to build their own home. An ant might well consider us backwards, not advanced.

> Humans can't lift fifty times their own weight.

Ants can do it as a consequence of their size. Relative lifting strength drops fast with increased size. Conversely, an ant scaled to human size would collapse under its own weight and cook itself to death - waste heat generation scales with volume (~ size³), while waste heat rejection scales with surface area (~ size²).

And it ain't a cognitive achievement anyway.

> Humans can't communicate in real-time with pheromones alone.

Yes. Because it's not as useful at our scale, nor is it real-time - chemical communication works better for small organisms and small volumes of living, as the travel speed and dissipation rate of pheromones is independent of organisms emitting them. Meanwhile, we have multiple ways of communicating real-time, some of which work at light speed which is the definition of "real time" (light speed in vacuum being the speed of causality itself).

> Most humans do not know how to build their own home.

Neither do ants.

> An ant might well consider us backwards, not advanced.

An ant can't consider us anything. The point I'm trying to get across is, just because the ant colony is capable of surprisingly high sophistication, doesn't mean the individual ants are.

As a counterpoint that's actually cognitive in nature: AFAIK individuals of many (most?) ant species can be easily tricked into following each other in a circle, and they will continue to do so until they start dying from starvation. This is because the ant isn't making a decision following a complex thought process - it's executing a simple algorithm, that works well in nature because nature is chaotic enough that stable ant circles are unlikely to form (and when they do, they're unlikely to be left undisturbed for long).

Re: Language models can explain neurons in language models

#410

Post by gitfan86 »

Noble_Lie wrote: post #314
didericis wrote: post #246

Earlier quoted context omitted.

> It might have been an interesting argument 20 years ago. It’s just silly now. Is it? These networks are capable of copying something , yes. Do we have a good understanding of what that is? Not really, no. At least I don’t. I’m sure lots of people have a much better understanding than I do, but I think its hard to know exactly whats going on. People dismiss the stochastic parrot argument because of how impressive bi…

That person's argument is borderline insane to me - a severe lack of knowing what is unknown, a reverence of current best-models (regards modern science, including neurology - yet, open minded investigations beyond are also a requisite here.) And the pompousness is what truly boggles my mind ("Its silly to believe this , now .) A look in the mirror would suffice to say the least... Anyway, thank you for a great answe…

Your theory makes sense in an evolutionary context. It is possible that all cells and organisms have some general intelligence. Humans do not have the ability to recognize this because evolutionarily it was only helpful to recognize intelligence when it could pose a threat to us. And the biggest threat to us in general was other humans as we are tribal animals. So we don't see it, we only see specialized intelligence that historically posed a threat to us.

It would explain why most "experts" didn't see GTP-4's abilities coming. Many of them expected that it would take a major algorithm or technology improvement to do "real intelligent" things, because they fundamentally misunderstood intelligence.

Post reply on HN