Language models can explain neurons in language models • Hacker News Board

Re: Language models can explain neurons in language models

#411

Post by istjohn » Wed, May 10, 2023, 12:30 PM UTC

Noble_Lie wrote: post #314

didericis wrote: post #246

Earlier quoted context omitted.

> It might have been an interesting argument 20 years ago. It’s just silly now. Is it? These networks are capable of copying something , yes. Do we have a good understanding of what that is? Not really, no. At least I don’t. I’m sure lots of people have a much better understanding than I do, but I think its hard to know exactly whats going on. People dismiss the stochastic parrot argument because of how impressive bi…

That person's argument is borderline insane to me - a severe lack of knowing what is unknown, a reverence of current best-models (regards modern science, including neurology - yet, open minded investigations beyond are also a requisite here.) And the pompousness is what truly boggles my mind ("Its silly to believe this , now .) A look in the mirror would suffice to say the least... Anyway, thank you for a great answe…

Please read the comment I was responding to. I was addressing the suggestion that perhaps the brain is as relevant to intelligence as the eye. Cognitive Neuroscience has been a thriving field for nearly half a century now. I didn't say we have it all figured out, just that it's obvious neurons are a piece of the puzzle.

Re: Language models can explain neurons in language models

#412

Post by TeMPOraL » Wed, May 10, 2023, 12:31 PM UTC

ASalazarMX wrote: post #187

TeMPOraL wrote: post #179

Earlier quoted context omitted.

To be honest, this description is leaning heavily on the associations we have with individual words used. Ant "architecture" isn't like our architecture. Ant "plumbing" and "ventilation" have little in common with the kind of plumbing and ventilation we use in buildings. "Nurseries", "rearing the young", that's just stretching the analogy to the point of breaking. "Agriculture", "animal husbandry" - I don't even know…

> There's a vast difference in complexity between what ants do, and what humans do. Interesting parallell with intelligence/sentience/sapience. Despite the means, isn't the end result what you have to judge? The end result looks like a rudimentary civilization. How much back in time would we have to go back to find more sophistication in ant societies than humans?

> Despite the means, isn't the end result what you have to judge? The end result looks like a rudimentary civilization.

I'm indeed talking about the ends. The operative phrase here is "it looks like". It looks like a civilization, there are even some structural similarities to be found - but it is not a civilization, anymore than a Lego brick with a computer sticker on it is a PC.

This is not to deny that ants are awesome and very much worth studying - my point is to avoid following flawed analogies too far. "Ant colonies look like rudimentary civilization, and civilization is/does X, therefore..." is going to be a nonsense statements for most of possible values of X.

Re: Language models can explain neurons in language models

#413

Post by TeMPOraL » Wed, May 10, 2023, 12:40 PM UTC

YawningAngel wrote: post #408

TeMPOraL wrote: post #224

Earlier quoted context omitted.

LLMs seem to me to be the "internal streams of thought". I.e. it's not LLMs that are missing an internal process that humans have, but rather it's humans that have an entire process of conscious thinking built on top of something akin to LLM.

That's possible I guess but is there positive evidence for that being the case?

The way I phrased it isn't exactly structured to admit any kind of evidence, so let me destructure it. My observation is that:

- The argument that LLMs are missing introspection / inner voice is based on attempting to compare LLMs directly with human minds.

- Human minds have conscious and unconscious parts; for many people, part of the boundary between unconscious and conscious mind manifests as the "inner voice" - the one that makes verbalized thoughts "appear" in their head (or rather perhaps become consciously observed).

- Based entirely on my own experience in using GPT-3.5 and GPT-4, and my own introspection, I feel that GPT-4 bears a lot of resemblance to my inner voice in terms of functioning.

- Therefore I propose that comparing LLMs directly to human minds is unproductive, and it's much more interesting/useful to compare them to the inner voice in human minds: the part of the boundary between unconscious and conscious that uses natural language for I/O.

Re: Language models can explain neurons in language models

#414

Post by jwestbury » Wed, May 10, 2023, 12:45 PM UTC

mr_toad wrote: post #236

otabdeveloper4 wrote: post #160

Earlier quoted context omitted.

There is no evidence that intelligence runs on neurons. Yes, there are neurons in brains, but there's also lots of other stuff in there too. And there are creatures that exhibit intelligent properties even though they have hardly any neurons at all. (An individual ant has only something like 250000 neurons, and yet they're the only creatures beside humans that managed to create a civilization.)

> There is no evidence that intelligence runs on neurons. 1. Neurons connect all our senses and all our muscles. 2. Neurons are the definitive difference between the brain and the rest of the body. There is “other stuff” in the brain, but it’s not so different from the “other stuff” that’s in your rear end. Don’t underestimate what a neuron can do. A single artificial neuron can fit a logistic regression model. A qua…

> 2. Neurons are the definitive difference between the brain and the rest of the body. There is “other stuff” in the brain, but it’s not so different from the “other stuff” that’s in your rear end.

Our digestive systems appear to be important to our behaviour, though. Some recent work in mice showed that if colonised with bacteria from faeces of humans with autism, the mice would begin to show autistic behaviours.

So, not sure your argument here is especially strong.

Re: Language models can explain neurons in language models

#415

Post by jwestbury » Wed, May 10, 2023, 12:53 PM UTC

account-5 wrote: post #332

jxf wrote: post #169

Earlier quoted context omitted.

I'll repost a comment via Reddit that I think makes this case [0]: Ants have developed architecture, with plumbing, ventilation, nurseries for rearing the young, and paved thoroughfares. Ants practice agriculture, including animal husbandry. Ants have social stratification that differs from but is comparable to that of human cultures, with division of labor into worker, soldier, and other specialties that do not have…

Do termites count as well then? Bees? I'm not being facetious, but these insects seem similar.

Perhaps an important point here is that ants pass the mirror test, which, to my knowledge, no other insects (that we know of) do.

Re: Language models can explain neurons in language models

#416

Post by burnished » Wed, May 10, 2023, 1:08 PM UTC

resource0x wrote: post #282

burnished wrote: post #183

Earlier quoted context omitted.

What else would intelligence run on?

Every cell in our body, and every bacterium living in a body (e.g. gut flora), contribute to our intelligence. It looks plausible (to me) that there's one "top cell" among them that represents the "person", others just contributing via layered signals, but whether this "top cell" is a neuron or another kind of cell is unknown.

Ah, so you're not studied in neuroscience with some revolutionary theory. A shame.

Re: Language models can explain neurons in language models

#417

Post by runlaszlorun » Wed, May 10, 2023, 1:18 PM UTC

ryandvm wrote: post #210

int_19h wrote: post #89

Of note: "... our technique works poorly for larger models, possibly because later layers are harder to explain." And even for GPT-2, which is what they used for the paper: "... the vast majority of our explanations score poorly ..." Which is to say, we still have no clue as to what's going on inside GPT-4 or even GPT-3, which I think is the question many want an answer to. This may be the first step towards that, bu…

Funny that we never quite understood how intelligence worked and yet it appears that we're pretty damn close to recreating it - still without knowing how it works. I wonder how often this happens in the universe...

We don’t know…

Re: Language models can explain neurons in language models

#418

Post by runlaszlorun » Wed, May 10, 2023, 1:25 PM UTC

int_19h wrote: post #382

AdieuToLogic wrote: post #335

Earlier quoted context omitted.

LLM's are not a "smart human being." They are predictive statistical models capable of producing results based on training data. LLM's do not think. LLM's are algorithms.

Your brain is also basically an algorithm that produces results based on training data. It's just a much more complicated and flexible one.

But it’s also based on neurons with far more complex behavior than artificial neurons and also has other separate dynamic systems involving neurochemicals, various effects across the nervous system and the rest of the body (the gut becoming seemingly more and more relevant), various EEG patterns, and most likely quantum effects.

I personally wouldn’t rule out that it can’t be emulated in a different substrate, but I think calling it “an algorithm” is to def stretch and misapply the usefulness of the term.

Re: Language models can explain neurons in language models

#419

Post by pfdietz » Wed, May 10, 2023, 1:35 PM UTC

olalonde wrote: post #333

ryandvm wrote: post #210

Earlier quoted context omitted.

Funny that we never quite understood how intelligence worked and yet it appears that we're pretty damn close to recreating it - still without knowing how it works. I wonder how often this happens in the universe...

The battery (Voltaic Pile, 1800) and the telegraph (1830s-1840s) were both invented before the electron was discovered (1897).

Hell, consider all the technology that was developed before science really got going.

Re: Language models can explain neurons in language models

#420

Post by thomastjeffery » Wed, May 10, 2023, 1:45 PM UTC

chaxor wrote: post #378

poulpy123 wrote: post #362

Earlier quoted context omitted.

It's a neural network. Neural network are not symbolic AI and are not designed to reason

There's a decent working paper that has benchmarks on this, if you're interested. There are many types of reasoning, but GPT-4 gets 97% on casual discovery, and 92% on counterfactuals (only 6% off from human, btw) with 86% on actual causality benchmarks. I'm not sure yet if the question is correct, or even appropriate/achievable to what many may want to ask (i.e. what 'the public's is interested in is typically lost…

Percent of what? Possible right or wrong answers to a test?

Remember that GPT is not trained on all possible text. It's trained on text that was written intentionally. What percentage of that text contains "correct" instances of causal discovery, counterfactuals, etc.?