Language models can explain neurons in language models • Hacker News Board

Re: Language models can explain neurons in language models

#461

Post by snapcaster » Thu, May 11, 2023, 1:17 PM UTC

causalmodels wrote: post #37

snapcaster wrote: post #13

Earlier quoted context omitted.

Agreed, Yud does seem to have been right about the course things will take but I'm not confident he actually has any solutions to the problem to offer

His solution is a global regulatory regime to ban new large training runs. The tools required to accomplish this are, IMO, out of the question but I will give Yud credit for being honest about them while others who share his viewpoint try to hide the ball.

Good point, but I think if he was serious he would be looking to Serena Butler for inspiration not nuclear non-proliferation treaties

Re: Language models can explain neurons in language models

#462

Post by tdrgabi » Thu, May 11, 2023, 2:10 PM UTC

k99x55 wrote: post #434

red_admiral wrote: post #398

Earlier quoted context omitted.

Also Darwin published a theory of evolution, and Mendel discovered genetics, before anyone even thought of the term "double helix".

No need for genetics to understand evolution...

But they "didn't know how it worked", DNA was not discovered yet

Re: Language models can explain neurons in language models

#463

Post by chaxor » Thu, May 11, 2023, 2:12 PM UTC

richardw wrote: post #266

p1esk wrote: post #219

Earlier quoted context omitted.

If you spent any time with GPT-4 it should be evident.

Still a while to go. I think there's at least a couple of algorithmic changes needed before we move to a system that says "You have the world's best god-like AI and you're asking me for poems. Stop wasting my time because we've got work to do. Here's what I want YOU to do."

This isn't a necessary part of intelligence.

Re: Language models can explain neurons in language models

#464

Post by ethanbond » Thu, May 11, 2023, 2:54 PM UTC

haldujai wrote: post #451

ethanbond wrote: post #442

Earlier quoted context omitted.

I'm not "ignoring" them at all. I'm saying that they point to interesting questions that are not answered. > The split-brain experiments are notable because the lab experiments SUGGEST the lack of communication between two hemispheres and a split conscious however this is paradoxical with everyday experience of these patients, far from providing evidence for anything. It is not "paradoxical" but yes it does conflict…

I'm arguing against the strength of your statements based on methodologically unsound experiments that do not "very very clearly indicate" anything beyond pose a few questions for which there are several different hypothetical answers. All of which have zero evidence behind them. Similarly, the initial comment of 'does the brain know what the brain is doing. The answer so far does not seem to be "yes."' is misleading…

The split brain experiments clearly indicate exactly what I said they clearly indicate. To refresh: "that different parts of the brain can independently conduct behavior and gain knowledge independently of other parts".

From your own linked article: "In short, callosotomy leads to a broad breakdown of functional integration ranging from perception to attention."

If the dopamine reward system is the full answer, then what explains drug addicts anguishing about their addiction while simultaneously actively seeking out their next hit? What part of the brain is producing the anguish if the whole behavior is conclusively described by the dopamine reward system?

Re: Language models can explain neurons in language models

#465

Post by fennecfoxy » Thu, May 11, 2023, 3:11 PM UTC

ryandvm wrote: post #210

int_19h wrote: post #89

Of note: "... our technique works poorly for larger models, possibly because later layers are harder to explain." And even for GPT-2, which is what they used for the paper: "... the vast majority of our explanations score poorly ..." Which is to say, we still have no clue as to what's going on inside GPT-4 or even GPT-3, which I think is the question many want an answer to. This may be the first step towards that, bu…

Funny that we never quite understood how intelligence worked and yet it appears that we're pretty damn close to recreating it - still without knowing how it works. I wonder how often this happens in the universe...

I feel like OAI's approach is kind of wrong. GPT4 is still just text transformation/completion with multi-headed attention for better prediction of the next word that should follow (versus only looking at the previous word).

In human brains, language is only a way to communicate thoughts in concept form, though we also seem to use language to communicate abstract thoughts to ourselves to break them apart/down in a way (imo).

I'd love to see someone train a model on the level of GPT4 to generate abstract thoughts/ideas based on input/context and then pair this model with GPT4 co-operatively and continue to train, such that the flow of abstract ideas is parsed by GPT. But like...how do you even train a model that operates on abstract ideas, there doesn't seem to be any way to do this.

Re: Language models can explain neurons in language models

#466

Post by Aerbil313 » Thu, May 11, 2023, 4:35 PM UTC

ly3xqhl8g9 wrote: post #446

Aerbil313 wrote: post #440

Earlier quoted context omitted.

Sorry, my english is not the best and I don't think there is a word for the thing I'm trying to explain. Meaning of 'consciousness' is too messy. I know brain is a neural network. I just don't understand how cold, hard matter can result in this experience of consciousness we are living right now. The experience. Me. You. Perceiving. Right now. I'm not talking about the relation between the brain and our conscious exp…

You are referring to qualia [1]: feeling something as-is , subjectively. I had similar metaphysics just a few years ago, but in the end, it's just that: bad metaphysics. And it's not even your or my fault: with 2,400+ years of bad metaphysics it's almost only luck to be able to pierce the veil into the metaphysics of tomorrow. The main point is that with the tremendous discoveries of people such as Church/Turing (mat…

I did not mean qualia, I knew qualia. I meant that my experience is not matter. Matter is matter and charges are charges, my experience right now is neither. Your perception, the act of you experiencing life right now is not matter.

> We looked insanely deep into the brain [4], there is no magic going on.

Indeed all computation and input collection and such happen in the brain. I just don't understand how I can experience anything if I'm composed only of matter. How come there happens to be a mind? Indeed the electrochemical charges from visual receptors in the eye will be transmitted and computed and memory and dopamine and all the neurons will fire regardless of whether I'm only matter or not. But how can the experiencing consciousness, 'me' arise from matter?

> only a simulation can be conscious, not physical systems

This is what I'm talking about, only that I don't see why simulations are not physical systems.

> yes, you are electrochemical charges, we all are, what else could we be?

It's nonsensical and unscientific to completely rule out the possibility that we can be something else as well, especially when we can't study it directly, like in the example of soul.

Re: Language models can explain neurons in language models

#467

Post by falsissime » Thu, May 11, 2023, 5:40 PM UTC

philomath_mn wrote: post #344

DaiPlusPlus wrote: post #324

Earlier quoted context omitted.

> But how is any information that isn't testable trusted? I'm open to the idea ChatGPT is as credible as experts in the dismal sciences given that information cannot be proven or falsified and legitimacy is assigned by stringing together words that "makes sense". I understand that around the 1980s-ish, the dream was that people could express knowledge in something like Prolog, including the test-case, which can then…

I bet GPT is really good at prolog, that would be interesting to explore. "Answer this question in the form of a testable prolog program"

You lost this bet: Write append3/4 which appends three lists to a fourth, such that append3(Xs,Ys,[e],[]) terminates.

Re: Language models can explain neurons in language models

#468

Post by haldujai » Thu, May 11, 2023, 9:05 PM UTC

ethanbond wrote: post #464

haldujai wrote: post #451

Earlier quoted context omitted.

I'm arguing against the strength of your statements based on methodologically unsound experiments that do not "very very clearly indicate" anything beyond pose a few questions for which there are several different hypothetical answers. All of which have zero evidence behind them. Similarly, the initial comment of 'does the brain know what the brain is doing. The answer so far does not seem to be "yes."' is misleading…

The split brain experiments clearly indicate exactly what I said they clearly indicate. To refresh: "that different parts of the brain can independently conduct behavior and gain knowledge independently of other parts". From your own linked article: "In short, callosotomy leads to a broad breakdown of functional integration ranging from perception to attention." If the dopamine reward system is the full answer, then…

Breakdown of functional integration =/= different parts of the brain can independently conduct behavior and gain knowledge.

Still not sure what the contradiction is, is it regret now? Cause that isn’t contradictory.

It’s pretty simple, drugs feel really good when you take them so a single consciousness prioritizes that feeling over long term interests. When one is not taking them and facing the consequences of those decisions they feel bad. To make the bad feelings go away one takes more drugs and the cycle repeats.

Re: Language models can explain neurons in language models

#469

Post by int_19h » Thu, May 11, 2023, 9:47 PM UTC

YeGoblynQueenne wrote: post #456

int_19h wrote: post #447

Earlier quoted context omitted.

If it performs a computation, it is by definition running some algorithm regardless of how it's implemented in hardware / wetware. How is it a stretch? The only way our brains could be not algorithmic is if something like soul is a real thing that actually drives our intelligence.

Why? Rain is not algorithmic, clouds are not algorithmic, waves in the sea are not algorithmic, yet they are entirely physical processes that have nothing to do with souls.

None of these are computational processes, unlike intelligence. Unless you're trying to argue that intelligence isn't, either?

Re: Language models can explain neurons in language models

#470

Post by krainboltgreene » Fri, May 12, 2023, 1:17 AM UTC

int_19h wrote: post #382

AdieuToLogic wrote: post #335

Earlier quoted context omitted.

LLM's are not a "smart human being." They are predictive statistical models capable of producing results based on training data. LLM's do not think. LLM's are algorithms.

Your brain is also basically an algorithm that produces results based on training data. It's just a much more complicated and flexible one.

You people are insufferable.