Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Language models can explain neurons in language models

openai.com

Post reply on HN
Print view View original

291–300 of 497 posts

Re: Language models can explain neurons in language models

#291

Post by calf »

drdeca wrote: post #43
mrcode007 wrote: post #19

Earlier quoted context omitted.

If the Gödel incompleteness theorem applies here, then the explanations are likely … incomplete or self-referential.

What leads you to suspect that Gödel incompleteness may be relevant here? There's no formal axiom system being dealt with here, afaict? Do you just generally mean "there may be some kind of self-reference, which may lead to some kind of liar-paradox-related issues"?

The relevance is because all (all known buildable aka algorithmic, and sufficiently powerful) models of computation are equivalent in terms of formal computability, so if you could violate/bypass the Godel or Turing theorems in neural networks, then you could do it in a Turing machine, and vice versa. (That's my understanding, feel free to correct me if I'm mistaken)

Re: Language models can explain neurons in language models

#292

Post by [deleted] »

chimpanzee wrote: post #290
p1esk wrote: post #285

Earlier quoted context omitted.

GPT-4 is better at reasoning than 90% of humans. At least. I won't be surprised if GPT-5 is better than 100% of humans. I'm saying this in complete seriousness.

Do you put yourself in the 10% or the 90%? I’m asking in complete seriousness.

[deleted]

Re: Language models can explain neurons in language models

#293

Post by brookst »

ryandvm wrote: post #210
int_19h wrote: post #89

Of note: "... our technique works poorly for larger models, possibly because later layers are harder to explain." And even for GPT-2, which is what they used for the paper: "... the vast majority of our explanations score poorly ..." Which is to say, we still have no clue as to what's going on inside GPT-4 or even GPT-3, which I think is the question many want an answer to. This may be the first step towards that, bu…

Funny that we never quite understood how intelligence worked and yet it appears that we're pretty damn close to recreating it - still without knowing how it works. I wonder how often this happens in the universe...

Imitation -> emulation -> duplication -> revolution is a very common pattern in nature, society, and business. Aka “fake it til you make it”.

Think of business / artistic / cultural leaders nurturing protégés despite not totally understanding why they’re successful.

Of course those protégés have agency and drive, so maybe not a perfect analogy. But I’m going to stand by the point intuitively even if a better example escapes me.

Re: Language models can explain neurons in language models

#294

Post by slacka »

mensetmanusman wrote: post #206
dennisy wrote: post #159

Earlier quoted context omitted.

Nature created humans to understand nature. We created GPT4 to understand ourselves.

Humans are the universe asking who made it.

Yes, or put a bit more elegantly, 'The cosmos is within us. We are made of star-stuff. We are a way for the universe to know itself.' — Carl Sagan

Re: Language models can explain neurons in language models

#296

Post by [deleted] »

necroforest wrote: post #269
derefr wrote: post #264

Earlier quoted context omitted.

AI research has put hardly any effort into building goal-directed agents / A-Life since the advent of Machine Learning. A-Life was last really "looked into" in the '70s, back when "AI" meant Expert Systems and Behavior Trees. All the effort in AI research since the advent of Machine Learning, has been focused on making systems that — in neurological terms — are given a sensory stimulus of a question, and then passive…

> AI research has put hardly any effort into building goal-directed agents The entire (enormous) field of reinforcement learning begs to differ.

[deleted]

Re: Language models can explain neurons in language models

#297

Post by p1esk »

chimpanzee wrote: post #290
p1esk wrote: post #285

Earlier quoted context omitted.

GPT-4 is better at reasoning than 90% of humans. At least. I won't be surprised if GPT-5 is better than 100% of humans. I'm saying this in complete seriousness.

Do you put yourself in the 10% or the 90%? I’m asking in complete seriousness.

Oh it's definitely better than me at reasoning. I'm the one asking it to explain things to me, not the other way around.

Re: Language models can explain neurons in language models

#298

Post by andsoitis »

esafak wrote: post #284
akiselev wrote: post #244

Earlier quoted context omitted.

If you spent even more time with GPT-4 it would be evident that it is definitely not. Especially if you try to use it as some kind of autonomous agent.

I think we'll soon be able to train models that answer any reasonable question. By that measure, computers are intelligent, and getting smarter by the day. But I don't think that is the bar we care about. In the context of intelligence, I believe we care about self-directed thought, or agency. And a computer program needs to keep running to achieve that because it needs to interact with the world.

> I believe we care about self-directed thought, or agency.

If you can't enjoy it, is it worth it? Do AI's experience joy?

Re: Language models can explain neurons in language models

#299

Post by ChatGTP »

famouswaffles wrote: post #267
ChatGTP wrote: post #263

Earlier quoted context omitted.

Humans can be held accountable so it’s not the same. Even if we’re a black box, we share common traits with other humans. We’re trained in similar ways. So we mostly understand what we will and won’t do. I think this constant degradation of humans is really foolish and harmful personally. “We’re just black boxes etc”, we might not know how brains work but we do and can understand each other. On the other hand I’m sta…

>Humans can be held accountable so it’s not the same. 1. Don't worry, LLMs will be held accountable eventually. There's only so much embodiment and unsupervised tool control we can grant machines before personhood is in the best interests of everybody. May be forced like all the times in the past but it'll happen. 2. not every use case cares about accountability 3. accountability can be shifted. we have experience. >…

Degrading is calling an achievement we hold people in high regard who accomplish stupid because a machine can do it.

Not sure you worded this as intended ?

Anyway if I read you correctly, this assumes you believe the idea of self and ego have anything to do with it.

Humans should treat ants, lab rats and each other with equal respect.

I don’t believe we should avoid self-degradation because we think we’re smart or special, but for completely opposite reasons. We are mostly lucky we have what we have because something bigger than us, call it God, nature whatever, has provided that existence. When we degrade one another, we degrade that magic. This is where we fuck up time and time again. I’m talking about the water you drink, the food you eat and the air your breathe, the inspiration for neural networks etc. We take that for granted.

I liken it to the idea that humans killed God, the idea of God and morals etc, so we could do terrible things to the world and living things. We just got rid of the idea someone is looking over our shoulder because it made wars and genocides easier to do. Killing God got rid of a whole lot of moral baggage.

Re: Language models can explain neurons in language models

#300

Post by 8bitsrule »

taneq wrote: post #258
hackernewds wrote: post #250

Earlier quoted context omitted.

If you spent even more time with GPT-4 it would be evident that it definitely is. Especialy if you try to use it as some kind of autonomous agent. (Notice how baseless comments can sway either way)

> (Notice how baseless comments can sway either way) No they can’t! ;)

Let's let John Cleese decide. Or maybe someone was looking for Abuse!
Post reply on HN