Live data from Hacker News

Language models can explain neurons in language models

openai.com

311–320 of 497 posts

Re: Language models can explain neurons in language models

#311
post #297

Earlier quoted context omitted.

Do you put yourself in the 10% or the 90%? I’m asking in complete seriousness.

Oh it's definitely better than me at reasoning. I'm the one asking it to explain things to me, not the other way around.

If you think it's better than you at reasoning then you cannot at all be confident in the truth of it's dialog.

Re: Language models can explain neurons in language models

#312
post #297

Earlier quoted context omitted.

Do you put yourself in the 10% or the 90%? I’m asking in complete seriousness.

Oh it's definitely better than me at reasoning. I'm the one asking it to explain things to me, not the other way around.

Ah ok. Here you use the word “explain” which implies more of a descriptive, reducing action rather than extrapolative and constructive. As in, it can explain what it has “read” (and it has obviously “read” far more than any human), but it can’t necessarily extrapolate beyond that or use that to find new truths. To me reasoning is more about the extrapolative, truth-finding process, ie “wisdom” from knowledge rather than just knowledge. But maybe my definition of “reasoning” isn’t quite right.

Edit: I probably should define reasoning as solely “deductive reasoning”, in which case, perhaps it is better than humans. But that seems like a premature claim. On the other hand, non-deductive reasoning, I have yet to see from it. I personally can’t imagine how it could do so reliably (from a human perspective) without real-world experiences and perceptions. I’m the sort that believes a true AGI would require a highly-perceptual, space-occupying organ. In other words it would have to be and “feel” embodied, in time and space, in order to perform other forms of reasoning.

Re: Language models can explain neurons in language models

#313
post #249

Earlier quoted context omitted.

What have you tried to do with it?

Use it to analyze the California & US Code, the California & Federal Codes of Regulation, and bills currently in the California legislation & Congress. It's far from useless but far more useful for creative writing than any kind of understanding or instruction following when it comes to complex topics. Even performing a map-reduce over large documents to summarize or analyze them for a specific audience is largely be…

Interesting - do you believe average humans (not professional lawyers) would do better on this task?

Re: Language models can explain neurons in language models

#314

Earlier quoted context omitted.

Why would you doubt neurons play a roll in intelligence when we've seen so much success in emulating human intelligence with artificial neural networks? It might have been an interesting argument 20 years ago. It's just silly now.

> It might have been an interesting argument 20 years ago. It’s just silly now. Is it? These networks are capable of copying something , yes. Do we have a good understanding of what that is? Not really, no. At least I don’t. I’m sure lots of people have a much better understanding than I do, but I think its hard to know exactly whats going on. People dismiss the stochastic parrot argument because of how impressive bi…

That person's argument is borderline insane to me - a severe lack of knowing what is unknown, a reverence of current best-models (regards modern science, including neurology - yet, open minded investigations beyond are also a requisite here.) And the pompousness is what truly boggles my mind ("Its silly to believe this, now.) A look in the mirror would suffice to say the least...

Anyway, thank you for a great answer and conversation throughout this thread.

Regards neural networks, parroting and the emulation of intelligence (or the difference between an emulation and the "real thing"):

Well, somewhat like you say, we cannot propose a valid comparison from one to the other without an understanding of one (consciousness) or both. It's fascinating that there are some open, valid and pressing questions about what / how the output of this new wave of software is concretized (from foundational, semi-statistical algorithms in this case.)

Yes, I do agree neurons have something to do with the "final output". But this is a subsection of the problem - organic neurons is-an/are order(s) of magnitude in complexity beyond what the tricky "parrot" is up to. Moreso, these components perform very different functionally - the known functions of the neuron compared to ANN, backprop etc. The entire stack.)

P.S: One interesting theory I like to simulate and/or entertain is that every organic cell in the body has something to do with the final output of consciousness.

Re: Language models can explain neurons in language models

#315
post #297

Earlier quoted context omitted.

Oh it's definitely better than me at reasoning. I'm the one asking it to explain things to me, not the other way around.

If you think it's better than you at reasoning then you cannot at all be confident in the truth of it's dialog.

I am not. I treat it as I'd treat any smart human being.

Re: Language models can explain neurons in language models

#316
post #297

Earlier quoted context omitted.

Oh it's definitely better than me at reasoning. I'm the one asking it to explain things to me, not the other way around.

Ah ok. Here you use the word “explain” which implies more of a descriptive, reducing action rather than extrapolative and constructive. As in, it can explain what it has “read” (and it has obviously “read” far more than any human), but it can’t necessarily extrapolate beyond that or use that to find new truths. To me reasoning is more about the extrapolative, truth-finding process, ie “wisdom” from knowledge rather t…

Why don't you suggest an example we can run and see what it's capable of (compared to what I, or other humans, are capable of)?

Re: Language models can explain neurons in language models

#317

Earlier quoted context omitted.

What's the argument that understanding neurons is necessary? Perhaps intelligence is like a black box input to our bodies (call it the "soul", even though this isn't testable and therefore not a hypothesis). The mind therefore wouldn't play any more of a role in intelligence than the eye. And I'm not sure people would say the eye is necessary for understanding intelligence. Now, I'm not really in a position to argue…

You can actually hypothesize that a soul exists and that intelligence is non-material, its just that your tests would quickly disprove that hypothesis - crude physical, mechanical modifications to the brain cause changes to intellect and character. If your hypothesis was correct you would not expect to see changes like that at all. Some people think that neurons specifically aren't necessary for understanding intelli…

[deleted]

Re: Language models can explain neurons in language models

#318
post #207

I built a toy neural network that runs in the browser[1] to model 2D functions with the goal of doing something similar to this research (in a much more limited manner, ofc). Since the input space is so much more limited than language models or similar, it's possible to examine the outputs for each neuron for all possible inputs, and in a continuous manner. In some cases, you can clearly see neurons that specialize t…

this is wonderful, thanks for sharing!

Re: Language models can explain neurons in language models

#319
post #285

Earlier quoted context omitted.

GPT-4 is better at reasoning than 90% of humans. At least. I won't be surprised if GPT-5 is better than 100% of humans. I'm saying this in complete seriousness.

Google search is better than reasoning than most humans - in that if you search for an explanation of something then Google's first result is often correct, or one of the following ones. GPT-4 will often come up with a solution to a problem, but only if it has learnt something similar (it's better than Google in some respects: it can extract and combine abstractions). However, both need handholding by a human (supply…

OK, you have a point. But, if you replace "GPT-4" with "an average human" in your response - it still makes sense.

Re: Language models can explain neurons in language models

#320
post #313

Earlier quoted context omitted.

Use it to analyze the California & US Code, the California & Federal Codes of Regulation, and bills currently in the California legislation & Congress. It's far from useless but far more useful for creative writing than any kind of understanding or instruction following when it comes to complex topics. Even performing a map-reduce over large documents to summarize or analyze them for a specific audience is largely be…

Interesting - do you believe average humans (not professional lawyers) would do better on this task?

Probably not.

I'm not sure that pointing out that LLMs are as useful for parsing legal code as the average human is something to brag about though.

Post reply on HN