Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Language models can explain neurons in language models

openai.com

Post reply on HN
Print view View original

341–350 of 497 posts

Re: Language models can explain neurons in language models

#341

Post by AdieuToLogic »

esafak wrote: post #284
akiselev wrote: post #244

Earlier quoted context omitted.

If you spent even more time with GPT-4 it would be evident that it is definitely not. Especially if you try to use it as some kind of autonomous agent.

I think we'll soon be able to train models that answer any reasonable question. By that measure, computers are intelligent, and getting smarter by the day. But I don't think that is the bar we care about. In the context of intelligence, I believe we care about self-directed thought, or agency. And a computer program needs to keep running to achieve that because it needs to interact with the world.

> I believe we care about self-directed thought, or agency. And a computer program needs to keep running to achieve that because it needs to interact with the world.

By that definition, every computer virus and worm qualifies as having "self-directed thought" and "agency." Their very existence "to keep running" and propagate satisfies the need "to interact with the world."

Re: Language models can explain neurons in language models

#342

Post by p1esk »

chimpanzee wrote: post #325
p1esk wrote: post #316

Earlier quoted context omitted.

Why don't you suggest an example we can run and see what it's capable of (compared to what I, or other humans, are capable of)?

(In case it was missed, I’ve added a relevant addendum to my previous comment.) Not sure an example is needed because I agree it “explains” better than pretty much everyone. (From my mostly lay perspective) It essentially uses the prompt as an argument in a probabilistic analysis of its incredibly vast store of prior inputs to transform them into an output that at least superficially satisfies the prompter’s goals. T…

What would be an example of “non-deductive” reasoning, which requires embodied perceptual experiences?

Re: Language models can explain neurons in language models

#343

Post by red75prime »

croes wrote: post #338
p1esk wrote: post #219

Earlier quoted context omitted.

If you spent any time with GPT-4 it should be evident.

Looks more like a Chinese Room to me.

Everything is a Chinese room if you expect to see reified comprehension inside (and, naturally, don't find it).

Re: Language models can explain neurons in language models

#344

Post by philomath_mn »

DaiPlusPlus wrote: post #324
0xParlay wrote: post #145

Earlier quoted context omitted.

I love the epistemology related discussions AI inevitably surfaces. How can we know anything that isn't empirically evident and all that. It seems NN output could be trusted in scenarios where a test exists. For example: "ChatGPT design a house using [APP] and make sure the compiled plans comply with structural/electrical/design/etc codes for area [X]". But how is any information that isn't testable trusted? I'm open…

> But how is any information that isn't testable trusted? I'm open to the idea ChatGPT is as credible as experts in the dismal sciences given that information cannot be proven or falsified and legitimacy is assigned by stringing together words that "makes sense". I understand that around the 1980s-ish, the dream was that people could express knowledge in something like Prolog, including the test-case, which can then…

I bet GPT is really good at prolog, that would be interesting to explore.

"Answer this question in the form of a testable prolog program"

Re: Language models can explain neurons in language models

#345

Post by janalsncm »

akiselev wrote: post #244
p1esk wrote: post #219

Earlier quoted context omitted.

If you spent any time with GPT-4 it should be evident.

If you spent even more time with GPT-4 it would be evident that it is definitely not. Especially if you try to use it as some kind of autonomous agent.

I don’t think intelligence is a binary property. GPT3 is definitely “intelligent” in some areas even if it is deeply flawed in others.

Re: Language models can explain neurons in language models

#346

Post by karpierz »

p1esk wrote: post #339
karpierz wrote: post #328

Earlier quoted context omitted.

Where did you get that the LLM was as _intelligent_ as a human? All we've shown is that LLMs are as useful for parsing legal text as the average human. Which is to say, not. A dog is also as useful for parsing legal texts as the average human. So is a rock.

Where did you get that the LLM was as _intelligent_ as a human? First hand experience -I’ve been using it daily for the past two months.

Ah, if that's what you mean then there are plenty of intelligent systems out there.

I've used Google search for decades and it's been able to answer questions better than humans ever could. Same for Google Maps, though arguably they're the same system at this point. My calculator is far more intelligent than any human I've met, at least when it comes to adding large numbers. My compiler can detect even the slightest syntax error with impeccable accuracy. Microsoft word has an incredible vocabulary. Wikipedia knows more historical events than any human dead or alive. And so on.

Shit, users thought Eliza was intelligent in the 60s.

If what you really mean is that LLMs are cool and useful, then sure. Just say that instead of couching it in some vague claim of intelligence.

Re: Language models can explain neurons in language models

#347

Post by imtringued »

AdieuToLogic wrote: post #341
esafak wrote: post #284

Earlier quoted context omitted.

I think we'll soon be able to train models that answer any reasonable question. By that measure, computers are intelligent, and getting smarter by the day. But I don't think that is the bar we care about. In the context of intelligence, I believe we care about self-directed thought, or agency. And a computer program needs to keep running to achieve that because it needs to interact with the world.

> I believe we care about self-directed thought, or agency. And a computer program needs to keep running to achieve that because it needs to interact with the world. By that definition, every computer virus and worm qualifies as having "self-directed thought" and "agency." Their very existence "to keep running" and propagate satisfies the need "to interact with the world."

Yes, computer viruses have more agency than ChatGPT.

Re: Language models can explain neurons in language models

#348

Post by p1esk »

karpierz wrote: post #346
p1esk wrote: post #339

Earlier quoted context omitted.

Where did you get that the LLM was as _intelligent_ as a human? First hand experience -I’ve been using it daily for the past two months.

Ah, if that's what you mean then there are plenty of intelligent systems out there. I've used Google search for decades and it's been able to answer questions better than humans ever could. Same for Google Maps, though arguably they're the same system at this point. My calculator is far more intelligent than any human I've met, at least when it comes to adding large numbers. My compiler can detect even the slightest…

No, what I meant was GPT-4 is more intelligent than most humans I interact with on a daily basis. In the fullest meaning of that word.

Re: Language models can explain neurons in language models

#349

Post by mxschll »

> We are open-sourcing our datasets and visualization tools for GPT-4-written explanations of all 307,200 neurons in GPT-2, as well as code for explanation and scoring using publicly available models on the OpenAI API. We hope the research community will develop new techniques for generating higher-scoring explanations and better tools for exploring GPT-2 using explanations.

Aww, that's so nice of them to let the community do the work they can use for free. I might even forget that most of OpenAI is closed source.

Re: Language models can explain neurons in language models

#350

Post by p1esk »

akiselev wrote: post #336
streakfix wrote: post #330

Earlier quoted context omitted.

Did you try fine tuning gpt4 with that book as input?

OpenAI doesn't support fine tuning of GPT4 and with context stuffing,the more of the book I include in the input the less of the bills I can include - which, again, are millions of tokens - and the less space there is for memory.

I believe you. But at the same time they showed during the demo how it can do taxes, using a multi page document. An ability to process longer documents seems more like an engineering challenge rather than a fundamental limitation.
Post reply on HN