Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Language models can explain neurons in language models

openai.com

Post reply on HN
Print view View original

481–490 of 497 posts

Re: Language models can explain neurons in language models

#481

Post by xcv123 »

YeGoblynQueenne wrote: post #479
xcv123 wrote: post #476

Earlier quoted context omitted.

Prompt: What is the sieve of Eratosthenes? Speak in chicken Answer from GPT-4: Sure, here's the explanation of the sieve of Eratosthenes explained in a simplified "chicken" style: Imagine you are a chicken. You have lots of chicken friends. Let's say there are 100 of you. Now, you all stand in a line. Each chicken is numbered, from 2 up to 100. Chicken number 1 decided to stay at home because he's a bit special in th…

Bad prompt. Try "Only use the word chicken" instead of "Speak in chicken".

Prompt: What is the sieve of Eratosthenes? Only use the word "chicken".

Answer from GPT-4:

The chicken is in a chicken coop with many other chickens. Each chicken represents a number in the chicken coop. The first chicken that hasn't been touched, after the chicken representing one, is a prime chicken. We'll call this chicken the prime chicken.

Now, imagine that the prime chicken has a unique chicken dance. Every chicken that copies this prime chicken's dance is removed from the coop because they are not prime chickens. These are the chickens that would be multiples of our prime chicken.

Then we find the next chicken that hasn't been touched and doesn't know the dance of the previous prime chicken. This chicken is also a prime chicken. This prime chicken has a different chicken dance, and all the chickens that copy this new dance are also removed from the coop.

We keep doing this - finding the next chicken that hasn't danced yet, designating it as a new prime chicken with its unique dance, and removing any chickens that copy the dance - until there are no more new dances.

All the chickens left in the coop, who haven't copied any dances, are the prime chickens. This is the chicken way to understand the Sieve of Eratosthenes. The prime chickens are like the prime numbers in the Sieve of Eratosthenes.

----

I then followed up with: "You must only use the word "chicken". Do not use any other words."

Then it wrote an entire page filled with "chicken chicken chicken"

I then asked: "By what logic did you construct your response? Why not use just one occurrence of the word "chicken" and finish there?"

Response from GPT-4:

Apologies for any confusion. Your question seemed to request a description of the Sieve of Eratosthenes using only the word "chicken". While it's true that a single "chicken" would technically meet your request, it seemed that you were interested in a more elaborate or extended response, hence the repetition.

In any case, attempting to convey any meaningful information using only a single word—let alone a complex concept like the Sieve of Eratosthenes—is inherently impossible due to the lack of distinct symbols or constructs to represent different aspects of the concept. Thus, the extended repetition of "chicken" was intended more as a playful response rather than a serious attempt at conveying the concept you requested.

Re: Language models can explain neurons in language models

#482

Post by [deleted] »

YeGoblynQueenne wrote: post #480
xcv123 wrote: post #477

Earlier quoted context omitted.

> There's no thinking, no reasoning, no calculation, no logic, no deduction, no intelligence, no anything. It's only token, token, token. False. The neural network inside the transformer LLM contains a hierarchical semantic model, and has inferred some rules of reasoning from the training set. It can apply those rules to new input. There are semantic layers above the "token token tokens". Explore them here: https://o…

What you're commenting here is simply repearting, without critical intent, the baseless claims connectionists have made about their systems for many decades. Similarlly those claims have been criticised but connectionsists simply ignore the criticisms and continue with the same old nonsense, as if nothing happened. For example, that ridiculous conceit that their systems have "neurons", or that the weights of function…

[deleted]

Re: Language models can explain neurons in language models

#483

Post by [deleted] »

YeGoblynQueenne wrote: post #480
xcv123 wrote: post #477

Earlier quoted context omitted.

> There's no thinking, no reasoning, no calculation, no logic, no deduction, no intelligence, no anything. It's only token, token, token. False. The neural network inside the transformer LLM contains a hierarchical semantic model, and has inferred some rules of reasoning from the training set. It can apply those rules to new input. There are semantic layers above the "token token tokens". Explore them here: https://o…

What you're commenting here is simply repearting, without critical intent, the baseless claims connectionists have made about their systems for many decades. Similarlly those claims have been criticised but connectionsists simply ignore the criticisms and continue with the same old nonsense, as if nothing happened. For example, that ridiculous conceit that their systems have "neurons", or that the weights of function…

[deleted]

Re: Language models can explain neurons in language models

#484

Post by xcv123 »

YeGoblynQueenne wrote: post #480
xcv123 wrote: post #477

Earlier quoted context omitted.

> There's no thinking, no reasoning, no calculation, no logic, no deduction, no intelligence, no anything. It's only token, token, token. False. The neural network inside the transformer LLM contains a hierarchical semantic model, and has inferred some rules of reasoning from the training set. It can apply those rules to new input. There are semantic layers above the "token token tokens". Explore them here: https://o…

What you're commenting here is simply repearting, without critical intent, the baseless claims connectionists have made about their systems for many decades. Similarlly those claims have been criticised but connectionsists simply ignore the criticisms and continue with the same old nonsense, as if nothing happened. For example, that ridiculous conceit that their systems have "neurons", or that the weights of function…

We are talking about artificial neurons here. Not biological neurons. These are mathematical structures.

https://en.wikipedia.org/wiki/Artificial_neuron

These models infer semantic categories that correlate to categories within the human mind, to the extent that they can solve natural language understanding tasks.

No one is saying they are biological neurons, or that they model semantics exactly as the human mind would. It is mechanical pattern recognition that approximates our understanding.

You can browse those artificial neurons online and view their associations.

Re: Language models can explain neurons in language models

#485

Post by jmfldn »

marshray wrote: post #453
jmfldn wrote: post #157

Earlier quoted context omitted.

That's a strawman since I didn't argue anything about humans being special. I don't think there is anything necessarily inherently special about human intelligence, I'm just advocating for caution around the language we use to talk about current systems. All this talk of AGI and sentience and so on is premature and totally unfounded . It's pure sci fi, for now at least.

> I didn't argue anything about humans being special Above you said about AI LMs: > There is no "their" and there is no "thought process" So, unless you're claiming that humans lack a thought process as well, then you're arguing that humans are special. > All this talk of AGI and sentience and so on is premature and totally unfounded I don't see any mention of AGI or sentience in this thread? Also, I don't think anyo…

I reject the words 'self' and 'cognition' in your comment. This is exactly what I'm talking about. A facsimile of them, maybe...

Re: Language models can explain neurons in language models

#486

Post by YeGoblynQueenne »

xcv123 wrote: post #484
YeGoblynQueenne wrote: post #480

Earlier quoted context omitted.

What you're commenting here is simply repearting, without critical intent, the baseless claims connectionists have made about their systems for many decades. Similarlly those claims have been criticised but connectionsists simply ignore the criticisms and continue with the same old nonsense, as if nothing happened. For example, that ridiculous conceit that their systems have "neurons", or that the weights of function…

We are talking about artificial neurons here. Not biological neurons. These are mathematical structures. https://en.wikipedia.org/wiki/Artificial_neuron These models infer semantic categories that correlate to categories within the human mind, to the extent that they can solve natural language understanding tasks. No one is saying they are biological neurons, or that they model semantics exactly as the human mind wou…

You're just saying words without ever explaining why. What am I supposed to do about that? There's nothing to argue with if you're just repeating nonsensical claims without even trying to support them.

For example:

>> It is mechanical pattern recognition that approximates our understanding.

That's just a claim and you're not even saying why you make it, what makes you think so, etc.

Re: Language models can explain neurons in language models

#487

Post by xcv123 »

YeGoblynQueenne wrote: post #486
xcv123 wrote: post #484

Earlier quoted context omitted.

We are talking about artificial neurons here. Not biological neurons. These are mathematical structures. https://en.wikipedia.org/wiki/Artificial_neuron These models infer semantic categories that correlate to categories within the human mind, to the extent that they can solve natural language understanding tasks. No one is saying they are biological neurons, or that they model semantics exactly as the human mind wou…

You're just saying words without ever explaining why. What am I supposed to do about that? There's nothing to argue with if you're just repeating nonsensical claims without even trying to support them. For example: >> It is mechanical pattern recognition that approximates our understanding. That's just a claim and you're not even saying why you make it, what makes you think so, etc.

> That's just a claim and you're not even saying why you make it, what makes you think so, etc.

Mechanical - it is an algorithm, not a living being.

Pattern recognition - a branch of machine learning that focuses on the detection and identification of regularities and patterns in data. It involves classifying or categorizing input data into identifiable classes based on extracted features. The patterns recognized could be in various forms, such as visual patterns, speech patterns, or patterns in text data.

Approximates our understanding - meaning the model is not exactly the same as human understanding

When I say 'mechanical pattern recognition that approximates our understanding,' what I mean is that large language models (LLMs) like GPT-4 learn patterns from the vast amounts of text data they're trained on. These patterns correspond to various aspects of language and meaning.

For example, the models learn that the word 'cat' often appears in contexts related to animals, pets, and felines, and they learn that it's often associated with words like 'meow' or 'fur'. In this sense, the model 'understands' the concept of a cat to the extent that it can accurately predict and generate text about cats based on the patterns it has learned.

This isn't the same as human understanding, of course. Humans understand cats as living creatures with certain behaviors and physical characteristics, and we have personal experiences and emotions associated with cats. A language model doesn't have any of this - its 'understanding' is purely statistical and based on text patterns.

The evidence for these claims comes from the performance of these models on various tasks. They can generate coherent, contextually appropriate text, and they can answer questions, translate languages, and perform other language-related tasks with a high degree of accuracy. All of this suggests that they have learned meaningful patterns from their training data.

Re: Language models can explain neurons in language models

#488

Post by xcv123 »

YeGoblynQueenne wrote: post #486
xcv123 wrote: post #484

Earlier quoted context omitted.

We are talking about artificial neurons here. Not biological neurons. These are mathematical structures. https://en.wikipedia.org/wiki/Artificial_neuron These models infer semantic categories that correlate to categories within the human mind, to the extent that they can solve natural language understanding tasks. No one is saying they are biological neurons, or that they model semantics exactly as the human mind wou…

You're just saying words without ever explaining why. What am I supposed to do about that? There's nothing to argue with if you're just repeating nonsensical claims without even trying to support them. For example: >> It is mechanical pattern recognition that approximates our understanding. That's just a claim and you're not even saying why you make it, what makes you think so, etc.

Your disagreement seems to be a philosophical one. It is not a technical argument. It seems that you won't accept that semantics can be modelled by an unconscious mechanical system. I am talking about mathematical concepts of semantics, not "true" human semantics that are the product of human insight and consciousness. https://en.wikipedia.org/wiki/Semantic_similarity

While AI doesn't have an innate understanding of the world as humans do, the semantic representations it learns from vast amounts of text data can be surprisingly rich and detailed. It can capture associations and nuances that are not immediately apparent from a purely syntactic analysis of the text.

Re: Language models can explain neurons in language models

#489

Post by YeGoblynQueenne »

xcv123 wrote: post #487
YeGoblynQueenne wrote: post #486

Earlier quoted context omitted.

You're just saying words without ever explaining why. What am I supposed to do about that? There's nothing to argue with if you're just repeating nonsensical claims without even trying to support them. For example: >> It is mechanical pattern recognition that approximates our understanding. That's just a claim and you're not even saying why you make it, what makes you think so, etc.

> That's just a claim and you're not even saying why you make it, what makes you think so, etc. Mechanical - it is an algorithm, not a living being. Pattern recognition - a branch of machine learning that focuses on the detection and identification of regularities and patterns in data. It involves classifying or categorizing input data into identifiable classes based on extracted features. The patterns recognized cou…

That is not "evidence" of anything. It's just assumptions. You keep saying what you think is going on without ever saying how or why. You are not describing any mechanisms and you are not explaining any observations.

I have a suggestion: try to convince yourself that you are wrong; not right. Science gives you the tools to know when you're wrong. If you're certain you're right about something then you're probably wrong and you should keep searching until you find where and how.

For example, try to trace in your mind the mechanisms and functionality of language models, and see where your assumptions about their abilities come from.

Good luck.

Re: Language models can explain neurons in language models

#490

Post by YeGoblynQueenne »

xcv123 wrote: post #488
YeGoblynQueenne wrote: post #486

Earlier quoted context omitted.

You're just saying words without ever explaining why. What am I supposed to do about that? There's nothing to argue with if you're just repeating nonsensical claims without even trying to support them. For example: >> It is mechanical pattern recognition that approximates our understanding. That's just a claim and you're not even saying why you make it, what makes you think so, etc.

Your disagreement seems to be a philosophical one. It is not a technical argument. It seems that you won't accept that semantics can be modelled by an unconscious mechanical system. I am talking about mathematical concepts of semantics, not "true" human semantics that are the product of human insight and consciousness. https://en.wikipedia.org/wiki/Semantic_similarity While AI doesn't have an innate understanding of…

Oh come on. "Semantic similarity" is just heuristic bullshit. It's not a scientific term, or even a mathematical concept. Don't try to pull rank on me without even knowing who I am or what I do just because you can read wikipedia.

And note you're still not saying "why" or "how", only repeating the "what" of someone else's claim.

Post reply on HN