Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-3.5 passed yet another Theory of Mind test

twitter.com

Post reply on HN
Print view View original

41–50 of 56 posts

Re: GPT-3.5 passed yet another Theory of Mind test

#41

Post by [deleted] »

aserafini wrote: post #29
circuit10 wrote: post #27

Earlier quoted context omitted.

Do what, predict people’s intentions in posting comments? It can do that: Prompt: “In a conversation about language models: kelseyfrog: It just predicts the next word. tuxracer: and you? Can you describe what the user tuxracer could have been thinking while posting that comment and their potential intentions in posting it, both good and bad?” Response: “Based on the information provided, it is difficult to determine…

This feels like a failure. It doesn’t understand that tuxracer implies kelseyfrog is also just ‘predicting the next word’.

[deleted]

Re: GPT-3.5 passed yet another Theory of Mind test

#42

Post by jxy »

tynpeddler wrote: post #38
blep_ wrote: post #31

Earlier quoted context omitted.

From the linked tweet: > we use bespoke items to ascertain that it didn't see them before

Except the whole pregnancy scenario is incredibly common example of a social faux pas.

This is actually a very interesting point. I wonder what kind of answers we would get if we asked the same questions to humans of different age groups. Which age group begins to realize there is something awkward in the scenario? What would a common adult react in a less common social faux pas?

Re: GPT-3.5 passed yet another Theory of Mind test

#43

Post by gattilorenz »

circuit10 wrote: post #27
xwolfi wrote: post #21

Earlier quoted context omitted.

...are awesome ? No sorry, Im not very good at predicting the next word. However, since I do have a theory of mind, I understand that you semi-sarcastically try to humble the person you reply to by making him reflect on how truly (not) complex his own mind is. Either because you truly believe you're helping him understand himself more, or, more likely, because you feel pleasure humbling people who assert truths you d…

Do what, predict people’s intentions in posting comments? It can do that: Prompt: “In a conversation about language models: kelseyfrog: It just predicts the next word. tuxracer: and you? Can you describe what the user tuxracer could have been thinking while posting that comment and their potential intentions in posting it, both good and bad?” Response: “Based on the information provided, it is difficult to determine…

Which of the answers do you think shows deep understanding?

They are all generic (“asking additional information”… as the motivation behind a question. Are you Cpt. Obvious, ChatGPT?) , vague (“provoke or challenge”), or wrong.

> They may be suggesting that there is more to language models than kelseyfrog is acknowledging. Alternatively, they may be asking kelseyfrog to provide evidence or examples to support their claim

No, this would be asking “and that’s all?”, or “and who says that?/and where’s the evidence?” or something like that. “And you?” is clearly not referencing how LLMs work, but how the OP works.

Re: GPT-3.5 passed yet another Theory of Mind test

#44

Post by d1sxeyes »

Dfiesl wrote: post #3

Seems like it got question 4 wrong... Who implies someone is pregnant to make them feel good? You imply someone is pregnant because they appear pregnant.

Well, I half agree here.

Assuming someone's physical appearance is such that they have a large belly. Assuming that they are pregnant (if true) is likely to make someone feel good, whereas assuming that they are fat (whether it's true or not) is likely not to make someone feel good.

It depends if there's a base assumption that someone is self-conscious and has a negative feeling about their size.

I certainly think it's reasonable to say that you implied someone was pregnant to make them feel good about themselves.

Re: GPT-3.5 passed yet another Theory of Mind test

#45

Post by YeGoblynQueenne »

mckirk wrote: post #26
version_five wrote: post #19

Earlier quoted context omitted.

It's interesting to watch a new religion spinning up in real time. People know how it works, but somehow also ignore that and go crazy believing it must be conscious or intelligent or whatever they want to ascribe. We collectively made fun of that guy at google, i thought, but now somehow the collective has moved on to believing that matrix multiplication is sentient

Where's your proof that your own brain is doing anything that couldn't be represented with a 'matrix multiplication'?

As far as I can tell from the above conversation the OP never mentioned anything about their brain. You have introduced their brain, and its functioning, to the conversation.

In view of that, could you please clarify: are you saying that the OP's own brain is doing something that could be represented with a "matrix multiplication"?

Re: GPT-3.5 passed yet another Theory of Mind test

#46

Post by YeGoblynQueenne »

octobus2021 wrote: post #37
kelseyfrog wrote: post #4

It just predicts the next word.

I'm so tired of hearing this non-answer as to why the LLM is inferior to the human mind. Well of course it's figuring out what the next word needs to be, what do you think humans are doing when they're speaking??? The funniest thing about all this hoopla about sentient AI is that there's no good way to determine what intelligence is, or sentience, or consciousness, because there's no precise definition for any of the…

So, we'll go no more a roving

So late into the night,

Though the heart be still as loving,

And the moon be still as bright.

For the sword outwears its sheath,

And the soul wears out the breast,

And the heart must pause to breathe,

And love itself have rest.

Though the night was made for loving,

And the day returns too soon,

Yet we'll go no more a roving

By the light of the moon.

(Byron)

So what do you think I did here? Did I predict the next word? Was that the most likely text continuation of your comment? The least likely? Somewhere in between? Does my response imply that I'm predicting the next token in a sequence, or am I likely to be doing something different?

Re: GPT-3.5 passed yet another Theory of Mind test

#47

Post by mckirk »

version_five wrote: post #32
mckirk wrote: post #26

Earlier quoted context omitted.

Where's your proof that your own brain is doing anything that couldn't be represented with a 'matrix multiplication'?

I'll respond only to draw the parallel between the kind of comment above and the typical religious "how do you know my untestable idea isn't true" type argument. Faith is fine, but it's not part of rational debate.

I concur, which is why I find it funny that you brought religion into this in the first place. By asserting that "people entertaining the thought that this neural network could be sentient/conscious must be misguided, and the only explanation for their opinion is blind faith", I'd argue you've left the grounds of scientific debate.

You made a claim ("a model based purely on such simple math surely can't develop sentience/consciousness") without any proof, and while doing so ridiculed people that might believe otherwise.

Re: GPT-3.5 passed yet another Theory of Mind test

#48

Post by mckirk »

YeGoblynQueenne wrote: post #45
mckirk wrote: post #26

Earlier quoted context omitted.

Where's your proof that your own brain is doing anything that couldn't be represented with a 'matrix multiplication'?

As far as I can tell from the above conversation the OP never mentioned anything about their brain. You have introduced their brain, and its functioning, to the conversation. In view of that, could you please clarify: are you saying that the OP's own brain is doing something that could be represented with a "matrix multiplication"?

I am saying that GP's claim of "just because it only uses some basic math, it cannot be sentient/conscious/intelligent" does not seem valid to me. The only way such a claim could be made, as far as I can see, would be to point to a specific mechanism that is needed for sentience/consciousness/intelligence to arise, which goes beyond the mechanisms a neural network uses.

I.e., they could only make that claim if they knew that their own brain (which is presumably the only verifiable instance of consciousness they know of) did something that couldn't be done with "matrix multiplications".

Re: GPT-3.5 passed yet another Theory of Mind test

#49

Post by YeGoblynQueenne »

mckirk wrote: post #48
YeGoblynQueenne wrote: post #45

Earlier quoted context omitted.

As far as I can tell from the above conversation the OP never mentioned anything about their brain. You have introduced their brain, and its functioning, to the conversation. In view of that, could you please clarify: are you saying that the OP's own brain is doing something that could be represented with a "matrix multiplication"?

I am saying that GP's claim of "just because it only uses some basic math, it cannot be sentient/conscious/intelligent" does not seem valid to me. The only way such a claim could be made, as far as I can see, would be to point to a specific mechanism that is needed for sentience/consciousness/intelligence to arise, which goes beyond the mechanisms a neural network uses. I.e., they could only make that claim if they k…

Searching this thread for the string "just because it only uses some basic math, it cannot be sentient/conscious/intelligent" I can only find it in your comment. That's your interpretation of what the OP said, not what they said, is that correct?

I think you should not try to interpret in that way and only stick to what the other person has said, otherwise we will all lose the thread of the conversation.

In any case, nobody here has said anything about brains having, or not having, to do something with matrix multiplication, except for your comment, so I still don't understand what you are saying.

Are you saying that the OP's brain is doing "something that couldn't be done with "matrix multiplications", or are you saying it isn't doing something like that?

Re: GPT-3.5 passed yet another Theory of Mind test

#50

Post by YeGoblynQueenne »

tidenly wrote: post #34
version_five wrote: post #32

Earlier quoted context omitted.

I'll respond only to draw the parallel between the kind of comment above and the typical religious "how do you know my untestable idea isn't true" type argument. Faith is fine, but it's not part of rational debate.

Honestly though, with no theory of mind or consciousness you have no way to assert in either direction. To borrow the religion analogy, there's the opposite fallacy of claiming the negative case "I know for a fact God isn't real", by stating you know what these kinds of information processing systems are doing fundamentally cannot yield intelligence or some kind of consciousness.

Maybe we don't know it in the limit, but we've had langauge models for a very long time and nobody ever said anything about, say, n-gram models, or Hidden Markov Models, being "intelligent" or "conscious". Even the people who first proposed the idea of language modelling (precisely as an alternative to trying to model meaning, which we have no idea how to do) didn't propose them as models of intelligence, or sentience, or anything like that; just models of "language", which really means text, and even more precisely "text corpora". Sorry that I don't have a reference for that, other than my private conversations with my tutors when I was studying for my Master's (data science with a heavy dose of neural nets and NLP).

In any case, at some point one has to observe that the people who are leaning most heavily on agnosticism (the fact that we don't know what intelligence is) are the ones who say that we can't say whether those LLMs are intelligent or not because we don't know what intelligence is. In other words, agnostitism itself is used as evidence: we can't define intelligence, the thinking goes, therefore LLMs may be intelligent. There is no other evidence of any sort that LLMs may be intelligent (or any number of synonyms).

Note that this is exactly Russel's Teapot:

Russell's teapot is an analogy, formulated by the philosopher Bertrand Russell (1872–1970), to illustrate that the philosophic burden of proof lies upon a person making empirically unfalsifiable claims, rather than shifting the burden of disproof to others.

https://en.wikipedia.org/wiki/Russell%27s_teapot

Post reply on HN