Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Theory of Mind May Have Spontaneously Emerged in Large Language Models

arxiv.org

Post reply on HN
Print view View original

111–120 of 321 posts

Re: Theory of Mind May Have Spontaneously Emerged in Large Language Models

#111

Post by pdonis »

mietek wrote: post #99
pdonis wrote: post #91

Earlier quoted context omitted.

See my response to strbean upthread.

Okay. It seems to me that, in analogy to humans having such a relationship with physical reality, machine learning systems have such a relationship with virtual reality — the environment in which they are trained and deployed.

> machine learning systems have such a relationship with virtual reality

No, they don't, because they can't take actions in that virtual reality and sense the consequences. They can't test hypotheses about how the reality works. They can't even frame hypotheses about how the reality works.

Re: Theory of Mind May Have Spontaneously Emerged in Large Language Models

#112

Post by s0ulphire »

pdonis wrote: post #60
canniballectern wrote: post #53

Earlier quoted context omitted.

Of course, today's LLMs only appear to have theory of mind at first glance and fall apart under closer scrutiny. But if they can continue to become more and more accurate replicas of the real thing, I don't think it matters at all. There's no way to know for sure that anyone other than yourself experiences consciousness. All you can do is judge for yourself that what they're describing matches closely enough with you…

> There's no way to know for sure that anyone other than yourself experiences consciousness. All you can do is judge for yourself that what they're describing matches closely enough with your own experiences that they're probably experiencing the same thing you are. That judgment is not just based on the words other people use. It is based on knowing that other people's brains and minds have the same sort of semantic…

I'll dig up a source in a bit, but there is a critical period of development in which a child must be exposed to language, or they will fail to develop the very core skills that you're suggesting are innate abilities in a person regardless of their upbringing. This is exactly how you learned everything you know; your parents talked to you. Language grants you the ability to define concepts in the first place, without which you have no ability to recognise them as you have no language with which to think about them in the first place. So what specifically differentiates the way your brain learned to classify objects and words from the way a NN does? And what stops a NN from being able to develop concepts based on the relationship of those new definitions in the same way you do? IMO arguably it's just a matter of processing power and configuration of the network.

Re: Theory of Mind May Have Spontaneously Emerged in Large Language Models

#113

Post by bigbluedots »

robertlagrant wrote: post #59
canniballectern wrote: post #53

Earlier quoted context omitted.

Of course, today's LLMs only appear to have theory of mind at first glance and fall apart under closer scrutiny. But if they can continue to become more and more accurate replicas of the real thing, I don't think it matters at all. There's no way to know for sure that anyone other than yourself experiences consciousness. All you can do is judge for yourself that what they're describing matches closely enough with you…

> But if they can continue to become more and more accurate replicas of the real thing, I don't think it matters at all. So, I suppose I'd ask: what does "matter" mean here? If you knew that everyone you loved had been destroyed and been replaced by exact replicas, would that matter?

That already happens frequently enough at a cellular level.

Re: Theory of Mind May Have Spontaneously Emerged in Large Language Models

#114

Post by strbean »

pdonis wrote: post #90
strbean wrote: post #78

Earlier quoted context omitted.

> Human minds have semantic relationships to the rest of the world that LLMs do not have. This is a bold claim that seems built on a presumption of mind-body dualism. Brains don't have semantic relationships with anything. They are neurons hooked up to sensors and actuators. Any inferences they produce are the result of statistical processes.

> This is a bold claim that seems built on a presumption of mind-body dualism. Not at all. I'm a physicalist; I don't believe the mind is a separate thing from the brain. > Brains don't have semantic relationships with anything. Yes, they do: you describe them yourself: > They are neurons hooked up to sensors and actuators. Those are semantic relationships with the rest of the world. Although your short description d…

I see what you're saying; ChatGPT doesn't have a physical relationship with the world, doesn't have agency (is essentially paused until given input), doesn't have reward/punishment stimulae, etc.

I do think that a large portion of what seems to be missing here is trivial to add, relative to the effort in creating ChatGPT in the first place.

Side note: I'm not sure 'semantic relationship' is the right term here. Pretty sure it is specific to relationships between linguistic constructs. That wording very much triggered my "Bah, dualism!" response, as I thought you were insinuating some metaphysical bond between the mind and the world. Maybe "meaningful relationship" would serve better?

Re: Theory of Mind May Have Spontaneously Emerged in Large Language Models

#115

Post by bawolff »

jfengel wrote: post #41
bawolff wrote: post #13

Earlier quoted context omitted.

> I'm reminded of the early days of "AI" when that consisted of building chess engines, because chess is what "intelligent" people do. They quickly realized that they were solving the wrong problem, doing the thing that humans are bad at and computers are good at. Is this really true? Because a lot of effort was spent on making computers as good at chess as human experts. It was considered a pretty big breakthrough w…

By the time they were trying to defeat the world champion, they didn't really think of it as an AI project any more. They often used the term "expert system", which had a much narrower scope. It had become a challenge unto itself, but they didn't expect it to lead to any kind of generalized intelligence. It came rather out of nowhere when neural-net-type engines suddenly swept back into dominance. Even after becoming…

> but they didn't expect it to lead to any kind of generalized intelligence.

Do experts really expect any stream of AI research to lead to generalized intelligence (except in the very long term)? I was under the impression we really have no idea how to get there.

Re: Theory of Mind May Have Spontaneously Emerged in Large Language Models

#116

Post by bryan0 »

avgcorrection wrote: post #76
space_fountain wrote: post #56

Earlier quoted context omitted.

I wonder every time I see this take what it would mean under this definition of knowing things for a machine learning algorithm to ever know something. I find that especially important because to every appearance we are a machine learning algorithm. I don’t know how different the sort of knowing this algorithm has to the sort of knowing a human has, but you’re far more confident than I am that it’s a difference of ki…

The problem with this facile view of things is that it seems to be a dead end for scientific theories. What if we just limited the science of birds to explaining how limb-flapping could produce levitation? Hmm yes. Birds are kind of like helicopters, it seems. Who’s to say that they are not basically one and the same? Moving on. If you are only interested in the most superficial tests and theories—like the Turing Tes…

> The problem with this facile view of things is that it seems to be a dead end for scientific theories.

You're overreaching quite a bit here, or I think you're misinterpreting what Parent said. I interpreted what they said as: it seems the difference in how we "know" something vs how an LLM "knows" something might actually be closer than some suspect. this certainly is not an "end of science".

Re: Theory of Mind May Have Spontaneously Emerged in Large Language Models

#117

Post by abeppu »

space_fountain wrote: post #56
lsy wrote: post #33

This highlights one of the types of muddled thinking around LLMs. These tasks are used to test theory of mind because for people, language is a reliable representation of what type of thoughts are going on in the person's mind. In the case of an LLM the language generated doesn't have the same relationship to reality as it does for a person. What is being demonstrated in the article is that given billions of tokens o…

I wonder every time I see this take what it would mean under this definition of knowing things for a machine learning algorithm to ever know something. I find that especially important because to every appearance we are a machine learning algorithm. I don’t know how different the sort of knowing this algorithm has to the sort of knowing a human has, but you’re far more confident than I am that it’s a difference of ki…

Defining what "knowing" is would be useful, yes, and analytic philosophers in epistemology do argue about this. One attribute that's classically part of the definition of "knowing" is that the thing which is known must be true. LLMs are pretty bad at this, but perhaps that can be fixed.

But I would challenge you to imagine the situation the LLM is actually in. Do you understand Thai? If so, in the following, feel free to imagine some other language which you don't know and is not closely related to any languages you do know. Suppose I gather reams and reams of Thai text, without images, without context. Books without their covers, or anything which would indicate genre. There's no Thai-English dictionary available, or any Thai speakers. You aren't taught which symbols map to which sounds. You're on your own with a giant pile of text, and asked to learn to predict symbols. If you had sufficient opportunity to study this pile of text, you'd begin to pick out patterns of which words appear together, and what order words often appear in. Suppose you study this giant stack of Thai text for years in isolation. After all this study, you're good enough that given a few written Thai words, you can write sequences of words that are likely to follow, given what you know of these patterns. You can fill in blanks. But should anyone guess that you "know" what you're saying? Nothing has ever indicated to you what any of these words _mean_. If you give back a sequence of words, which a Thai speakers understands to be expressing an opinion about monetary policy, because you read several similar sequences in the pile, is that even your opinion?

I think algorithms can 'know' something, given sufficient grounding. LLMs 'know' what text looks like. They can 'know' what tokens belong where, even if they don't know anything about the things referred to. That's all, because that's what they have to learn from. I think an game-playing RL-trained agent can 'know' the likely state-change that a given action will cause. An image segmentation model can 'know' which value-differences in adjacent pixels are segment boundaries.

But if we want AIs that 'know' the same things we know, then we have to build them to perceive in a multi-modal way, and interact with stuff in the world, rather than just self-supervising on piles of internet data.

Re: Theory of Mind May Have Spontaneously Emerged in Large Language Models

#118

Post by shawnz »

avgcorrection wrote: post #76
space_fountain wrote: post #56

Earlier quoted context omitted.

I wonder every time I see this take what it would mean under this definition of knowing things for a machine learning algorithm to ever know something. I find that especially important because to every appearance we are a machine learning algorithm. I don’t know how different the sort of knowing this algorithm has to the sort of knowing a human has, but you’re far more confident than I am that it’s a difference of ki…

The problem with this facile view of things is that it seems to be a dead end for scientific theories. What if we just limited the science of birds to explaining how limb-flapping could produce levitation? Hmm yes. Birds are kind of like helicopters, it seems. Who’s to say that they are not basically one and the same? Moving on. If you are only interested in the most superficial tests and theories—like the Turing Tes…

> If you are only interested in the most superficial tests and theories—like the Turing Test—then consider psychology conquered once you’ve tricked a human with your chat bot.

What's the counterargument? What's a less superficial test that we can use instead, which conclusively shows that actually human minds aren't just like very sophisticated LLMs? There isn't one -- this is nothing but the same Chinese room problem which we've been discussing for decades. The topmost poster is simply assuming that language models can't possibly understand the same way a human does without relying on any kind of "test" at all, which I think is the real scientific dead end here.

Re: Theory of Mind May Have Spontaneously Emerged in Large Language Models

#119

Post by swatcoder »

didntreadarticl wrote: post #103
swatcoder wrote: post #84

Earlier quoted context omitted.

The future AI you describe has language at the center of its understanding. For example, you would expect that the cameras and arm-feedback-sensors would produce text (or text-associated weights/tokens) that describe what the AI "sees" and the robotic arms would be receive some kind of language-derived directives from the LLM. It will be very interesting to see what that system is capable of. I think a lot of people…

So your proposed system would be a extremely interesting exploration of that! I look forward to it. Right, so lets virtualise it. Actually training AIs using real cameras and real robot arms will be really slow and expensive. So we provide a system that renders a photorealistic graphical room with teapot and robot arm, and a virtual camera inside the room is 'seeing' parts of the room and a vision model then processe…

Let me know when you find out.

The interesting and open question to me is what the limitations are of a language model at the center of that experience. How much of a a relationship with reality can be captured by language at all, and specifically with the specific sort of statistical models of language that we're exploring now? For some of us, the intuitive answer is not all that much and for others it seems to be at least as much as any human.

Whether conducted virtually or physically, coming up with an answer sounds like an empirical study, and one that we're some years away from having results for.

Re: Theory of Mind May Have Spontaneously Emerged in Large Language Models

#120

Post by crancher »

abeppu wrote: post #117
space_fountain wrote: post #56

Earlier quoted context omitted.

I wonder every time I see this take what it would mean under this definition of knowing things for a machine learning algorithm to ever know something. I find that especially important because to every appearance we are a machine learning algorithm. I don’t know how different the sort of knowing this algorithm has to the sort of knowing a human has, but you’re far more confident than I am that it’s a difference of ki…

Defining what "knowing" is would be useful, yes, and analytic philosophers in epistemology do argue about this. One attribute that's classically part of the definition of "knowing" is that the thing which is known must be true. LLMs are pretty bad at this, but perhaps that can be fixed. But I would challenge you to imagine the situation the LLM is actually in. Do you understand Thai? If so, in the following, feel fre…

Excellent and useful analogy. Thank you.
Post reply on HN