Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Theory of Mind May Have Spontaneously Emerged in Large Language Models

arxiv.org

Post reply on HN
Print view View original

191–200 of 321 posts

Re: Theory of Mind May Have Spontaneously Emerged in Large Language Models

#191

Post by helf »

pdonis wrote: post #111
mietek wrote: post #99

Earlier quoted context omitted.

Okay. It seems to me that, in analogy to humans having such a relationship with physical reality, machine learning systems have such a relationship with virtual reality — the environment in which they are trained and deployed.

> machine learning systems have such a relationship with virtual reality No, they don't, because they can't take actions in that virtual reality and sense the consequences. They can't test hypotheses about how the reality works. They can't even frame hypotheses about how the reality works.

Wait, you mean virtual reality isn’t like the cartoon Reboot!?

Re: Theory of Mind May Have Spontaneously Emerged in Large Language Models

#192

Post by hackinthebochs »

pdonis wrote: post #162
hackinthebochs wrote: post #154

Earlier quoted context omitted.

>Those are semantic relationships with the rest of the world. If this is all that counts as semantic relationships, then I see no reason why a language model doesn't have this kind of semantic relationship, albeit in a very different modality. Tokens and their co-occurrences are a kind of sensor to the world. In the same way we discover quantum mechanics by way of induction over indirect relationships among the signa…

> I see no reason why a language model doesn't have this kind of semantic relationship One certainly could hook up a language model to sensors and actuators to give it semantic relationships with the rest of the world. But nobody has done this. And giving it semantic relationships of the same order of complexity and richness that human brains have is an extremely tall order, one I don't expect anyone to come anywhere…

So much rides on your implicit notion of semantic relationship, but this dependence needs demonstration. The fact that some pattern of signals on my perceptual apparatus is caused by an apple in the real world does not mean that I have knowledge or understanding of an apple in virtue of this causal relation. That my sensory signals are caused by apples is an accident of this world, one we are completely blind to. If all apples in the world were swapped with fapples (fake apples), where all sensory experiences that have up to now been caused by apples are now caused by fapples, we would be none the wiser. The semantics (i.e. wide content) of our perceptual experiences is irrelevant to literally everything we know and how we interact with the world. Our knowledge of the world is limited to our sensory experiences and our deductions, inferences, etc derived from our sensory experiences. Our situatedness in the world is only relevant insofar as it entails the space of our sensory experiences.

>the model would need the ability to frame hypotheses about what this sensor data means, and test them by interacting with the world and seeing what the results were.

Why do we need to actively test our model to come to an understanding of the world? Yes, that is how we biological organisms happen to learn about the world. But it is not clear that it is required. Language models learn by developing internal models that predict the next token. But this prediction implicitly generates representations of the processes that generate the tokens. There is no in principle limit to the resolution of this model given a sufficiently large and diverse training set.

Re: Theory of Mind May Have Spontaneously Emerged in Large Language Models

#193

Post by uh_uh »

pdonis wrote: post #93
strbean wrote: post #78

Earlier quoted context omitted.

> Human minds have semantic relationships to the rest of the world that LLMs do not have. This is a bold claim that seems built on a presumption of mind-body dualism. Brains don't have semantic relationships with anything. They are neurons hooked up to sensors and actuators. Any inferences they produce are the result of statistical processes.

> Any inferences they produce are the result of statistical processes. I don't know how you would justify this claim since we don't know nearly enough about how the brain actually does things like make inferences.

What else would it be? Are you suggesting some non-physical process?

Re: Theory of Mind May Have Spontaneously Emerged in Large Language Models

#194

Post by thaumasiotes »

abeppu wrote: post #117
space_fountain wrote: post #56

Earlier quoted context omitted.

I wonder every time I see this take what it would mean under this definition of knowing things for a machine learning algorithm to ever know something. I find that especially important because to every appearance we are a machine learning algorithm. I don’t know how different the sort of knowing this algorithm has to the sort of knowing a human has, but you’re far more confident than I am that it’s a difference of ki…

Defining what "knowing" is would be useful, yes, and analytic philosophers in epistemology do argue about this. One attribute that's classically part of the definition of "knowing" is that the thing which is known must be true. LLMs are pretty bad at this, but perhaps that can be fixed. But I would challenge you to imagine the situation the LLM is actually in. Do you understand Thai? If so, in the following, feel fre…

> Suppose you study this giant stack of Thai text for years in isolation. After all this study, you're good enough that given a few written Thai words, you can write sequences of words that are likely to follow, given what you know of these patterns. You can fill in blanks. But should anyone guess that you "know" what you're saying? Nothing has ever indicated to you what any of these words _mean_. If you give back a sequence of words, which a Thai speakers understands to be expressing an opinion about monetary policy, because you read several similar sequences in the pile, is that even your opinion?

Note that this isn't just an exotic thought experiment. People like this already exist; the condition is known as "Wernicke's aphasia". People displaying this condition can speak normally. They can't understand things; they are missing a normal mental mapping from words to meanings.

Re: Theory of Mind May Have Spontaneously Emerged in Large Language Models

#195

Post by tialaramex »

avgcorrection wrote: post #92
tialaramex wrote: post #36

Earlier quoted context omitted.

Insistence that the brain just isn't a computer is extremely widespread among those who are experts in the brain and know little about computers. As someone in the opposite situation I must say that their observations about what's special about the brain fit most closely to what I understand about computers and bring me to exactly the opposite conclusion. If the brain is not a computer, it's frankly eerie how similar…

Of course metaphor trumps domain expertise.

The trouble is domain expertise in what domain. The "Brains aren't computers" rants are predominantly from Psychologists who I agree have domain expertise when it comes to the brain but know very little about computers.

An example would be the problem of understanding. The Psychologists are confident we can't expect to fully understand our own minds. But, they are also confident we can expect to understand any possible Computer Program. And they're just wrong about that, that's the implication of Kurt Gödel's work, we definitely can't expect to understand arbitrary Computer Programs, we have instead chosen to mostly try to write programs from a narrow set we can understand, although not altogether successfully. Thus, the Psychologist thinks they've found an obvious difference, but I think they found an obvious similarity!

Re: Theory of Mind May Have Spontaneously Emerged in Large Language Models

#196

Post by TheOtherHobbes »

space_fountain wrote: post #56
lsy wrote: post #33

This highlights one of the types of muddled thinking around LLMs. These tasks are used to test theory of mind because for people, language is a reliable representation of what type of thoughts are going on in the person's mind. In the case of an LLM the language generated doesn't have the same relationship to reality as it does for a person. What is being demonstrated in the article is that given billions of tokens o…

I wonder every time I see this take what it would mean under this definition of knowing things for a machine learning algorithm to ever know something. I find that especially important because to every appearance we are a machine learning algorithm. I don’t know how different the sort of knowing this algorithm has to the sort of knowing a human has, but you’re far more confident than I am that it’s a difference of ki…

Stick a pin in your finger.

That pain is what knowing something means.

Philosophically we're talking about embodied qualia, which is how humans experience objects and more basic sensations.

Language happens later - much later.

The defining property of a bag isn't that you can put things in it. Like language that comes later. The defining properties are how it feels when you hold it, when you open it, the differences in sensation between empty/partially empty/full. And so on.

An LLM has no embodied experience, so it has no idea what a bag feels like as a set of physical sensations and directly perceived relationships.

Failure to understand embodiment has done more to hold back AI than any other philosophical error. Researchers have assumed - wrongly - that you can define an object by its visual properties and its linguistic associations.

That's simply not how it works for humans. We get there after a while, but we start from something far more visceral - so much so that many fundamental linguistic abstractions are metaphors based on the simplest and most common qualia.

Re: Theory of Mind May Have Spontaneously Emerged in Large Language Models

#197

Post by helf »

oliveshell wrote: post #81
pdonis wrote: post #52

Earlier quoted context omitted.

> In the case of an LLM the language generated doesn't have the same relationship to reality as it does for a person. I wish I could upvote this 1000 times. It is the core issue that all the hype surrounding LLMs consistently fails to address or even acknowledge.

Strongly agree. It’s been eye-opening to see how often otherwise very bright, highly technical people stumble at this sort of critical thinking hurdle.

It’s hard to watch it happen over and over again. I think it’s because they /want/ something to be a certain way regardless of reality.

There is no realm in which a LLMs have spontaneously gained theory of mind.

I don’t understand why people are so eager to jump to conclusions on these things lately.

Re: Theory of Mind May Have Spontaneously Emerged in Large Language Models

#198

Post by harrisontan »

soiler wrote: post #131
PaulHoule wrote: post #9

Earlier quoted context omitted.

By "pointing" I don't mean pointing your finger, the universal version of "pointing" involves your eyes, shoulders and the rest of your body. (Think of a how a hunting dog points... Plenty of times I've seen cats point out things to other cats not to mention to me.)

Those behaviors could be better described as "looking" and "facing", couldn't they? Pointing means something pretty specific to humans, especially when we're talking about human-animal interactions.

From an evolutionary/physiology standpoint, human bipedal locomotion freed up our forelimbs/arms such that "limb assisted pointing" became possible/practical. Perhaps why elephants seem to grasp human pointing is because their trunks serve as a similarly "free" limb that they can use to point at things. Sort of like convergent evolution, but behavioral rather than morphological...

> "When they detect something alarming, they characteristically face towards it and raise their trunk above their head with the tip of the trunk pointed to [the danger]," Byrne said. "We've always thought they were sniffing the breeze, but maybe they're also pointing; our results suggest that's more than possible."

https://www.nationalgeographic.com/animals/article/131010-el...

Re: Theory of Mind May Have Spontaneously Emerged in Large Language Models

#199

Post by abeppu »

TeMPOraL wrote: post #185
space_fountain wrote: post #144

Earlier quoted context omitted.

Let me respond with an analogy of my own. Imagine you are a scientist on an alien world. The aliens primary experience the world through magnetic fields. They live deep in the atmosphere of a hot Jupiter like planet and rarely touch anything and have no eyes. Still they are intelligent beings and so quickly they are able to establish communication with you. A computer translates and you both have to become a bit more…

My intuition is that the difference between GP's analogy and the Chinese room is in computing power of the system, in the sense of Chomsky hierarchy[0] (as opposed to instructions per second). In the Chinese room, the instructions you're given to manipulate symbols could be Turing-complete programs, and thus capable of processing arbitrary models of reality without you knowing about them. I have no problem accepting…

> I have no problem accepting the "entire room" as a system understands Chinese.

> you'll fail whenever the task requires you to use the kind of computational model your interlocutor uses.

I think it's important to distinguish between knowing the language and knowing anything about the stuff being discussed in the language. The top level comment all this is under mentioned knowing what a bag is or what popcorn is. These don't require computational complexity, but do require some other data than just text, and a model that can relate multiple kinds of input.

Re: Theory of Mind May Have Spontaneously Emerged in Large Language Models

#200

Post by JieJie »

bawolff wrote: post #115
jfengel wrote: post #41

Earlier quoted context omitted.

By the time they were trying to defeat the world champion, they didn't really think of it as an AI project any more. They often used the term "expert system", which had a much narrower scope. It had become a challenge unto itself, but they didn't expect it to lead to any kind of generalized intelligence. It came rather out of nowhere when neural-net-type engines suddenly swept back into dominance. Even after becoming…

> but they didn't expect it to lead to any kind of generalized intelligence. Do experts really expect any stream of AI research to lead to generalized intelligence (except in the very long term)? I was under the impression we really have no idea how to get there.

"Carmack sees a 60% chance of achieving initial success in AGI by 2030. Here’s how, and why, he’s working independently to make it happen."

https://dallasinnovates.com/exclusive-qa-john-carmacks-diffe...

Post reply on HN