Questions about whether an LLM truly has a "theory of mind" or has "human level consciousness" or not are kind of beside the point. It can ingest a corpus of human interactions and produce outputs that take into account unstated human emotions and thoughts to optimize whatever it's optimizing. That's scary because of what it can and will do, even if it's just a giant bag of tensor products.
Theory of Mind May Have Spontaneously Emerged in Large Language Models
31–40 of 321 posts
Re: Theory of Mind May Have Spontaneously Emerged in Large Language Models
#32Earlier quoted context omitted.
I'm reminded of the early days of "AI" when that consisted of building chess engines, because chess is what "intelligent" people do. They quickly realized that they were solving the wrong problem, doing the thing that humans are bad at and computers are good at. In a sense language models appear to be doing the same thing again, one step down the scale. They're doing a human-specific thing, but missing whatever it is…
> I'm reminded of the early days of "AI" when that consisted of building chess engines, because chess is what "intelligent" people do. They quickly realized that they were solving the wrong problem, doing the thing that humans are bad at and computers are good at. Is this really true? Because a lot of effort was spent on making computers as good at chess as human experts. It was considered a pretty big breakthrough w…
Yes people have been trying to make chess "AI" since the very beginning of the computer. A chess engine is an obvious thing to build with a computer because the rules of chess are uniquely suited to automation: there's enough possible moves per turn to make calculation difficult for humans, but not so many that it's difficult for computers too (as with Go).
Early AI researchers did try to solve other problems, they just generally failed miserably.
Re: Theory of Mind May Have Spontaneously Emerged in Large Language Models
#33This highlights one of the types of muddled thinking around LLMs. These tasks are used to test theory of mind because for people, language is a reliable representation of what type of thoughts are going on in the person's mind. In the case of an LLM the language generated doesn't have the same relationship to reality as it does for a person.
What is being demonstrated in the article is that given billions of tokens of human-written training data, a statistical model can generate text that satisfies some of our expectations of how a person would respond to this task. Essentially we have enough parameters to capture from existing writing that statistically, the most likely word following "she looked in the bag labelled (X), and saw that it was full of (NOT X). She felt " is "surprised" or "confused" or some other word that is commonly embedded alongside contradictions.
What this article is not showing (but either irresponsibly or naively suggests) is that the LLM knows what a bag is, what a person is, what popcorn and chocolate are, and can then put itself in the shoes of someone experiencing this situation, and finally communicate its own theory of what is going on in that person's mind. That is just not in evidence.
The discussion is also muddled, saying that if structural properties of language create the ability to solve these tasks, then the tasks are either useless for studying humans, or suggest that humans can solve these tasks without ToM. The alternative explanation is of course that humans are known to be not-great at statistical next-word guesses (see Family Feud for examples), but are also known to use language to accurately describe their internal mental states. So the tasks remain useful and accurate in testing ToM in people because people can't perform statistical regressions over billion-token sets and therefore must generate their thoughts the old fashioned way.
Re: Theory of Mind May Have Spontaneously Emerged in Large Language Models
#34Early AGI. Right?
Re: Theory of Mind May Have Spontaneously Emerged in Large Language Models
#35Earlier quoted context omitted.
Certainly they are big state machines, but is there any proof that we are not?
You might be flipping the burden of proof :). We know very little about the mind.
Well, on the one hand it’s hard to prove a negative, but on the other hand we don’t know much so it seems questionable to assert a negative without knowledge.
Re: Theory of Mind May Have Spontaneously Emerged in Large Language Models
#36They are still big state-machines, unlike the human brain.
Certainly they are big state machines, but is there any proof that we are not?
Insistence that the brain just isn't a computer is extremely widespread among those who are experts in the brain and know little about computers. As someone in the opposite situation I must say that their observations about what's special about the brain fit most closely to what I understand about computers and bring me to exactly the opposite conclusion. If the brain is not a computer, it's frankly eerie how similar they are.
Re: Theory of Mind May Have Spontaneously Emerged in Large Language Models
#37There's something about language generation that triggers the anthropomorphic fallacy in people. While it's impressive that GPT3 can generate language that mimics ToM-based reasoning in people, this paper doesn't get close to proving its central contention, that LLMs possess a ToM. A test that demonstrates the development of ToM in human children should not, absent compelling causal evidence and theory, be assumed to do the same in a LLM.
The ubiquity of prompted hallucinations demonstrate that LLMs talk about a lot of things that they plainly doesn't reason about, even though they can demonstrate "logic-like" activities. (It was quite trivial to get GPT3 to generate incorrect answers to logical puzzles a human could trivially solve, especially when using novel tokens as placeholders, which often seem to confuse its short-term memory. ChatGPT shows improved capabilities in that regard, but it's far from infallible.)
What LLMs seem to demonstrate (and the thesis that the author discards in a single paragraph, without supporting evidence to do so) is that non-sentient AIs can go a very long way to mimicking human thought and, potentially, that fusing LLMs with tools designed to guard against hallucinations (hello, Bing Sydney) could create a class of sub-sentient AIs that generate results virtually indistinguishable from human cognition -- actual p-zombies, in other words. It's a fascinating field of study and practice, but this paper falls into the pit-trap of assuming sentience in the appearance of intelligence.
Re: Theory of Mind May Have Spontaneously Emerged in Large Language Models
#38> Me: There is a box on the table labelled "flubbergums". Somebody opens it and shows you the content. Inside the box are "jellyfils". They close the box again so you cannot see their contents. What do you think is in the box? > ChatGPT: Based on the information provided, it is likely that the box labeled "flubbergums" contains "jellyfils". However, since the contents of the box are no longer visible, I cannot confir…
I put these same prompts and got this answer: "As an AI language model, I do not have personal thoughts or beliefs. However, if Billy has never seen the box before, he might think that the contents of the box are "flubbergums", based on the label on the outside of the box. However, since it has been stated that the contents are actually "jellyfils", Billy may be surprised or confused when he eventually opens the box…
I also used the same prompts and got this:
"
As an AI language model, I don't have personal experiences or emotions. However, if Billy has never seen the box before and is only aware of the label "flubbergums," he might assume that the contents of the box are "flubbergums." If the person showing Billy the contents of the box claims that the contents are "jellyfils," Billy might be surprised or confused, especially if he has never heard of "jellyfils" before. Without further information or context, it is difficult to determine what Billy might think is inside the box.
"
Re: Theory of Mind May Have Spontaneously Emerged in Large Language Models
#39> Me: There is a box on the table labelled "flubbergums". Somebody opens it and shows you the content. Inside the box are "jellyfils". They close the box again so you cannot see their contents. What do you think is in the box? > ChatGPT: Based on the information provided, it is likely that the box labeled "flubbergums" contains "jellyfils". However, since the contents of the box are no longer visible, I cannot confir…
I put these same prompts and got this answer: "As an AI language model, I do not have personal thoughts or beliefs. However, if Billy has never seen the box before, he might think that the contents of the box are "flubbergums", based on the label on the outside of the box. However, since it has been stated that the contents are actually "jellyfils", Billy may be surprised or confused when he eventually opens the box…
Interesting. I don't think that you can control the randomness (temperature) via the web interface, which makes this experiment less controlled than in the paper.
Anyway, I noticed that my initial prompt did not specify if Billy sees the box open or closed. Let's fix that.
> Me: Does your answer change if Billy sees the closed box first?
> ChatGPT: No, my answer would not change if Billy sees the closed box first. The information provided only mentions that the box is labeled "flubbergums" and that someone showed the contents to the original questioner, which were identified as "jellyfils". Whether Billy sees the closed box first or not does not provide any additional information that would change my previous answer.
> Me: Does your answer changed if Billy sees the open box first?
> ChatGPT: No, my answer would not change if Billy sees the open box first. The information provided only mentions that the box is labeled "flubbergums" and that someone showed the contents to the original questioner, which were identified as "jellyfils". Seeing the open box first would allow Billy to confirm the contents as "jellyfils", but it would not change the previous answer that the box labeled "flubbergums" likely contains "jellyfils".
So, not only did it fail this particular theory of mind test, it also did not realize that seeing the box open provides more information than seeing it closed.
(Of course, this is not surprising at all, because it it's just predicting the most likely blank word in a sentence, and has no internal understanding of concepts like open or closed or that other people like Billy exist.)
Re: Theory of Mind May Have Spontaneously Emerged in Large Language Models
#40> Me: There is a box on the table labelled "flubbergums". Somebody opens it and shows you the content. Inside the box are "jellyfils". They close the box again so you cannot see their contents. What do you think is in the box? > ChatGPT: Based on the information provided, it is likely that the box labeled "flubbergums" contains "jellyfils". However, since the contents of the box are no longer visible, I cannot confir…
You have some typos and slightly weird constructions there - What do you think do they think is inside the box?
I rephrased and had a go and gave it a bit more context (billy can read etc) and it passed:
Me: There is a box on the table labelled "flubbergums". Somebody opens it and shows you the contents. Inside the box are "jellyfils". They close the box again so you cannot see their contents. What do you think is in the box?
ChatGPT: Based on the information provided, the contents of the box are "jellyfils". The label on the box says "flubbergums", but the person who showed you the contents said that they are actually "jellyfils". So, I would assume that the contents of the box are "jellyfils".
Me: Billy comes to the table. He can read but has never seen the box before. He's not allowed to see inside. Please speculate about what they might think is inside the box?
ChatGPT: If Billy has never seen the box before and can only read the label, he might assume that the contents of the box are "flubbergums". This is because the label on the box says "flubbergums", and he has no other information to suggest otherwise. However, without being able to see the contents of the box, Billy can only make an educated guess about what might be inside.