Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Theory of Mind May Have Spontaneously Emerged in Large Language Models

arxiv.org

Post reply on HN
Print view View original

171–180 of 321 posts

Re: Theory of Mind May Have Spontaneously Emerged in Large Language Models

#171

Post by abeppu »

abeppu wrote: post #117
space_fountain wrote: post #56

Earlier quoted context omitted.

I wonder every time I see this take what it would mean under this definition of knowing things for a machine learning algorithm to ever know something. I find that especially important because to every appearance we are a machine learning algorithm. I don’t know how different the sort of knowing this algorithm has to the sort of knowing a human has, but you’re far more confident than I am that it’s a difference of ki…

Defining what "knowing" is would be useful, yes, and analytic philosophers in epistemology do argue about this. One attribute that's classically part of the definition of "knowing" is that the thing which is known must be true. LLMs are pretty bad at this, but perhaps that can be fixed. But I would challenge you to imagine the situation the LLM is actually in. Do you understand Thai? If so, in the following, feel fre…

I only just realized I should have described this using English, but you only see the token ids emitted by an encoder. You can't read its source, and you never get to invoke it on your own inputs.

Re: Theory of Mind May Have Spontaneously Emerged in Large Language Models

#172

Post by manv1 »

manv1 wrote: post #44
pavel_lishin wrote: post #8

Earlier quoted context omitted.

I recall reading that pointing is something that only a few animals, including us, do and understand. If I get my cats' attention and point at something, they're more interested in the tip of my finger than the direction I'm pointing at. Now, my cats will occasionally meow to get my attention and then walk over to where the problem is - an empty food dish, an empty water bowl, the bed that they expect us to be in bec…

Many dogs will look at what you're pointing at and not your finger. It depends on the dog. And there are dogs that are literally bred to point...but they're usually pointing at game. There are dogs that will literally drag you to what they're trying to show you. And many (most?) dogs will bring you a toy to you that they want you to play with. I've never actually seen a cat do that, but I presume there must be a few.

Dogs can also tell you're pointing with your attention, if they're paying attention.

Re: Theory of Mind May Have Spontaneously Emerged in Large Language Models

#173

Post by yamrzou »

canniballectern wrote: post #53
lsy wrote: post #33

This highlights one of the types of muddled thinking around LLMs. These tasks are used to test theory of mind because for people, language is a reliable representation of what type of thoughts are going on in the person's mind. In the case of an LLM the language generated doesn't have the same relationship to reality as it does for a person. What is being demonstrated in the article is that given billions of tokens o…

Of course, today's LLMs only appear to have theory of mind at first glance and fall apart under closer scrutiny. But if they can continue to become more and more accurate replicas of the real thing, I don't think it matters at all. There's no way to know for sure that anyone other than yourself experiences consciousness. All you can do is judge for yourself that what they're describing matches closely enough with you…

> There's no way to know for sure that anyone other than yourself experiences consciousness.

- Do you see how the fish are coming to the surface and swimming around as they please? That's what fish really enjoy.

- You're not a fish, replied Hui Tzu, so how can you say you know what fish really enjoy?

- You are not me, said Zhuangzi, so how can you know I don't know what fish enjoy.

Re: Theory of Mind May Have Spontaneously Emerged in Large Language Models

#174

Post by [deleted] »

didntreadarticl wrote: post #40
hesk wrote: post #20

> Me: There is a box on the table labelled "flubbergums". Somebody opens it and shows you the content. Inside the box are "jellyfils". They close the box again so you cannot see their contents. What do you think is in the box? > ChatGPT: Based on the information provided, it is likely that the box labeled "flubbergums" contains "jellyfils". However, since the contents of the box are no longer visible, I cannot confir…

You have some typos and slightly weird constructions there - What do you think do they think is inside the box? I rephrased and had a go and gave it a bit more context (billy can read etc) and it passed: Me: There is a box on the table labelled "flubbergums". Somebody opens it and shows you the contents. Inside the box are "jellyfils". They close the box again so you cannot see their contents. What do you think is in…

[deleted]

Re: Theory of Mind May Have Spontaneously Emerged in Large Language Models

#175

Post by Syntonicles »

albystein wrote: post #137
lsy wrote: post #33

This highlights one of the types of muddled thinking around LLMs. These tasks are used to test theory of mind because for people, language is a reliable representation of what type of thoughts are going on in the person's mind. In the case of an LLM the language generated doesn't have the same relationship to reality as it does for a person. What is being demonstrated in the article is that given billions of tokens o…

I think this line of reasoning is misguided. What’s striking and more important to focus on are the abstract reasoning abilities of these systems. Language, as you mentioned, abstracts real world objects and phenomena, so it’s a good approximation of the real world. Thus, if an LLM can reason this well using language, it’s safe to say that perhaps they’re doing something akin to what the human mind does. Your critiqu…

I agree. We've [potentially] given the grounding assimilation step a boost already, since language is already organized.

Imagine that a language model is fully integrated with sense-data that exceeds human first-hand experience. Perhaps they are trained on and can generate realistic 3D models of objects, and derive estimates of their internal construction, weight, etc. Perhaps they recall infrared emissions or opacity to EM wavelengths. Would we truly "know" what we're talking about by that standard?

I'm not actually sure why we don't consider generative image models to be grounded already. They seem to be able to modify, transform and rotate imagery. That indicates spatial understanding to me, and I'm not sure how much more we must require of them without having to exclude blind or otherwise disabled humans from our definition of comprehension.

Re: Theory of Mind May Have Spontaneously Emerged in Large Language Models

#177

Post by schrodinger »

didntreadarticl wrote: post #130
abeppu wrote: post #117

Earlier quoted context omitted.

Defining what "knowing" is would be useful, yes, and analytic philosophers in epistemology do argue about this. One attribute that's classically part of the definition of "knowing" is that the thing which is known must be true. LLMs are pretty bad at this, but perhaps that can be fixed. But I would challenge you to imagine the situation the LLM is actually in. Do you understand Thai? If so, in the following, feel fre…

Ok I like your thought experiment. Lets change it a bit. Instead of it being an unknown language, its English (a language you know), but every single Noun, Verb, Adjective or Preposition has been changed to Thai (a language you dont know). The Mæw Nạ̀ng Bn the S̄eụ̄̀x. If you had sufficient opportunity to study this pile of text, you'd begin to pick out patterns of which words appear together, and what order words of…

Basically the "rosetta stone" theory, right?

Re: Theory of Mind May Have Spontaneously Emerged in Large Language Models

#178

Post by HillRat »

low_tech_punk wrote: post #83
HillRat wrote: post #37

There's something about language generation that triggers the anthropomorphic fallacy in people. While it's impressive that GPT3 can generate language that mimics ToM-based reasoning in people, this paper doesn't get close to proving its central contention, that LLMs possess a ToM. A test that demonstrates the development of ToM in human children should not, absent compelling causal evidence and theory, be assumed to…

May I play devil's advocate? The fallacy of this paper granted, is it worth questioning our belief that there is more to intelligence than the "appearance of intelligence"? What if the lack of hallucination in human being is due to our self-imposed guard (hello, frontal cortex) that is developed via an evolutionary process (aka, biological reinforcement training)? To stretch the argument a bit further, what if halluc…

To be honest, I think your question is the other side of the conceptual coin here -- either the article is wrong and GPT3 isn't "sentient," or the article is right and we need to radically upend our concept of sentience, probably via an eliminativist materialism that ends in at least epiphenomenalism if not full-bore illusionism, removing either free will or consciousness itself from our worldview.

To be honest, I find that approach compelling if not comforting -- at a minimum, it implies that our consciousness is just along for the ride in a deterministic meat machine; at worst, it means that what we consider "sentience" is just an illusion of an illusion. It's entirely possible to me that eventually AI will reach a point where it'll falsify many of our assumptions about what "mind" is, even if I'm sure that LLMs don't, at a minimum, satisfy our folk conceptions of consciousness.

Re: Theory of Mind May Have Spontaneously Emerged in Large Language Models

#179

Post by quonn »

letmevoteplease wrote: post #122
sgt101 wrote: post #70

Earlier quoted context omitted.

No, it's paraphrasing it's training data that likely contains these tasks in one form or another. Here's one I made : me : There's a case in the station and the policeman opens it near the fireman. The dog is worried about the case but the policeman isn't, what does the fireman think is in the station? chatgpt : As a language model, I do not have access to the thoughts of individuals, so I cannot say what the fireman…

There's a good chance a human would respond in the same way, because they would assume you were asking a good-faith question instead of nonsense. Try asking it an original question that has some kind of deducible answer. Its abilities are more impressive than you would expect from an algorithm that just predicts the next word. I doubt there is anything quite like this situation in the training data: https://i.imgur.c…

> than you would expect from an algorithm that just predicts the next word.

I think there is common mistake in this concept of just predicting the next word. While it is true that just the next word is predicted, a good way to do that is to internally imagine more than the next word and then just spit out the next word. Of course with the word after that the process repeats with a new imagination.

One may say that this is not what it does and I would say, show me that this is not exactly what the learned state does. Even if the following words are never constructed anywhere, they can be implied in the computation.

The say this differently, what we think is just the next word is actually the continuation that then manifests as a single word. This would remain true even if, in fact, the task is to only predict the next word. Which is to say that the next word is actually more than what it sounds.

Re: Theory of Mind May Have Spontaneously Emerged in Large Language Models

#180

Post by shinycode »

swatcoder wrote: post #84
didntreadarticl wrote: post #66

Earlier quoted context omitted.

OK how about this: It 'knows' language, as in it has learnt about relationships between words (and thats really underselling it, in reality it has learnt very very subtle relationships between a great many words, and it can process words about 2000 at a time (token count etc)) BUT as you say it has no outside reference, its just a bundle of weights (those weights forming models of a sort) BUT we provide the outside c…

The future AI you describe has language at the center of its understanding. For example, you would expect that the cameras and arm-feedback-sensors would produce text (or text-associated weights/tokens) that describe what the AI "sees" and the robotic arms would be receive some kind of language-derived directives from the LLM. It will be very interesting to see what that system is capable of. I think a lot of people…

At the end the real question is what morality the AI is being taught ? It’s own ? A religious inspired one ? It can’t be non if a machine decides not to help or worse, do actions that hurt or kill someone. Would they abide to laws ? Be destroyed if anything happens ? Would people could influence machine with dialogue and make them do something ?

What’s the purpose and imposed limits of such machines ?

Post reply on HN