Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Theory of Mind May Have Spontaneously Emerged in Large Language Models

arxiv.org

Post reply on HN
Print view View original

141–150 of 321 posts

Re: Theory of Mind May Have Spontaneously Emerged in Large Language Models

#141

Post by abeppu »

ghayes wrote: post #129
abeppu wrote: post #117

Earlier quoted context omitted.

Defining what "knowing" is would be useful, yes, and analytic philosophers in epistemology do argue about this. One attribute that's classically part of the definition of "knowing" is that the thing which is known must be true. LLMs are pretty bad at this, but perhaps that can be fixed. But I would challenge you to imagine the situation the LLM is actually in. Do you understand Thai? If so, in the following, feel fre…

Personally, I'm not convinced, in your hypothetical, that the participant does not "know" Thai at that point. Seeing a young child learn language, it's a lot more adaptive than I think we tend to see language learning, as we often think about learning language as a teenager and not a toddler. I agree the machine does not know what a pizza tastes like nor does it know what it is to _want_ pizza, but I'm not sure that…

Maybe the participant "knows" something about the Thai language? But that's different from knowing anything about the things being discussed. The jumping off point for this, which motivated a question about what it is to know, was the comment:

> What this article is not showing (but either irresponsibly or naively suggests) is that the LLM knows what a bag is, what a person is, what popcorn and chocolate are, and can then put itself in the shoes of someone experiencing this situation, and finally communicate its own theory of what is going on in that person's mind. That is just not in evidence.

Knowing something about the patterns of word order in Thai is not the same as knowing about the world being discussed in Thai.

Re: Theory of Mind May Have Spontaneously Emerged in Large Language Models

#142

Post by lee101 »

albystein wrote: post #137
lsy wrote: post #33

This highlights one of the types of muddled thinking around LLMs. These tasks are used to test theory of mind because for people, language is a reliable representation of what type of thoughts are going on in the person's mind. In the case of an LLM the language generated doesn't have the same relationship to reality as it does for a person. What is being demonstrated in the article is that given billions of tokens o…

I think this line of reasoning is misguided. What’s striking and more important to focus on are the abstract reasoning abilities of these systems. Language, as you mentioned, abstracts real world objects and phenomena, so it’s a good approximation of the real world. Thus, if an LLM can reason this well using language, it’s safe to say that perhaps they’re doing something akin to what the human mind does. Your critiqu…

[flagged]

Re: Theory of Mind May Have Spontaneously Emerged in Large Language Models

#143

Post by p0pcult »

low_tech_punk wrote: post #83
HillRat wrote: post #37

There's something about language generation that triggers the anthropomorphic fallacy in people. While it's impressive that GPT3 can generate language that mimics ToM-based reasoning in people, this paper doesn't get close to proving its central contention, that LLMs possess a ToM. A test that demonstrates the development of ToM in human children should not, absent compelling causal evidence and theory, be assumed to…

May I play devil's advocate? The fallacy of this paper granted, is it worth questioning our belief that there is more to intelligence than the "appearance of intelligence"? What if the lack of hallucination in human being is due to our self-imposed guard (hello, frontal cortex) that is developed via an evolutionary process (aka, biological reinforcement training)? To stretch the argument a bit further, what if halluc…

Have you ever read any Julian Jaynes? Check out "The Origin of Consciousness in the Breakdown of the Bicameral Mind"

Re: Theory of Mind May Have Spontaneously Emerged in Large Language Models

#144

Post by space_fountain »

abeppu wrote: post #117
space_fountain wrote: post #56

Earlier quoted context omitted.

I wonder every time I see this take what it would mean under this definition of knowing things for a machine learning algorithm to ever know something. I find that especially important because to every appearance we are a machine learning algorithm. I don’t know how different the sort of knowing this algorithm has to the sort of knowing a human has, but you’re far more confident than I am that it’s a difference of ki…

Defining what "knowing" is would be useful, yes, and analytic philosophers in epistemology do argue about this. One attribute that's classically part of the definition of "knowing" is that the thing which is known must be true. LLMs are pretty bad at this, but perhaps that can be fixed. But I would challenge you to imagine the situation the LLM is actually in. Do you understand Thai? If so, in the following, feel fre…

Let me respond with an analogy of my own. Imagine you are a scientist on an alien world. The aliens primary experience the world through magnetic fields. They live deep in the atmosphere of a hot Jupiter like planet and rarely touch anything and have no eyes. Still they are intelligent beings and so quickly they are able to establish communication with you. A computer translates and you both have to become a bit more familiar with each other's modes of perceiving the world. You could write a whole novel explaining this sort of difference in modes of perception, but my question is if you, the human, can learn to understand what it is to perceive magnetic fields? I think obviously the answer is yes. In fact, if you are to communicate you'll have to. I think the sort of modal/sense difference your analogy plays on is similar because I think for a human to get good at responding you'd have to start knowing things about the symbols. That knowledge obviously wouldn't be grounded in a way that you could translate it back into English. But you might for example learn that one word is a type of another or even that some words describe entities that are then referenced later and to actually get good at it, which it's not at all clear a human could, even that some entities have hidden state

This feels related to the idea of the Chinese room. There I think the resolution is that the human following instructions does not understand Chinese but the room, the system of instructions + the human to follow them does. In a similar way obviously an individual neuron doesn't understand anything but brains do.

I guess it just feels like this general argument, that merely seeing things and making predictions that turn out to be right isn't enough to understand it will never go away. We could have a full fledged robot walking around having conversations and I could dispute its ability to really understand. It's just learned to imitate other humans I'd say. It doesn't really know anything, it's just following a statistical model to decide how to move an arm

Re: Theory of Mind May Have Spontaneously Emerged in Large Language Models

#145

Post by didntreadarticl »

soiler wrote: post #125
didntreadarticl wrote: post #74

Earlier quoted context omitted.

There are two camps, evident in this thread. one camp is 'its just a statistical model, it cant possibly know these things' The other camp (that I'm in) sees that we might be onto something. We humans are obviously just more than a statistical model, but nonetheless learning words and how they fit together is a big part of who we are. With LLMs we have our first glimpse of 'emergent' behaviour from simple systems sca…

I don't think you've represented the camps fairly (actually, I don't think there are two camps). Most people (here) are probably not arguing that AGI is impossible, but that current AI is not generally intelligent. The John Carmack quote is exactly in line with this. He says "ride those to [AGI]," meaning they are not AGI. The idea that genuine intelligence and self-awareness could emerge from increasingly powerful s…

Oh of course its not it. The question is how it relates to some future better thing. Is it a step on the road or a dead end.

I'm arguing against the 'its just a statistical model and its playing a clever trick on us' camp.

Re: Theory of Mind May Have Spontaneously Emerged in Large Language Models

#146

Post by abeppu »

LarryMullins wrote: post #133
abeppu wrote: post #117

Earlier quoted context omitted.

Defining what "knowing" is would be useful, yes, and analytic philosophers in epistemology do argue about this. One attribute that's classically part of the definition of "knowing" is that the thing which is known must be true. LLMs are pretty bad at this, but perhaps that can be fixed. But I would challenge you to imagine the situation the LLM is actually in. Do you understand Thai? If so, in the following, feel fre…

Your Thai text generator example seems like a reformulation of the "Chinese Room" thought experiment, except you're running the system using a single human brain instead of many. I'm not sure that makes a difference. The human running the system doesn't understand Thai, but perhaps that system itself does.

I agree that the system of OpenAI, ChatGPT, and a user entering text on their website taken together may contain knowledge of "what a bag is, what a person is, what popcorn and chocolate are", etc. I do not agree that the LLM on its own "knows" what any of those things are.

Re: Theory of Mind May Have Spontaneously Emerged in Large Language Models

#147

Post by mshake2 »

vbezhenar wrote: post #106

I wonder if we can train network on some person data (like diaries and so on) and let it imitate this person? Something like died person resurrected in computer. Kind of spooky.

Future psychiatrists will prescribe sessions with a model trained on everything your deceased loved one has ever written or said, to help you with the grieving process. It will be by prescription only because it is very addictive and should only be used to help bring closure.

Re: Theory of Mind May Have Spontaneously Emerged in Large Language Models

#148

Post by hackinthebochs »

pdonis wrote: post #57
spuz wrote: post #48

Earlier quoted context omitted.

> So the tasks remain useful and accurate in testing ToM in people because people can't perform statistical regressions over billion-token sets and therefore must generate their thoughts the old fashioned way. Is it not also possible that the study suggests that the human mind actually operates as a statistical regression over billions of data points rather than through some kind of Baysian logic? You say humans are…

> Is it not also possible that the study suggests that the human mind actually operates as a statistical regression over billions of data points rather than through some kind of Baysian logic? No. Human minds have semantic relationships to the rest of the world that LLMs do not have. The comparisons being made between the two, not just in this paper but in all of the hype surrounding LLMs, are simply invalid. But the…

>Human minds have semantic relationships to the rest of the world that LLMs do not have

Can you make this point without stuffing all the hard work into poorly defined words like "semantic relationships"? Until then, I'm not sure the point is intelligible.

Re: Theory of Mind May Have Spontaneously Emerged in Large Language Models

#149

Post by spuz »

pdonis wrote: post #111
mietek wrote: post #99

Earlier quoted context omitted.

Okay. It seems to me that, in analogy to humans having such a relationship with physical reality, machine learning systems have such a relationship with virtual reality — the environment in which they are trained and deployed.

> machine learning systems have such a relationship with virtual reality No, they don't, because they can't take actions in that virtual reality and sense the consequences. They can't test hypotheses about how the reality works. They can't even frame hypotheses about how the reality works.

Right, but some neural networks are trained in virtual 3D physical simulations and are able to apply their knowledge acquired through statistical regression of a simulated world to perform tasks and even make predictions. It's logical that a network trained purely on language has no real understanding of physical reality, but it doesn't preclude that a sense of reality in the way we humans think we have can't be gained by the same statistical analysis of the real world.

Re: Theory of Mind May Have Spontaneously Emerged in Large Language Models

#150

Post by didntreadarticl »

kilgnad wrote: post #126
ux-app wrote: post #65

Earlier quoted context omitted.

this is mind blowing to me. can anyone with more knowledge on the topic explain how ChatGPT is demonstrating this level of what seems like genuine understanding and reasoning? Like others I assumed that ChatGPT is gluing words together that commonly occur together. This is way more than that.

it indeed understands you. A lot of people are just parroting the same thing over and over again saying it's just a probabilistic word generator. No, it's not, it's more then that. Take a look at this: https://www.engraved.blog/building-a-virtual-machine-inside/ Read to the end. The beginning is trivial the ending is unequivocal: chatGPT understands you. I think a lot of people are just in denial. Because the last ye…

I think about its 4000 token length. For the brief amount of time that it absorbs and processes those 4000 tokens, is there a glimmer of a hint of sentience? Like it is microscopically sentient for very short bursts and then resets back to zero.
Post reply on HN