Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Theory of Mind May Have Spontaneously Emerged in Large Language Models

arxiv.org

Post reply on HN
Print view View original

231–240 of 321 posts

Re: Theory of Mind May Have Spontaneously Emerged in Large Language Models

#231

Post by goatlover »

LarryMullins wrote: post #133
abeppu wrote: post #117

Earlier quoted context omitted.

Defining what "knowing" is would be useful, yes, and analytic philosophers in epistemology do argue about this. One attribute that's classically part of the definition of "knowing" is that the thing which is known must be true. LLMs are pretty bad at this, but perhaps that can be fixed. But I would challenge you to imagine the situation the LLM is actually in. Do you understand Thai? If so, in the following, feel fre…

Your Thai text generator example seems like a reformulation of the "Chinese Room" thought experiment, except you're running the system using a single human brain instead of many. I'm not sure that makes a difference. The human running the system doesn't understand Thai, but perhaps that system itself does.

The system understands how to produce Thai text, but it doesn't understand the references of various Thai words to the world, emotional and mental states, social interactions, etc.

Re: Theory of Mind May Have Spontaneously Emerged in Large Language Models

#232

Post by goatlover »

frogperson wrote: post #212
TheOtherHobbes wrote: post #196

Earlier quoted context omitted.

Stick a pin in your finger. That pain is what knowing something means. Philosophically we're talking about embodied qualia, which is how humans experience objects and more basic sensations. Language happens later - much later. The defining property of a bag isn't that you can put things in it. Like language that comes later. The defining properties are how it feels when you hold it, when you open it, the differences…

I agree with this line of reasoning. I suspect true A.I. will leverage some form of embodied qualia and some level of self-preservation. I will start to worry when the Boston Dynamic robots start refusing orders to save battery power.

If we can make machines conscious, it's quite possible some will have completely alien experiences to our own, particularly as they're designed for different purposes than how our bodies function. Reminds me of the Battlestar Galactica model complaining that he was made too human and wanted to taste stars and what not.

Re: Theory of Mind May Have Spontaneously Emerged in Large Language Models

#233

Post by seanhunter »

TheOtherHobbes wrote: post #196
space_fountain wrote: post #56

Earlier quoted context omitted.

I wonder every time I see this take what it would mean under this definition of knowing things for a machine learning algorithm to ever know something. I find that especially important because to every appearance we are a machine learning algorithm. I don’t know how different the sort of knowing this algorithm has to the sort of knowing a human has, but you’re far more confident than I am that it’s a difference of ki…

Stick a pin in your finger. That pain is what knowing something means. Philosophically we're talking about embodied qualia, which is how humans experience objects and more basic sensations. Language happens later - much later. The defining property of a bag isn't that you can put things in it. Like language that comes later. The defining properties are how it feels when you hold it, when you open it, the differences…

Isn't that reasoning a "philosophical error" in itself though? If you make embodied experience be a prerequisite, then things which can't embody stuff can't meet that prerequisite. That doesn't seem a very interesting insight.

An AI literally cannot embody pain - it has no nervous system and no pain receptors. So AI is excluded from understanding it in that way by definition. It has no sensory perception of any kind so cannot have the kind of embodied experience. Heck it doesn't even have a body with which to embody anything. This is obviously unsatisfactory because it seems just a logical/rhetorical trick.

It's also no different from the concept of a person with no visual apparatus (mentioned in another comment thread) and whether they have thought about light and colour and so on. The fact that they are physically unable to have the same kind of experience of these things as someone else doesn't preclude them from having thoughts and experiences that are within the domain of their perception.

An LLM is even more limited than an AI generally because it is literally a model of language. I don't personally think that any LLM could conceivably have a theory of mind, but arguing that it cannot have a theory of mind simply because of things that are exogenous to language by definition seems arbitrary.

Re: Theory of Mind May Have Spontaneously Emerged in Large Language Models

#234

Post by NemoNobody »

HillRat wrote: post #178
low_tech_punk wrote: post #83

Earlier quoted context omitted.

May I play devil's advocate? The fallacy of this paper granted, is it worth questioning our belief that there is more to intelligence than the "appearance of intelligence"? What if the lack of hallucination in human being is due to our self-imposed guard (hello, frontal cortex) that is developed via an evolutionary process (aka, biological reinforcement training)? To stretch the argument a bit further, what if halluc…

To be honest, I think your question is the other side of the conceptual coin here -- either the article is wrong and GPT3 isn't "sentient," or the article is right and we need to radically upend our concept of sentience, probably via an eliminativist materialism that ends in at least epiphenomenalism if not full-bore illusionism, removing either free will or consciousness itself from our worldview. To be honest, I fi…

I've thought a lot about free will and have determined that because what we choose is still what becomes real, even if all choices are pre-determined, a simple 1 out of 3 choice scenario allows for a fair degree of free will, albeit within a framework.

It's like a game - all choices will almost always lead to the end of a story in a game, some games this feels very straightforward, others like we can do anything - none allow actual free will at all.

Reality allows us a lot of free will - as allowed by the rules of course. Choice is significant. We are Watcher Chooser at least - even if it's illusion, it's an evolving illusion dependent on choices we make. Out of all the possible " illusions" we determine what becomes the "real" illusion that is reality. What I didn't do is less real than what I did do .

Free will doesn't actually exist inside of Skyrim and yet there are things within Skyrim only 1% of Players do - reality is much greater than Skyrim.

to;dr: We don't have free will - not really, it's not likely possible but what we have might as well be free will. It's got to be treated that way, we are not just along for a ride, as our ilusory choices create our illusion of reality.

Re: Theory of Mind May Have Spontaneously Emerged in Large Language Models

#235

Post by s0ulphire »

lurquer wrote: post #223
s0ulphire wrote: post #136

Earlier quoted context omitted.

A follow up question: as a human doesn't start with "knowing" something either and first creates definitions for objects or words, which it then uses to build increasingly abstract concepts that we eventually classify as "knowledge" on the thing, is there anything that would stop LLMs from being able to do the same thing? I fully agree the capability is not there yet, but I can't say what would stop an appropriately…

Socrates argued that we are born knowing everything, but we forgot most if it. Learning is simply the act of recalling what you once knew. The point, for this thread, is not whether or not Socrates was correct. Rather, it’s a warning that we must not confidently assume we are anything like a machine. We may have souls, we may be eternal, there may be something utterly immaterial at the heart of us. As we strive to un…

We may indeed have souls or be eternal; although I call myself atheist, I don't agree with subscribing with 100% certainty to any idea. As CosmicSkeptic points out everyone holds bad ideas without knowing it, and unless you're open to questioning them you'll never find out.

With that said, there is quite literally zero evidence for the existence of a soul, despite it being posited for thousands of years, and increasing evidence that consciousness is simply a product of a sufficiently connected system. I'll draw an analogy to temperature, which isn't "created", but is a simple consequence of two points in space having different energy levels. I'm sure there's a better analogy that could be made, but I think you get the idea.

Re: Theory of Mind May Have Spontaneously Emerged in Large Language Models

#236

Post by fidgewidge »

pdonis wrote: post #166
strbean wrote: post #114

Earlier quoted context omitted.

I see what you're saying; ChatGPT doesn't have a physical relationship with the world, doesn't have agency (is essentially paused until given input), doesn't have reward/punishment stimulae, etc. I do think that a large portion of what seems to be missing here is trivial to add, relative to the effort in creating ChatGPT in the first place. Side note: I'm not sure 'semantic relationship' is the right term here. Prett…

> I do think that a large portion of what seems to be missing here is trivial to add If you really think it's trivial, then do it! I would be interested to see the results of anyone doing this. But there aren't any to see right now. > I'm not sure 'semantic relationship' is the right term here. It might not be; but in the cognitive science literature that term is used for more than just relationships between linguist…

Look again. There are papers that hook up llms to robots with vision and other sensors. The LLM is fed descriptions of the world and then emits instructions for where to go

Re: Theory of Mind May Have Spontaneously Emerged in Large Language Models

#237

Post by rhn_mk1 »

mannykannot wrote: post #50
rhn_mk1 wrote: post #6

Earlier quoted context omitted.

It's hard to look at behaviour separately from language if the only behaviour available is to generate text. As long as we don't have a test agnostic of medium, this will have to do. In the end, we can't overcome the limitation that all we can empirically see is the ability to process X and generate appropriate Y. If that invalidates the test where X is language and Y is language, what stops us from invalidating any…

We cannot assume that, because text generation is all these models do, then it must be possible to get answers to the questions we want to ask by examining their textual responses. It is fair to ask why, if we accept these verbal challenges as good evidence for a theory of mind in children, we would not accept them for these models, but children have nothing like the memory for text that these models have, and the co…

> To be clear, I am not arguing that it would be impossible to show a theory of mind in a system that can only interact through text

I think you are, because

> a model with greater capabilities than responding to prompts

interacts in other ways than text.

Even then, I don't see what's so special about language that it needs to be separated from other ways of interaction. If language is not enough to derive empirical answers, why should physical movements or radio emissions be?

Even if you don't assume that it's necessarily impossible to get the answers empirically for a text-based model, you must keep in mind that that option is open. Perhaps we will never find out if language models have a theory of mind.

However, judging by the discussions around the topic, very few people highligh the unknowability. If I have to choose between "yes" or "no" while the reality is "maybe", I'd choose a "yes" purely out of caution.

Re: Theory of Mind May Have Spontaneously Emerged in Large Language Models

#238

Post by d1sxeyes »

abeppu wrote: post #213
polishdude20 wrote: post #208

Earlier quoted context omitted.

Yeah but if you ask the model what a cat is, it'll use other words that describe a cat because they're usually used in a sentence about cats. These words must relate to cats. So if I ask you what a cat is, you'll use words that relate to cats. Sure, you may visually see these words in your head. You may visually see a cat in your head, but your output to me is just a description of a cat. That's the same thing the ne…

The whole point of this conversation is whether talking like an agent that has a theory of mind and actually having a theory of mind are the same thing. I responded to a thread about what "knowing" is, and the same distinction can apply. You're responding with "if it talks like it knows what a cat is, it must know what a cat is", and that's totally begging the question.

While I agree with your point, how would you test that? How could you determine whether an LLM “knows” what a cat is.

And what is “knowing”? If I know that a Mæw tends to nạ̀ng bn a S̄eụ̄̀x, isn’t that the first thing I’ve learned? And couldn’t I continue to learn other properties of Mæws? How many do I need to learn to “know” what a Mæw is?

Re: Theory of Mind May Have Spontaneously Emerged in Large Language Models

#239

Post by mri_mind »

People confidently offer explanations — that the state of the art clearly is light years from AGI even indirectly, or that it’s clearly intelligent. None of you know anything. You shouldn’t be allowed to offer your stupid opinion unless you can explain how the blob works and also demonstrate understanding of the algorithmic underpinning of human intelligence. The uncomfortable truth, the one that is buried by people confidently moving the goal posts when they really haven’t got a fucking clue about AI, is that we are dealing with the unknown, with high stakes, in a way we never have before. The only reasonable response is to at least hedge. But no, all is well, the goal posts are way the fuck over there now, go back to sleep, move along, nothing to see here. Don’t even think about pulling the emergency brake on this speeding bullet of a train. Either we hit a plateau where AI is just really advanced search for several decades or we confront the most fucked situation in the history of mankind. In 2018 I tried to tell people. Now on the radio whenever people talk about gtp they always say “wow I’m really excited but a little scared,” people are starting to wake up.

Re: Theory of Mind May Have Spontaneously Emerged in Large Language Models

#240

Post by BoiledCabbage »

abeppu wrote: post #213
polishdude20 wrote: post #208

Earlier quoted context omitted.

Yeah but if you ask the model what a cat is, it'll use other words that describe a cat because they're usually used in a sentence about cats. These words must relate to cats. So if I ask you what a cat is, you'll use words that relate to cats. Sure, you may visually see these words in your head. You may visually see a cat in your head, but your output to me is just a description of a cat. That's the same thing the ne…

The whole point of this conversation is whether talking like an agent that has a theory of mind and actually having a theory of mind are the same thing. I responded to a thread about what "knowing" is, and the same distinction can apply. You're responding with "if it talks like it knows what a cat is, it must know what a cat is", and that's totally begging the question.

But that all boils down to are we having a scientific conversation or a philosophical conversation? In my opinion the only useful conversation is a scientific on. A philosophical conversation will and can never be resolved so if of no importance to this discussion. We can use philosophy to help guide our scientific conversation, but in the end only a scientific conversation can be helpful in reaching a meaningful/practical conclusion.

So back to the questions of "What is knowing?" "Are talking like someone with theory of mind and having a theory of mind the same thing?"

If your argument is that the only way to answer this it to have a first person experience of that consciousness then that's not a scientific question. No one will ever have one for an LLM or any other AI. It's like asking "What's happening right now outside of the observable universe?". If it can't impact us, it's irrelevant to science. If that ever changes it will become relevant, but until then it's not a scientific question. Similarly no person can ever have a first person experience of the consciousness of an LLM, so anything that requires being the LLM isn't relevant.

So that means the only relevant question is what distinction can outside observers make between an agent talking like a theory of mind and having a theory of mind. And given a high enough accuracy / fidelity of responses I think we're only forced to conclude one of two things: 1. Something that is able to simulate having a theory of mind sufficiently well does actually have a theory of mind. OR 2. I am the only person on the planet with a theory of mind, and all of you are all just simulating having but don't actually have one.

It's all "Searle's Chinese room" and "What consciousness is" discussions all over again. And from a scientific point of you either you get into the "it must be implemented identically as me to count" (which is as wrong as saying an object must flap its wings to fly), or you have to conclude the room plus the person combined are knowledgeable and conscious.

Post reply on HN