Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Theory of Mind May Have Spontaneously Emerged in Large Language Models

arxiv.org

Post reply on HN
Print view View original

281–290 of 321 posts

Re: Theory of Mind May Have Spontaneously Emerged in Large Language Models

#281

Post by Leo_Germond »

space_fountain wrote: post #144
abeppu wrote: post #117

Earlier quoted context omitted.

Defining what "knowing" is would be useful, yes, and analytic philosophers in epistemology do argue about this. One attribute that's classically part of the definition of "knowing" is that the thing which is known must be true. LLMs are pretty bad at this, but perhaps that can be fixed. But I would challenge you to imagine the situation the LLM is actually in. Do you understand Thai? If so, in the following, feel fre…

Let me respond with an analogy of my own. Imagine you are a scientist on an alien world. The aliens primary experience the world through magnetic fields. They live deep in the atmosphere of a hot Jupiter like planet and rarely touch anything and have no eyes. Still they are intelligent beings and so quickly they are able to establish communication with you. A computer translates and you both have to become a bit more…

I agree with you. I really enjoy this idea that understanding, conscience, are emerging properties of a system, which does not need to limit itself to any scope to happen. In that light the current approach most people take on this, taking an arbitary selection of parts to see if it exhibits those properties, is not right at all.

A ion channel does not have even a tiny spec of conscience, no matter how you organize them, but our brain does indeed need those to be conscient (and incidentally it relies on a whole lot more "stupid" parts than that: try being conscient without oxygen, or glucose).

I would go as far as making conscience an emergent property of interaction with the environment: what does it mean to be conscious if nothing is there to confirm that you are indeed of a singular conscience? Is it possible to understand the concept of self if you have no concept of other beings?

Re: Theory of Mind May Have Spontaneously Emerged in Large Language Models

#282

Post by Tv9m »

toss1 wrote: post #277
Tv9m wrote: post #273

Earlier quoted context omitted.

"statistical re-mixer" doesn't describe these systems very well. I see this complaint a lot, that supposedly DL models can only manipulate existing content without creating anything of their own. That's just false, unless your standard for originality is so high that humans can't reach it either. These models that have hundreds of billions of "synapses", it's not very shocking to me that they can learn the abstract f…

Yes, I understand that it can appear to synthesize something new, and no, I'm not looking for some inner experience. I'm looking for it to show an ability to wield not only a set of strings (with language associations), but something actually like the platonic ideals - objects, with properties and relations. A few errors show quickly there is no such concept being weilded. >> I saw a fine example of this failure the…

> A few errors show quickly there is no such concept being weilded

I would have given similar examples to show that ChatGPT makes the same kinds of mistakes that humans do. The first one is good, because ChatGPT can solve it easily when you present it as a riddle rather than being a genuine question. Humans use context and framing in the same way; I'm sure you've heard of the Wason selection task: https://en.wikipedia.org/wiki/Wason_selection_task

When posed as a logic problem, few people can solve it. But when framed in social terms, it becomes apparently simple. This shows how humans aren't using fundamental abstract concepts here, but rather heuristics and contextual information.

The second example you give is even better. It's designed to trick the reader into thinking of the number 30 by putting the phrase "half my age" before the number 60. It's using context as obfuscation. In this case, showing ChatGPT an analogous problem with different wording lets it see how to solve the first problem. You might even say it's able to notice the fundamental abstract concepts that both problems share.

The third problem is also a good example, but for the wrong reason: I can't solve it either. If you had spoken it to me slowly five times in a row, I doubt I could have given the right answer. If you gave me a pencil and paper, I could work through the steps one by one in a mechanical way... but solving it mentally? Impossible for me.

> It is run through a grammatical filter/generator at the end so it's usually grammatical, but no sort of truth filter (or ethical filter for that matter either).

I kind of thought it did get censored by a sort of "ethical filter" (very poorly, obviously), and also I wasn't aware of it needing grammatical assistance. Do you remember where you heard this?

Here's my chat with it, if you're interested: https://pastebin.com/raw/hQQ8bpsB

But comparing 1 human to 1 GPT is mistaken to begin with. It's like comparing 1 human with 1 Wernicke's area or 1 angular gyrus. If you had 100 different ChatGPTs, each optimized for a different task and able to communicate with each other, then you'd have something more similar to the human brain.

Re: Theory of Mind May Have Spontaneously Emerged in Large Language Models

#283

Post by toss1 »

Tv9m wrote: post #282
toss1 wrote: post #277

Earlier quoted context omitted.

Yes, I understand that it can appear to synthesize something new, and no, I'm not looking for some inner experience. I'm looking for it to show an ability to wield not only a set of strings (with language associations), but something actually like the platonic ideals - objects, with properties and relations. A few errors show quickly there is no such concept being weilded. >> I saw a fine example of this failure the…

> A few errors show quickly there is no such concept being weilded I would have given similar examples to show that ChatGPT makes the same kinds of mistakes that humans do. The first one is good, because ChatGPT can solve it easily when you present it as a riddle rather than being a genuine question. Humans use context and framing in the same way; I'm sure you've heard of the Wason selection task: https://en.wikipedi…

>>trick the reader into thinking of the number 30 by putting the phrase "half my age" before the number 60

Yet it is exactly the process of conceptualizing "half" and applying it to "at six years old" instead of "of 60" that is the key to solving it.

These things aren't abstracting out any concepts, they only operate at the level of "being fooled by" semantics. The fact that humans sometimes fail this way gives us little more than [sure a human not really thinking about the problem may offer a bad solution based only on the superficial semantic]. ChatGPT reliably gives us the error based on the superficial semantics.

>>If you had 100 different ChatGPTs, each optimized for a different task and able to communicate with each other, then you'd have something more similar to the human brain.

YES, that is the route we need to go to get towards actual intelligent processing. Taking 100 of these tuned for different areas, and abstracting out the various entities and relationships.

Kind of like the visual cortex model that extracts out edges, motion, etc., and then higher areas in the visual cortex, combined with other areas of the brain allow us to sort out faces, bodies, objects passing behind each other, the fact that Alice entered the room before Bob, and that this is because Bob was polite...

They also mut know when they are making errors, and NONE of these systems comes even close — they happily spout their bullshirt as confidently as any fact.

I gave a deposition in a legal case where the deposing attnys used an "AI" transcription system. Where a human would ask if anything was unclear, and always at the next break get proper spellings of all names, addresses, etc., this thing just went merrily along inserting whatever seemed most likely in the slot. Entire meanings of sentences were reversed (e.g., "you have a problem" edited to "I have a problem"), names were substituted (e.g., the common "Jack Kennedy" replaced "John Kemeny").

There's the Stable Diffusion error with a bikini-clad girl sitting on a boat, where we see her head and torso facing us, as well as her butt cheeks, with thighs & knees facing away. It looks great for about 1.5 sec. until you see the error that NO human would make (except as a joke).

The mere fact that some humans can sometimes make superficial errors which resemble the superficial errors these "AI" things frequently and consistently make does not mean that because humans often have a deeper mode, these "AI"s must also have a deeper understanding.

It means either nothing, i.e., insufficient data to decide, or that these are indeed different, because there is zero evidence of deeper understanding in a ChatGPT or Stable Diffusion.

EDIT: Typos

Re: Theory of Mind May Have Spontaneously Emerged in Large Language Models

#284

Post by skjoldr »

abeppu wrote: post #117
space_fountain wrote: post #56

Earlier quoted context omitted.

I wonder every time I see this take what it would mean under this definition of knowing things for a machine learning algorithm to ever know something. I find that especially important because to every appearance we are a machine learning algorithm. I don’t know how different the sort of knowing this algorithm has to the sort of knowing a human has, but you’re far more confident than I am that it’s a difference of ki…

Defining what "knowing" is would be useful, yes, and analytic philosophers in epistemology do argue about this. One attribute that's classically part of the definition of "knowing" is that the thing which is known must be true. LLMs are pretty bad at this, but perhaps that can be fixed. But I would challenge you to imagine the situation the LLM is actually in. Do you understand Thai? If so, in the following, feel fre…

>But if we want AIs that 'know' the same things we know, then we have to build them to perceive in a multi-modal way, and interact with stuff in the world, rather than just self-supervising on piles of internet data.

In other words, a LLM that is tied to a GAN that generates images, produces an system that can both describe to you what is a cat verbally and show you a picture of a cat. Does it, then, know what "a cat" is?

Edit: Furthermore, if you then tie this AI to a CV model with a camera which you can point at a cat and it will tell you that it is, indeed, a cat, and then it will also be able to produce a verbal description of a cat as well as show you an abstract picture of a cat or pick cats out of a random set of images, does this whole system know what "a cat" is?

If you, then, make a robot with a camera and hands, attach to the system a more complex CV model that can see in 3D, ask the LLM to produce you a set of code instructions that can be parametrized to produce a motion that would pet the cat, input those instructions into the robot to make it pet a specific cat that has the specific 3D point cloud (I guess that's currently difficult but solveable), and the system will then indeed pet the cat, would it then know what "a cat" is?..

The underlying LLM is still the same in all these scenarios. Where is the boundary?

Re: Theory of Mind May Have Spontaneously Emerged in Large Language Models

#285

Post by didntreadarticl »

didntreadarticl wrote: post #134
swatcoder wrote: post #119

Earlier quoted context omitted.

Let me know when you find out. The interesting and open question to me is what the limitations are of a language model at the center of that experience. How much of a a relationship with reality can be captured by language at all, and specifically with the specific sort of statistical models of language that we're exploring now? For some of us, the intuitive answer is not all that much and for others it seems to be a…

Ideally in that scenario you'd have a model that unified vision, language and an understanding of 'doing things' and manipulating objects. so it wouldnt just be an LLM, it would be a language-vision-doingthings model. There's no reason why we cant build one.

Come to think of it, thats kindof what Tesla are building

Re: Theory of Mind May Have Spontaneously Emerged in Large Language Models

#286

Post by ves »

space_fountain wrote: post #56
lsy wrote: post #33

This highlights one of the types of muddled thinking around LLMs. These tasks are used to test theory of mind because for people, language is a reliable representation of what type of thoughts are going on in the person's mind. In the case of an LLM the language generated doesn't have the same relationship to reality as it does for a person. What is being demonstrated in the article is that given billions of tokens o…

I wonder every time I see this take what it would mean under this definition of knowing things for a machine learning algorithm to ever know something. I find that especially important because to every appearance we are a machine learning algorithm. I don’t know how different the sort of knowing this algorithm has to the sort of knowing a human has, but you’re far more confident than I am that it’s a difference of ki…

> I find that especially important because to every appearance we are a machine learning algorithm.

Speak for yourself.

Re: Theory of Mind May Have Spontaneously Emerged in Large Language Models

#288

Post by markmaglana »

skjoldr wrote: post #284
abeppu wrote: post #117

Earlier quoted context omitted.

Defining what "knowing" is would be useful, yes, and analytic philosophers in epistemology do argue about this. One attribute that's classically part of the definition of "knowing" is that the thing which is known must be true. LLMs are pretty bad at this, but perhaps that can be fixed. But I would challenge you to imagine the situation the LLM is actually in. Do you understand Thai? If so, in the following, feel fre…

>But if we want AIs that 'know' the same things we know, then we have to build them to perceive in a multi-modal way, and interact with stuff in the world, rather than just self-supervising on piles of internet data. In other words, a LLM that is tied to a GAN that generates images, produces an system that can both describe to you what is a cat verbally and show you a picture of a cat. Does it, then, know what "a cat…

At some point, when multiple components (including the LLM) have been connected to form a system that exhibits "knowing" (the way humans do), wouldn't the "intelligence" be distributed across the entire system rather than attributed primarily to the LLM?

In other words, the LLM wouldn't be the equivalent of the human brain. Instead, it would just be equivalent to that part of the human brain that processes language.

Re: Theory of Mind May Have Spontaneously Emerged in Large Language Models

#289

Post by CamperBob2 »

pegasus wrote: post #241
d1sxeyes wrote: post #238

Earlier quoted context omitted.

While I agree with your point, how would you test that? How could you determine whether an LLM “knows” what a cat is. And what is “knowing”? If I know that a Mæw tends to nạ̀ng bn a S̄eụ̄̀x, isn’t that the first thing I’ve learned? And couldn’t I continue to learn other properties of Mæws? How many do I need to learn to “know” what a Mæw is?

Like GP said, the LLM has no chance at knowing what a cat is, regardless of how much data it ingests, because a cat is not made of data. It's not like you're getting closer and closer to knowing what a "Mæw" is. You were at the same remote distance all the time. This is called the "grounding problem" in AI. As for how you would test it, I think one-shot learning would get one closer to proving understanding.

because a cat is not made of data.

Your perception of what a cat is, however, is most certainly made of nothing but data, encoded as chemical relationships at the neuronal level. And your perception is all there is, as far as you're concerned. The cat is just another shadow on Plato's cave wall.

Arguably you "know" something when you can recognize it outside its usual context, classify it in terms of its relationships with other objects, and anticipate its behavior. To the extent that's true, ML models have been there for quite a while now.

What else besides recognition, classification, and prediction based on either experience or inference is needed for "knowledge?" Doesn't everything human minds can do boil down to pattern recognition and curve fitting at the end of the day?

Re: Theory of Mind May Have Spontaneously Emerged in Large Language Models

#290

Post by markmaglana »

BoiledCabbage wrote: post #240
abeppu wrote: post #213

Earlier quoted context omitted.

The whole point of this conversation is whether talking like an agent that has a theory of mind and actually having a theory of mind are the same thing. I responded to a thread about what "knowing" is, and the same distinction can apply. You're responding with "if it talks like it knows what a cat is, it must know what a cat is", and that's totally begging the question.

But that all boils down to are we having a scientific conversation or a philosophical conversation? In my opinion the only useful conversation is a scientific on. A philosophical conversation will and can never be resolved so if of no importance to this discussion. We can use philosophy to help guide our scientific conversation, but in the end only a scientific conversation can be helpful in reaching a meaningful/pra…

> Something that is able to simulate having a theory of mind sufficiently well does actually have a theory of mind.

That presupposes that our existing tools for detecting the presence of ToM are 100% accurate. Might it be possible that they are imprecise and it’s only now that their critical flaws have been exposed?

Post reply on HN