Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Large language models lack deep insights or a theory of mind

arxiv.org

Post reply on HN
Print view View original

31–40 of 270 posts

Re: Large language models lack deep insights or a theory of mind

#31

Post by NoMoreNicksLeft »

danenania wrote: post #23
fnordpiglet wrote: post #19

In Buddhism there’s the idea that our core self is awareness, which is silent - it doesn’t think in a perceptible way, it doesn’t feel in a visceral way, but it underpins thought and feeling, and is greatly impacted by it. A large part of meditation and “release of suffering” is learning to let your awareness lead your thinking rather than your thinking lead your awareness. To be clear, I think this is in fact a corr…

Another idea from Buddhism is that this core of awareness you're talking about is nothingness. So when you stop all thought (if such a thing is really possible), you temporarily cease to exist as an individual consciousness. "Awareness" is when the thoughts come back online and you think "whoa, I was just gone for a bit". If that's how it works, then the "soul" is more like an emergent phenomenon created by the inter…

> So when you stop all thought (if such a thing is really possible),

It's not. They don't realize it, they're merely referring to stopping your internal monologue. There are dozens of other mental processes going on in any given waking moment. Even actual top shelf cognition is going on, it just occurs in a "language of thought".

Re: Large language models lack deep insights or a theory of mind

#32

Post by munificent »

sdwr wrote: post #29
fnordpiglet wrote: post #19

In Buddhism there’s the idea that our core self is awareness, which is silent - it doesn’t think in a perceptible way, it doesn’t feel in a visceral way, but it underpins thought and feeling, and is greatly impacted by it. A large part of meditation and “release of suffering” is learning to let your awareness lead your thinking rather than your thinking lead your awareness. To be clear, I think this is in fact a corr…

Maybe the soul is social, and oriented towards others? I believe it can be constructed. If you assume that "the eyes are the window to the soul", you notice some interesting properties. 1. It is far more observable from the outside (eyes open/lidded/closed, emotion read in eyes) 2. It affects behavior in a diffuse way 3. It pays attention but does not dictate

> Maybe the soul is social

My pet theory about human consciousness is that is that consciousness is simply recursive theory of mind. Theory of mind [1] is our ability to simulate and reason about the mental states of others. It's how we predict what people are thinking and how they will react to our actions, which is critical for choosing how to act in a social environment.

But when you're thinking about what's in someone's head, one of the things might be them thinking about you. So now you're imagining your own mind from the perspective of another mind. I believe that's entirely what our sense of consciousness is. It's our social reasoning applied to ourselves.

If my pet theory is correct, it implies that the level of consciousness of any species would directly correlate to how social the species is. Solitary animals with little need for theory of mind would have no self awareness in the way that we experience it. They'd live in a zen-like perpetual auto-pilot where they do but couldn't explain why they do what they do... because they will never explain it to anyone anyway.

[1]: https://en.wikipedia.org/wiki/Theory_of_mind

Re: Large language models lack deep insights or a theory of mind

#33

Post by [deleted] »

fnordpiglet wrote: post #19

In Buddhism there’s the idea that our core self is awareness, which is silent - it doesn’t think in a perceptible way, it doesn’t feel in a visceral way, but it underpins thought and feeling, and is greatly impacted by it. A large part of meditation and “release of suffering” is learning to let your awareness lead your thinking rather than your thinking lead your awareness. To be clear, I think this is in fact a corr…

[deleted]

Re: Large language models lack deep insights or a theory of mind

#34

Post by dimal »

fnordpiglet wrote: post #19

In Buddhism there’s the idea that our core self is awareness, which is silent - it doesn’t think in a perceptible way, it doesn’t feel in a visceral way, but it underpins thought and feeling, and is greatly impacted by it. A large part of meditation and “release of suffering” is learning to let your awareness lead your thinking rather than your thinking lead your awareness. To be clear, I think this is in fact a corr…

I’ve been thinking along similar lines. It’s like with LLMs, they’ve created the part of the mind that is endlessly chattering, generating stories, sometimes true, sometimes false, but there’s no awareness or consciousness that ever steps back and can see thoughts as thoughts. And I don’t see how awareness or consciousness would arise from just more of the same (bigger models). It seems to be a fundamentally different part of the mind. I wonder if AGI is possible without this. AGI under some definition (good enough to replace most humans) may be possible. But it wouldn’t be aware. And without awareness, I don’t see how it could be aligned. It may appear to be aligned but then eventually it would probably get caught in a delusional feedback loop that it has no capacity to escape, because it can’t be aware of its own delusion.

Re: Large language models lack deep insights or a theory of mind

#35

Post by vjerancrnjak »

jacobsimon wrote: post #24
fnordpiglet wrote: post #19

In Buddhism there’s the idea that our core self is awareness, which is silent - it doesn’t think in a perceptible way, it doesn’t feel in a visceral way, but it underpins thought and feeling, and is greatly impacted by it. A large part of meditation and “release of suffering” is learning to let your awareness lead your thinking rather than your thinking lead your awareness. To be clear, I think this is in fact a corr…

This is a profound question but I also wonder if this non-thinking “awareness” you’re referring to is largely defined by quieting the thinking mind and listening to the senses more directly. A lot of meditation is about tuning out thoughts and focusing on proprioception like breathing, the feelings of the body, etc.

Fundamentally, this "awareness" isn't defined by quieting the thinking. It is a description of fundamental reality. No individual should be able to experience it, and the "glimpses" are just forms of brain dysfunction.

Meditation techniques that focus on breath or the body are an attempt to make you do the breathing/sensing consciously. If you film yourself and later look at what you did, you'll notice you aren't breathing well when you're breathing consciously, so you're probably depriving yourself of oxygen, lowering blood concentration in certain brain regions and you hope it will be the brain region associated with conceptualizing, language etc.

You can do the same with sleep. You can try to consciously fall asleep, and just like breathing, you will have a hard time because there's a reason why falling asleep is not conscious (or in other words it does not go through the regions of the brain that conceptualize). You can experience the balance center shutting down (feels like falling or turning) and you can go even deeper and feel the fear of the "ego" dying (temporarily). What remains is definitely much different than waking or dreaming state. But it is still not that "awareness/nothingness".

Re: Large language models lack deep insights or a theory of mind

#36

Post by uoaei »

tinco wrote: post #30

I think that if they would, that would be very surprising and indicative of a lot of wastefulness inside the model architecture. All these tests are simple single prompt experiments, so the LLM's get no chance to reason about their responses. They're just system 1 thinking, the equivalent of putting a gun to someone's head and asking them to solve a large division in 2 seconds. I bet a lot of these experiments would…

> They're just system 1 thinking, the equivalent of putting a gun to someone's head and asking them to solve a large division in 2 seconds.

No, it's the equivalent of putting a gun to someone's head and asking them "what are my intentions?" Which is readily available to any being with a theory of mind.

Re: Large language models lack deep insights or a theory of mind

#37

Post by crdrost »

munificent wrote: post #32
sdwr wrote: post #29

Earlier quoted context omitted.

Maybe the soul is social, and oriented towards others? I believe it can be constructed. If you assume that "the eyes are the window to the soul", you notice some interesting properties. 1. It is far more observable from the outside (eyes open/lidded/closed, emotion read in eyes) 2. It affects behavior in a diffuse way 3. It pays attention but does not dictate

> Maybe the soul is social My pet theory about human consciousness is that is that consciousness is simply recursive theory of mind. Theory of mind [1] is our ability to simulate and reason about the mental states of others. It's how we predict what people are thinking and how they will react to our actions, which is critical for choosing how to act in a social environment. But when you're thinking about what's in so…

Theory of mind is interesting but one wouldn't want to hinge consciousness upon it.

That direction would likely contain weird outcomes if the science progressed, something like "Dogs are barely-conscious due to their pack structure, they have a couple levels of recursive theory of mind but they can't sustain it as deep as we can. But cats didn't have that pack structure, they're not conscious at all." Or, "this person has such severe autism that he cannot fundamentally understand others' minds or what others interpret his mind to be, so we've downgraded his classification to unconscious. He'll talk your ear off about the various cars produced in a golden age between 1972 and 1984, but because he doesn't really know what it means for you to be listening we regard it as sleep-talking."

It also just kind of doesn't sound right. "What happens when we go to sleep? Well, we stop thinking about what others think we think, and we simply accept what they think about us." That doesn't sound like any sleep I experience -- it might describe some of my dreams, but of course dreams are anomalous conscious experiences that happen during sleep so that also misses the mark.

Re: Large language models lack deep insights or a theory of mind

#38

Post by famouswaffles »

uoaei wrote: post #36
tinco wrote: post #30

I think that if they would, that would be very surprising and indicative of a lot of wastefulness inside the model architecture. All these tests are simple single prompt experiments, so the LLM's get no chance to reason about their responses. They're just system 1 thinking, the equivalent of putting a gun to someone's head and asking them to solve a large division in 2 seconds. I bet a lot of these experiments would…

> They're just system 1 thinking, the equivalent of putting a gun to someone's head and asking them to solve a large division in 2 seconds. No, it's the equivalent of putting a gun to someone's head and asking them "what are my intentions?" Which is readily available to any being with a theory of mind.

LLMs don't fail those kind of tasks though. 4 is very good at keeping track of who knows what and why in a story. You can test this yourself.

Re: Large language models lack deep insights or a theory of mind

#39

Post by marcosdumay »

hiddencost wrote: post #7

Another paper in a long series that confuses "our tests against currently available LLMs tuned for specific tasks found that they didn't perform well on our task" with "LLMs are architecturally unsuitable for our task".

Our tests against current cars found that they didn't perform well on transatlantic flights... But who knows what the future holds? Maybe we should test them again next year.

LLM names an specific product, aimed at solving an specific problem.

Re: Large language models lack deep insights or a theory of mind

#40

Post by rf15 »

tinco wrote: post #30

I think that if they would, that would be very surprising and indicative of a lot of wastefulness inside the model architecture. All these tests are simple single prompt experiments, so the LLM's get no chance to reason about their responses. They're just system 1 thinking, the equivalent of putting a gun to someone's head and asking them to solve a large division in 2 seconds. I bet a lot of these experiments would…

they can't reason though, sadly - the premise does not hold.
Post reply on HN