Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Large language models lack deep insights or a theory of mind

arxiv.org

Post reply on HN
Print view View original

121–130 of 270 posts

Re: Large language models lack deep insights or a theory of mind

#121

Post by elicksaur »

theptip wrote: post #117

This is a terrible eval. Do not update your beliefs on whether LLMs have Theory of Mind based on this paper. The eval is a weird, noisy visual task (picture of astronaut with “care packages”). Their results are hopelessly narrow. A better eval is to use actual scientifically tested psychology test on text (the native and strongest domain for LLMs), for example the sort of scenarios used to gauge when children develop…

Or there are enough of those examples in the training set that it can guess well. Not sure how such an example would prove anything when we know an LLM is just guessing the best words.

Nothing I’ve seen shows evidence of any sort of abstract concepts in there.

Re: Large language models lack deep insights or a theory of mind

#122

Post by astrange »

NoMoreNicksLeft wrote: post #31
danenania wrote: post #23

Earlier quoted context omitted.

Another idea from Buddhism is that this core of awareness you're talking about is nothingness. So when you stop all thought (if such a thing is really possible), you temporarily cease to exist as an individual consciousness. "Awareness" is when the thoughts come back online and you think "whoa, I was just gone for a bit". If that's how it works, then the "soul" is more like an emergent phenomenon created by the inter…

> So when you stop all thought (if such a thing is really possible), It's not. They don't realize it, they're merely referring to stopping your internal monologue. There are dozens of other mental processes going on in any given waking moment. Even actual top shelf cognition is going on, it just occurs in a "language of thought".

> It's not. They don't realize it, they're merely referring to stopping your internal monologue.

They certainly have realized that. It's one of the first things you notice doing awareness meditation; thoughts appear from nowhere even if you didn't try to think them.

Re: Large language models lack deep insights or a theory of mind

#123

Post by pixl97 »

thegiogi wrote: post #118
pixl97 wrote: post #67

Earlier quoted context omitted.

https://en.wikipedia.org/wiki/Moravec%27s_paradox While you're adding a bunch of eastern philosophy to it, we need to take a step back from 'human' intelligence and go to animal and plant intelligence to get a better idea of the massive variation in what covers thought. In animal/insects we can see that thinking is not some binary function of on or off. It is an immense range of different electrical and chemical proc…

> A good example of this at the human level is a reflex. Your hand didn't go back to your brain to ask for instructions on how to get away from the fire. Is this actually true? I thought it just involved a different part of the brain. Is there actually no brain involvement? Sure it does not need your awareness or decision making, but no brain? I find that hard to believe.

https://en.wikipedia.org/wiki/Reflex_arc

Simple answer: No, it does not go to the brain

Detailed answer: We are complex as all hell.

>A reflex arc is a neural pathway that controls a reflex. In vertebrates, most sensory neurons do not pass directly into the brain, but synapse in the spinal cord. This allows for faster reflex actions to occur by activating spinal motor neurons without the delay of routing signals through the brain. The brain will receive the input while the reflex is being carried out and the analysis of the signal takes place after the reflex action.

Re: Large language models lack deep insights or a theory of mind

#124

Post by Merrill »

fnordpiglet wrote: post #19

In Buddhism there’s the idea that our core self is awareness, which is silent - it doesn’t think in a perceptible way, it doesn’t feel in a visceral way, but it underpins thought and feeling, and is greatly impacted by it. A large part of meditation and “release of suffering” is learning to let your awareness lead your thinking rather than your thinking lead your awareness. To be clear, I think this is in fact a corr…

Decision making seems fundamental to intelligence, is done by animals and humans, and can be done without the use of language or logic. This is the case when someone "decided without thinking".

Decision making requires imagination or the ability to envision alternative future states that may result from various choices.

Imagination is the start of abstract thinking. Consciousness results from the individual thinking abstractly about itself and how it interacts with the world.

Re: Large language models lack deep insights or a theory of mind

#125

Post by bongodongobob »

uoaei wrote: post #36
tinco wrote: post #30

I think that if they would, that would be very surprising and indicative of a lot of wastefulness inside the model architecture. All these tests are simple single prompt experiments, so the LLM's get no chance to reason about their responses. They're just system 1 thinking, the equivalent of putting a gun to someone's head and asking them to solve a large division in 2 seconds. I bet a lot of these experiments would…

> They're just system 1 thinking, the equivalent of putting a gun to someone's head and asking them to solve a large division in 2 seconds. No, it's the equivalent of putting a gun to someone's head and asking them "what are my intentions?" Which is readily available to any being with a theory of mind.

What? My answer would be "I have no idea!"

Are they faking it? Will they murder me? Are they just trying to scare me?

I don't understand the point you're trying to make.

Re: Large language models lack deep insights or a theory of mind

#126

Post by ctoth »

sdwr wrote: post #29
fnordpiglet wrote: post #19

In Buddhism there’s the idea that our core self is awareness, which is silent - it doesn’t think in a perceptible way, it doesn’t feel in a visceral way, but it underpins thought and feeling, and is greatly impacted by it. A large part of meditation and “release of suffering” is learning to let your awareness lead your thinking rather than your thinking lead your awareness. To be clear, I think this is in fact a corr…

Maybe the soul is social, and oriented towards others? I believe it can be constructed. If you assume that "the eyes are the window to the soul", you notice some interesting properties. 1. It is far more observable from the outside (eyes open/lidded/closed, emotion read in eyes) 2. It affects behavior in a diffuse way 3. It pays attention but does not dictate

> If you assume that "the eyes are the window to the soul", you notice some interesting properties.

And people say LLM output is nonsense.

I'm blind with glass eyeballs. Does this mean my soul is easier to access than yours? Or is it harder because there's something specific about the eyeball that makes it the window?

Re: Large language models lack deep insights or a theory of mind

#127

Post by tivert »

poulsbohemian wrote: post #74
joduplessis wrote: post #66

For me, the entire AGI conversation is hyperbolic / hype. How can we infer intelligence to something when we, ourselves, have such a poor (none) grasp of what makes us conscience? I'm associating intelligence with consciousness - because it seems correlated. Are we really ready to associate "AGI" with solving math problems ("new Q algo.")? That seems incredibly naive & reinforces my opinion that LLM's are much more l…

Completely agree, and while we are at it... look I'm just a guy, not an expert, but I can't understand why there's so much focus on AGI. It feels like there are so many niche areas where we could apply some kind of analytical augmentation and by solving problems in the small, might learn something that would help figure the larger question of intelligence. I don't need the AI to replace everything I do, I need it to…

> ...but I can't understand why there's so much focus on AGI.

Lots of software engineers have spent their lives reading sci-fi that features AGI, and they're excited by/lost in that fantasy.

It's interesting to see that in people who often view themselves as hyper-rational.

Re: Large language models lack deep insights or a theory of mind

#128

Post by twobitshifter »

tinco wrote: post #30

I think that if they would, that would be very surprising and indicative of a lot of wastefulness inside the model architecture. All these tests are simple single prompt experiments, so the LLM's get no chance to reason about their responses. They're just system 1 thinking, the equivalent of putting a gun to someone's head and asking them to solve a large division in 2 seconds. I bet a lot of these experiments would…

There are plenty of models that use introspection and check answers, that’s the idea behind let’s think about it step by step.

Re: Large language models lack deep insights or a theory of mind

#129

Post by NoMoreNicksLeft »

astrange wrote: post #122
NoMoreNicksLeft wrote: post #31

Earlier quoted context omitted.

> So when you stop all thought (if such a thing is really possible), It's not. They don't realize it, they're merely referring to stopping your internal monologue. There are dozens of other mental processes going on in any given waking moment. Even actual top shelf cognition is going on, it just occurs in a "language of thought".

> It's not. They don't realize it, they're merely referring to stopping your internal monologue. They certainly have realized that. It's one of the first things you notice doing awareness meditation; thoughts appear from nowhere even if you didn't try to think them.

[flagged]

Re: Large language models lack deep insights or a theory of mind

#130

Post by tinco »

twobitshifter wrote: post #128
tinco wrote: post #30

I think that if they would, that would be very surprising and indicative of a lot of wastefulness inside the model architecture. All these tests are simple single prompt experiments, so the LLM's get no chance to reason about their responses. They're just system 1 thinking, the equivalent of putting a gun to someone's head and asking them to solve a large division in 2 seconds. I bet a lot of these experiments would…

There are plenty of models that use introspection and check answers, that’s the idea behind let’s think about it step by step.

I feel the "let's think about it step by step" is a bit of a hack. To circumvent the fact that there's no external loop you use the fact that it gets re-run on every token so you can store a bit of state in the tokens that it's already generated.

Or am I misunderstanding something about that technique?

Post reply on HN