Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Large language models lack deep insights or a theory of mind

arxiv.org

Post reply on HN
Print view View original

261–270 of 270 posts

Re: Large language models lack deep insights or a theory of mind

#261

Post by mcguire »

Barrin92 wrote: post #103
mcguire wrote: post #77

Earlier quoted context omitted.

I was right there with you until you mentioned Searle. :-) The Chinese room argument is bad in that it hides an assumption of mind/body dualism. If you believe that humans have "souls" and other things do not, then you have a qualitative difference between a human or a machine. On the other hand, if you are a materialist then you are faced with the problem that humans don't have much understanding of semantics either…

Searle's one of the most die-hard materialist philosophers of mind around. It's his materialism that leads him to make his argument. Computers and human brains are both made out of atoms but that doesn't mean they're not qualitatively different. By that logic I"d be no different from a tree. There's qualitative differences between computers and human brains. Our cognition is biochemical and horribly slow , just by vi…

The Chinese Room gag asserts that, although the room behaves in every way like it is an intelligent Chinese speaker, we can see inside the room and determine that there is nothing there that intelligently understands Chinese.

Searle seems to see a distinction between the "substrates" (which means an LLM cannot be intelligent; it's running on regular computer hardware and there's nothing to be found which "understands"), but unless someone can point out exactly what part of the substrate is intelligent, I am going to continue to suggest that his substrate difference is exactly identical to having a soul or not having a soul.

I, as a materialist, assert that, if you dug through my noggin down to the level of subatomic particles you will never find anything that is intelligent or which understands. (Quantum mechanics does not help here, by the way. It just replaces a (rather naive) determinism with randomness---see Bell's Inequality.) There is no magic going on there. You and your tree are both doing nothing but chemistry and physics (you share something like 50% of your genes, by the way). And that means that a computer could, in principle, behave as intelligently as I do. That's "substrate independence".

Now, whether a given system does do so or not is another question entirely. :-)

Re: Large language models lack deep insights or a theory of mind

#262

Post by omginternets »

FrustratedMonky wrote: post #260
omginternets wrote: post #255

Earlier quoted context omitted.

If you want to equate "emergence" with "making something up", then fine, I guess. I'm just not sure what what you can possibly conclude from that equivalence.

That's why I'm not big on using word "hallucination" for this. It is really our subjective experience. Our 'view' of the outside world is constructed in the brain from the senses, like what is 'hot', or 'pressure', or 'blue', these don't exist in the world, we build them internally. Some people call it a 'hallucination', others 'subjective reality', or 'mental map'. The problem is, our brain is constructing reality,…

>The problem is, our brain is constructing reality, it can 'make things up', this has been shown in countless studies. And now that we have AI doing it too, it seems like easy association to make.

I'm with you until this pair of sentences because I believe you are confusing ontological subjectivity (which is fine, for our purposes) with epistemic subjectivity (which isn't).

Hallucination is an ontologically subjective phenomenon, requiring an experiencer to experience it. "Making shit up" similarly implies an "intentional stance" (Dennett), wherein the AI agent is constructing a world model as it interacts with the world. That isn't required to arrive at a "stochastic parrot" that spouts nonsense.

"Generating nonsense" is closer to what the AI is doing. It's generating text that we are unable to interpret, not revealing its errors of reasoning through its speech. It's not reasoning; it's generating tokens.

tl;dr: Ontological vs epistemic subjectivity. There's no reason to affirm AI is hallucinating because there's no reason to affirm its experiencing anything.

(Please forgive my multiple edits; it's a clumsy-words kind of day.)

Re: Large language models lack deep insights or a theory of mind

#263

Post by munificent »

crdrost wrote: post #239
munificent wrote: post #145

Earlier quoted context omitted.

Ah, that looks perfect! Thank you! I knew other people smarter than me must have stumbled onto this idea as well.

FWIW if you're looking for book recommendations this is similar to the thesis of Gödel, Escher, Bach: An Eternal Golden Braid.

Yes, I read GEB many years ago and deeply enjoyed it.

Re: Large language models lack deep insights or a theory of mind

#264

Post by creer »

neuronerdgirl wrote: post #254
creer wrote: post #242

Earlier quoted context omitted.

At the same time, if LLMs are based on all or enough human writings then don't they necessarily contain a theory of mind? A rather general, smoothed out and still neurotic one probably. But still just like an LLM can't be expected to have a specific knowledge of hydraulics, it also has read more hydraulics than even experts might be expected to. That's the entire issue about it, right? This issue of "is most of our m…

That's not what theory of mind is. Theory of mind is the understanding - not academic understanding, but functional, meaningful understanding that guides your behavior - that the other people (AIs?) around you have their own separate thoughts and feelings from your own. This is what leads to the stark behavioral differences as a child develops from ~3-6 years of age. Imagine a child is playing with their toys and put…

I think this branch of the discussion was granting an "of course, for the moment" on this because of the lack of loop / internal conversation.

After that, the entire field of AI is about making bits and register transfers - a written form of knowledge - into something indistinguishable from the wet version. Making a priori and irreconcilably different "academic understanding" and "functional, meaningful understanding" is going to be a problem.

Re: Large language models lack deep insights or a theory of mind

#265

Post by TeMPOraL »

AnimalMuppet wrote: post #252
TeMPOraL wrote: post #247

Earlier quoted context omitted.

No, that just means you need a bigger engine . (And, perhaps, an inertial dampener, to survive the gees.) Think of it this way: the more thrust you get, the farther you'll go before control problems start to manifest. Keep pushing, and eventually the rest of the craft becomes a rounding error in the math :).

This is so wrong, I'm not sure even where to start. I will limit myself to saying that more thrust without control just means you're going faster when you crash.

> I will limit myself to saying that more thrust without control just means you're going faster when you crash.

Correct.

Note that the problem specification talks about flying, and says nothing about the landing or surviving it.

Re: Large language models lack deep insights or a theory of mind

#266

Post by FrustratedMonky »

omginternets wrote: post #262
FrustratedMonky wrote: post #260

Earlier quoted context omitted.

That's why I'm not big on using word "hallucination" for this. It is really our subjective experience. Our 'view' of the outside world is constructed in the brain from the senses, like what is 'hot', or 'pressure', or 'blue', these don't exist in the world, we build them internally. Some people call it a 'hallucination', others 'subjective reality', or 'mental map'. The problem is, our brain is constructing reality,…

>The problem is, our brain is constructing reality, it can 'make things up', this has been shown in countless studies. And now that we have AI doing it too, it seems like easy association to make. I'm with you until this pair of sentences because I believe you are confusing ontological subjectivity (which is fine, for our purposes) with epistemic subjectivity (which isn't). Hallucination is an ontologically subjectiv…

I was speaking as in 'ontologically subjective', the experiencer. Like, 'what is it like to be a bat'.

The problem is, if a brain neural net has experiences, does a computer neural net also have experiences.

I don't like 'hallucinate' as description because everyone gets confused with mental illness. But it is more like the old Buddhist fable, mistaking a 'rope' for a 'snake', doing a double take, and then realizing it is just a 'rope'.

Humans can make mistakes based on their incomplete model of the inputs, and thus 'hallucinate' an answer just like an LLM might.

But I'm more in the camp that philosophical zombies are impossible. That by the time an AI can completely mimic a human, they would also be having some internal experience of some kind like a human, maybe not identical but something. But of course, it can't be proven either way, for AI or Humans. We only assume other minds experience reality like we do, but even that can't be 'proven'.

Re: Large language models lack deep insights or a theory of mind

#267

Post by nprateem »

corethree wrote: post #257
nprateem wrote: post #248

Earlier quoted context omitted.

Yeah I stopped reading since it's based on a false predicate. I've read the rest and it's still undermined by the same unprovable assumption. Perhaps consciousness is binary and the rock is just as conscious as a human. We don't know. But consciousness != intelligence, and it's reasonable to assume humans are more intelligent than rocks of course, but we can't say anything about each one's level of consciousness.

"consciousness" is a word made up by humans. There's no way someone can make up a word without "knowing" what it means. If we made up a word and we don't know what it means that means we "chose" not to know what it means. We made up the fact that we don't know. Ultimately all words in the english language are made up by humans. The concept of consciousness itself doesn't exist. It exists because we made up a word for…

Sorry, just wrong on so many levels. Consciousness is not a concept, it's a self-evident experience. A rose by any other name would be just as beautiful, etc.

Re: Large language models lack deep insights or a theory of mind

#268

Post by corethree »

nprateem wrote: post #267
corethree wrote: post #257

Earlier quoted context omitted.

"consciousness" is a word made up by humans. There's no way someone can make up a word without "knowing" what it means. If we made up a word and we don't know what it means that means we "chose" not to know what it means. We made up the fact that we don't know. Ultimately all words in the english language are made up by humans. The concept of consciousness itself doesn't exist. It exists because we made up a word for…

Sorry, just wrong on so many levels. Consciousness is not a concept, it's a self-evident experience. A rose by any other name would be just as beautiful, etc.

No need to apologize for your own ignorance. All things are concepts even a self evident experience is in itself a concept.

A rose, beauty. All are concepts. I'm sorry that your blinded by illusion of it all.

Re: Large language models lack deep insights or a theory of mind

#269

Post by corethree »

corethree wrote: post #268
nprateem wrote: post #267

Earlier quoted context omitted.

Sorry, just wrong on so many levels. Consciousness is not a concept, it's a self-evident experience. A rose by any other name would be just as beautiful, etc.

No need to apologize for your own ignorance. All things are concepts even a self evident experience is in itself a concept. A rose, beauty. All are concepts. I'm sorry that your blinded by illusion of it all.

Also, you're not being rational and consistent. First you say no body truly knows what consciousness is, that philosophers debate about it... then you define it as an "experience." Two inconsistent statements. For your second statement? Isn't that just a definition you chose that many "philosophers" disagree/agree with?

Don't answer that question. It's rhetorical. A rational discussion. can't be had with someone who is inconsistent with the statements he makes or someone who is deliberately trolling. I suspect you are the later.

Re: Large language models lack deep insights or a theory of mind

#270

Post by stuckinhell »

FrustratedMonky wrote: post #73
omginternets wrote: post #6

Earlier quoted context omitted.

We don't "make up" consciousness, but yes, there is a processing latency of around 250-300ms.

In the book "Being You" Anil Seth. It does postulate that we make up consciousness. The brain is trying to 'predict' the next sensory input, and that prediction is our awareness. What we would call our 'conscious self'. It makes point of calling it a 'controlled hallucinations', in that what we experience as our self. "Hallucination" being the experience we have as our brain 'predicting/controlling' for the sensory i…

yep that's what I was referring too!
Post reply on HN