Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Large language models lack deep insights or a theory of mind

arxiv.org

Post reply on HN
Print view View original

251–260 of 270 posts

Re: Large language models lack deep insights or a theory of mind

#251

Post by blovescoffee »

nyrikki wrote: post #92
joduplessis wrote: post #66

For me, the entire AGI conversation is hyperbolic / hype. How can we infer intelligence to something when we, ourselves, have such a poor (none) grasp of what makes us conscience? I'm associating intelligence with consciousness - because it seems correlated. Are we really ready to associate "AGI" with solving math problems ("new Q algo.")? That seems incredibly naive & reinforces my opinion that LLM's are much more l…

A particular subset of Connectivism have a philosophical belief that the mind IS a neutral net, not that it is a reductive practical model. Hinton is one of these individuals and with no definition of what intelligence is it is an understandable of dogmatic position. This whole problem of not being able to define what intelligence is pretty much allows us all to pick and choose. In my mind BPP is the complexity class…

You should state what AGI is and then state why it’s nog possible.

Re: Large language models lack deep insights or a theory of mind

#252

Post by AnimalMuppet »

TeMPOraL wrote: post #247
AnimalMuppet wrote: post #236

Earlier quoted context omitted.

Not at all. "Anything can fly", but not everything can fly without killing you. The fundamental problem is three-axis control with stability. A source of power does exactly nothing to solve that.

No, that just means you need a bigger engine . (And, perhaps, an inertial dampener, to survive the gees.) Think of it this way: the more thrust you get, the farther you'll go before control problems start to manifest. Keep pushing, and eventually the rest of the craft becomes a rounding error in the math :).

This is so wrong, I'm not sure even where to start.

I will limit myself to saying that more thrust without control just means you're going faster when you crash.

Re: Large language models lack deep insights or a theory of mind

#254

Post by neuronerdgirl »

creer wrote: post #242
tinco wrote: post #30

I think that if they would, that would be very surprising and indicative of a lot of wastefulness inside the model architecture. All these tests are simple single prompt experiments, so the LLM's get no chance to reason about their responses. They're just system 1 thinking, the equivalent of putting a gun to someone's head and asking them to solve a large division in 2 seconds. I bet a lot of these experiments would…

At the same time, if LLMs are based on all or enough human writings then don't they necessarily contain a theory of mind? A rather general, smoothed out and still neurotic one probably. But still just like an LLM can't be expected to have a specific knowledge of hydraulics, it also has read more hydraulics than even experts might be expected to. That's the entire issue about it, right? This issue of "is most of our m…

That's not what theory of mind is. Theory of mind is the understanding - not academic understanding, but functional, meaningful understanding that guides your behavior - that the other people (AIs?) around you have their own separate thoughts and feelings from your own. This is what leads to the stark behavioral differences as a child develops from ~3-6 years of age. Imagine a child is playing with their toys and puts one of them in a box out of the room, then later asks you to retrieve it for them. A 3 year old will assume that, because they know they put it in the box, that you know it is there as well. (Incidentally, this is where a lot of tantrums come from, because they can't understand why you aren't reading their mind and get frustrated.) A 6 year old will not make this assumption.

While this is a simple example, the concept extrapolates to more complex issues as we develop cognitively and emotionally. Not only is theory of mind about recognizing that your thoughts and feelings are separate and distinct, but also being able to project your understanding into others to predict how they might be thinking and feeling in their own separate circumstance. This is the fundamental basis of empathy, or being able to predict and understand how others around you may be feeling given their unique current circumstances even if they are different from your own.

LLMs have not demonstrated any of the above understanding. Again, I'm sure they could generate a definition for it, and on any given prompt they could probably even generate some generic-ish text that could pose as empathy, but so can a horoscope. But there is not the deep, continuous comprehension of the user as a separate thinking and feeling agent that is fundamental to the idea of theory of mind.

Re: Large language models lack deep insights or a theory of mind

#255

Post by omginternets »

FrustratedMonky wrote: post #73
omginternets wrote: post #6

Earlier quoted context omitted.

We don't "make up" consciousness, but yes, there is a processing latency of around 250-300ms.

In the book "Being You" Anil Seth. It does postulate that we make up consciousness. The brain is trying to 'predict' the next sensory input, and that prediction is our awareness. What we would call our 'conscious self'. It makes point of calling it a 'controlled hallucinations', in that what we experience as our self. "Hallucination" being the experience we have as our brain 'predicting/controlling' for the sensory i…

If you want to equate "emergence" with "making something up", then fine, I guess. I'm just not sure what what you can possibly conclude from that equivalence.

Re: Large language models lack deep insights or a theory of mind

#256

Post by theptip »

tivert wrote: post #120
theptip wrote: post #117

This is a terrible eval. Do not update your beliefs on whether LLMs have Theory of Mind based on this paper. The eval is a weird, noisy visual task (picture of astronaut with “care packages”). Their results are hopelessly narrow. A better eval is to use actual scientifically tested psychology test on text (the native and strongest domain for LLMs), for example the sort of scenarios used to gauge when children develop…

> A better eval is to use actual scientifically tested psychology test on text (the native and strongest domain for LLMs), for example the sort of scenarios used to gauge when children develop theory of mind (“Alice puts her keys on the table then leaves the room. Bob moves the keys to the drawer. Alice returns. Where does she think the keys are?”) which GPT-4 can handle easily; it is very clear from this that GPT ha…

Of course when testing an LLM you do not use the exact text from studies in the training set. There are an infinite number of structural permutations that preserve the methodology sufficiently to be comparable to the existing literature, while being distinct enough to be outside the training distribution.

Of course, another approach you can take if you suspect contamination is to ask follow-up questions that are less likely to be in any training data; if the LLM is a mere stochastic parrot without a ToM it will not be able to give well-formed answers to follow-up questions.

Perhaps if I'd said "established psychology research methodology" the intent would have been clearer?

Re: Large language models lack deep insights or a theory of mind

#257

Post by corethree »

nprateem wrote: post #248
corethree wrote: post #238

Earlier quoted context omitted.

You're not reading my argument. You responded to one part of it. As a whole my argument is not about the problem you're describing. My argument is saying that the problem is a sham. An illusion. The problem doesn't even exist. Read my whole write up.

Yeah I stopped reading since it's based on a false predicate. I've read the rest and it's still undermined by the same unprovable assumption. Perhaps consciousness is binary and the rock is just as conscious as a human. We don't know. But consciousness != intelligence, and it's reasonable to assume humans are more intelligent than rocks of course, but we can't say anything about each one's level of consciousness.

"consciousness" is a word made up by humans. There's no way someone can make up a word without "knowing" what it means.

If we made up a word and we don't know what it means that means we "chose" not to know what it means. We made up the fact that we don't know. Ultimately all words in the english language are made up by humans.

The concept of consciousness itself doesn't exist. It exists because we made up a word for it. And the concept seems fuzzy because the word was made up and a fuzzy definition for it was chosen.

When you debate what "consciousness" is you are debating the definition of the word. This is not profound. The word is made up, the definition is arbitrarily chosen. You are debating a vocabulary problem what arbitrary definition should be attached to what arbitrary word. You are attempting to refine the fuzzy boundaries of a definition that we as humans already made fuzzy by our own choice.

Take a car and a boat. If I made some vehicle that can both drive on the road and sail on the water, is it a car or a boat? What you're not seeing is that it doesn't matter. It's just a vehicle, but the words "car" and "boat" lock you into this delusional debate that's attempting to classify the car-boat as one or the other. Do you understand? The concept of a car and a boat is poorly defined language influencing the way you think. Whether the vehicle is a car or a boat is meaningless. Same with consciousness.

If you still don't get it. How about this. I'll make up a new concept called Flurmo. Flurmo is something that is 30-40% a car and 60-70% a boat. Now when you debate whether the vehicle is a car or a boat you have to consider whether it's a flurmo as well. Car, boat or flurmo? It's easy to see how flurmo is made up, it's harder to see why car and boat are the SAME thing, they are also words that are made up. And so is consciousness. Consciousness is flurmo.

You stopped reading, because you made a false assumption. You then continued reading on my prompt and you came out with a conclusion based off of you misunderstanding the point. Hopefully you get what I'm saying now.

Re: Large language models lack deep insights or a theory of mind

#258

Post by hskalin »

lukev wrote: post #114
Androider wrote: post #41

Earlier quoted context omitted.

The equivalent for a human would be an reflexive response to a question, the kind you could immediately answer after being woken up at 3am in the morning. That type of answer has been deeply trained into the human networks and also requires no deep insight. But if a human is allowed time and internal reasoning iterations, so should the LLM when determining if it has deep insight. Right now we're simply observing inpu…

Completely agree. From a computer science point of view: a single prompt/response cycle from a LLM is equivalent to a pure function; the answer is a function of the prompt and the model weights and is fundamentally reducible to solving a big math equation (in which each model parameter is a term.) It seems almost self evident that "reasoning" worthy of the name would involve some sort of iterative/recursive search pr…

Do you have any good resources for chain-of-thought prompting experiments?

Re: Large language models lack deep insights or a theory of mind

#259

Post by hskalin »

lixy wrote: post #100
tinco wrote: post #30

I think that if they would, that would be very surprising and indicative of a lot of wastefulness inside the model architecture. All these tests are simple single prompt experiments, so the LLM's get no chance to reason about their responses. They're just system 1 thinking, the equivalent of putting a gun to someone's head and asking them to solve a large division in 2 seconds. I bet a lot of these experiments would…

Yep, prototype exactly that this past week. With a strong instruction spec prompt from the start, you can have an AI come up with a much better answer by making sure it knows it has time to answer the questions and how it should approach the problem in stages. The great part is with clear enough directions it also knows how to evaluate whether its done or not.

Can you share the kind of prompts you used?

Re: Large language models lack deep insights or a theory of mind

#260

Post by FrustratedMonky »

omginternets wrote: post #255
FrustratedMonky wrote: post #73

Earlier quoted context omitted.

In the book "Being You" Anil Seth. It does postulate that we make up consciousness. The brain is trying to 'predict' the next sensory input, and that prediction is our awareness. What we would call our 'conscious self'. It makes point of calling it a 'controlled hallucinations', in that what we experience as our self. "Hallucination" being the experience we have as our brain 'predicting/controlling' for the sensory i…

If you want to equate "emergence" with "making something up", then fine, I guess. I'm just not sure what what you can possibly conclude from that equivalence.

That's why I'm not big on using word "hallucination" for this. It is really our subjective experience. Our 'view' of the outside world is constructed in the brain from the senses, like what is 'hot', or 'pressure', or 'blue', these don't exist in the world, we build them internally. Some people call it a 'hallucination', others 'subjective reality', or 'mental map'.

The problem is, our brain is constructing reality, it can 'make things up', this has been shown in countless studies. And now that we have AI doing it too, it seems like easy association to make.

Post reply on HN