Large language models lack deep insights or a theory of mind • Hacker News Board

Re: Large language models lack deep insights or a theory of mind

#41

Post by Androider » Thu, Nov 30, 2023, 6:01 PM UTC

tinco wrote: post #30

I think that if they would, that would be very surprising and indicative of a lot of wastefulness inside the model architecture. All these tests are simple single prompt experiments, so the LLM's get no chance to reason about their responses. They're just system 1 thinking, the equivalent of putting a gun to someone's head and asking them to solve a large division in 2 seconds. I bet a lot of these experiments would…

The equivalent for a human would be an reflexive response to a question, the kind you could immediately answer after being woken up at 3am in the morning. That type of answer has been deeply trained into the human networks and also requires no deep insight.

But if a human is allowed time and internal reasoning iterations, so should the LLM when determining if it has deep insight. Right now we're simply observing input -> output of LLMs, the equivalent of snap answers from a human. But nothing says it couldn't instead be an input -> extensive internal dialogue, maybe even between multiple expert models for seconds, minutes or hours, that are not at all visible to the prompter -> final insightful answer. Maybe future LLMs will say, "let me get back to you on that".

Re: Large language models lack deep insights or a theory of mind

#42

Post by fredliu » Thu, Nov 30, 2023, 6:03 PM UTC

I have small kids, toddlers, who can already speak the language but still developing their "sense of the world" or "theory of mind" if you will. Maybe it's just me, but talking to toddlers often reminds me of interacting with LLMs, where you would have this realization from time to time "oh, they don't get this, need to break down more to explain". Of course LLM has more elaborate language skills due to its exposure to a lot more text (toddlers definitely can't speak like Shakespeare if you ask them, unless, maybe, you are the tiger parents that's been feeding them Romeo and Juliet since 1.), but their ability of "reasoning" and "understanding" seems to be on a similar level. Of course, the other "big" difference, is that you expect toddlers to "learn and grow" to eventually be able to understand and develop meta cognitive abilities, while LLMs, unless you retrain them (maybe with another architecture, or meta architecture), "stay the same".

Re: Large language models lack deep insights or a theory of mind

#43

Post by famouswaffles » Thu, Nov 30, 2023, 6:04 PM UTC

swatcoder wrote: post #21

menssen wrote: post #9

I appreciate this paper for relatively clearly stating what "human-like" might entail, which in this case involves "reasoning about the causes behind other people's behavior" which is "critical to navigate the social world" as outlined in this citation: https://www.sciencedirect.com/science/article/abs/pii/S00100... I get frustrated often when people argue "well, it isn't really intelligent" and then give examples th…

The underlying problem is that "intelligence" is itself a crappy, poorly defined word with a fraught and inconsistent history. It doesn't appear until the early 20th century, in the shadow of compulsory education and the challenges it presented, first as a technical label for attempts to sort students -- and later soldiers -- into the tracks in which they're most likely to succeed, and then being haphazardly asserted…

There does seem to be a general factor of intelligence in humans though that is the single biggest indicator of performance. Yes there are other factors too.

>Here, the authors point out that the current batch of programs are not good at tasks that benefit from a theory of mind.

Not good at tasks that benefit from a theory of mind extracted from visual data.

Re: Large language models lack deep insights or a theory of mind

#44

Post by choudharism » Thu, Nov 30, 2023, 6:05 PM UTC

uoaei wrote: post #36

tinco wrote: post #30

I think that if they would, that would be very surprising and indicative of a lot of wastefulness inside the model architecture. All these tests are simple single prompt experiments, so the LLM's get no chance to reason about their responses. They're just system 1 thinking, the equivalent of putting a gun to someone's head and asking them to solve a large division in 2 seconds. I bet a lot of these experiments would…

> They're just system 1 thinking, the equivalent of putting a gun to someone's head and asking them to solve a large division in 2 seconds. No, it's the equivalent of putting a gun to someone's head and asking them "what are my intentions?" Which is readily available to any being with a theory of mind.

I don’t think LLMs have theory of mind, but your point is not very strong. You can literally query ChatGPT right now and see that it can figure out intentions (both superficial and deep) of a gun is held to a head quite easily.

Because, obviously, training data probably includes a decent amount of motivation breakdowns as a function of coercion.

It doesn’t know why, but it knows what to say.

Re: Large language models lack deep insights or a theory of mind

#45

Post by menssen » Thu, Nov 30, 2023, 6:08 PM UTC

swatcoder wrote: post #21

menssen wrote: post #9

I appreciate this paper for relatively clearly stating what "human-like" might entail, which in this case involves "reasoning about the causes behind other people's behavior" which is "critical to navigate the social world" as outlined in this citation: https://www.sciencedirect.com/science/article/abs/pii/S00100... I get frustrated often when people argue "well, it isn't really intelligent" and then give examples th…

The underlying problem is that "intelligence" is itself a crappy, poorly defined word with a fraught and inconsistent history. It doesn't appear until the early 20th century, in the shadow of compulsory education and the challenges it presented, first as a technical label for attempts to sort students -- and later soldiers -- into the tracks in which they're most likely to succeed, and then being haphazardly asserted…

For what it's worth, I don't take that framing of intelligence seriously either. It's useful to have a word to describe the far-future state of the increasing capabilities of Computers.

I'm just saying that I don't think there's any point on that line where we will be comfortable admitting that the machine is "intelligent" or "conscious" or "AGI," or whatever, and that I appreciate attempts to quantify (or at least qualify) what we MEAN when we say that, rather than just goalpost-moving.

Re: Large language models lack deep insights or a theory of mind

#46

Post by trash_cat » Thu, Nov 30, 2023, 6:12 PM UTC

deeviant wrote: post #26

> A chief goal of artificial intelligence is to build machines that think like people. I disagree with the topic sentence. The goal should not be to "build machines that think like people", but to build machines that think, period. The way humans think is unlikely to be the optimal way to go about thinking anyways. Instead of talking about thinking, we should be talking about function. Less philosophy and more realit…

We don't make planes based on how birds flap their wings.

Re: Large language models lack deep insights or a theory of mind

#47

Post by Barrin92 » Thu, Nov 30, 2023, 6:16 PM UTC

No LLMs don't think like people, they're architecturally incapable of doing so. They have, physically unlike humans no access to their own internal state and they're, save for a small context window, static systems. They also have no insights. There's a hilarious video about LLM Jailbreaks by Karpathy[1] from a week ago, where he shows how you can break model responses by asking the same question with a base64 string, preceding the prompt with an image of a panda(???) or just random word salad.

LLM's are basically a validation of Searle's Chinese room. What they've proven is that you can build functioning systems that perform intelligent tasks purely at the level of syntax. But there is no (or very little) understanding of semantics. If I ask a person on how to end the world, whether I ask in French or English or base64 or perform a 50 word incantation beforehand likely does not matter. (unless of course the human is also just parroting an answer)

[1] https://youtu.be/zjkBMFhNj_g?t=2974

Re: Large language models lack deep insights or a theory of mind

#48

Post by jacobsimon » Thu, Nov 30, 2023, 6:16 PM UTC

munificent wrote: post #32

sdwr wrote: post #29

Earlier quoted context omitted.

Maybe the soul is social, and oriented towards others? I believe it can be constructed. If you assume that "the eyes are the window to the soul", you notice some interesting properties. 1. It is far more observable from the outside (eyes open/lidded/closed, emotion read in eyes) 2. It affects behavior in a diffuse way 3. It pays attention but does not dictate

> Maybe the soul is social My pet theory about human consciousness is that is that consciousness is simply recursive theory of mind. Theory of mind [1] is our ability to simulate and reason about the mental states of others. It's how we predict what people are thinking and how they will react to our actions, which is critical for choosing how to act in a social environment. But when you're thinking about what's in so…

You might find this book interesting! This is essentially the theory put forward. https://www.google.com/books/edition/Consciousness_and_the_S...

Re: Large language models lack deep insights or a theory of mind

#49

Post by pixl97 » Thu, Nov 30, 2023, 6:17 PM UTC

swatcoder wrote: post #21

menssen wrote: post #9

I appreciate this paper for relatively clearly stating what "human-like" might entail, which in this case involves "reasoning about the causes behind other people's behavior" which is "critical to navigate the social world" as outlined in this citation: https://www.sciencedirect.com/science/article/abs/pii/S00100... I get frustrated often when people argue "well, it isn't really intelligent" and then give examples th…

The underlying problem is that "intelligence" is itself a crappy, poorly defined word with a fraught and inconsistent history. It doesn't appear until the early 20th century, in the shadow of compulsory education and the challenges it presented, first as a technical label for attempts to sort students -- and later soldiers -- into the tracks in which they're most likely to succeed, and then being haphazardly asserted…

Most of what you're saying here is describing the alignment issue.

We (mostly) don't want unaligned A(G|S)I. The outcomes of that could be extenstential.

Re: Large language models lack deep insights or a theory of mind

#50

Post by throwanem » Thu, Nov 30, 2023, 6:22 PM UTC

famouswaffles wrote: post #43

swatcoder wrote: post #21

Earlier quoted context omitted.

The underlying problem is that "intelligence" is itself a crappy, poorly defined word with a fraught and inconsistent history. It doesn't appear until the early 20th century, in the shadow of compulsory education and the challenges it presented, first as a technical label for attempts to sort students -- and later soldiers -- into the tracks in which they're most likely to succeed, and then being haphazardly asserted…

There does seem to be a general factor of intelligence in humans though that is the single biggest indicator of performance. Yes there are other factors too. >Here, the authors point out that the current batch of programs are not good at tasks that benefit from a theory of mind. Not good at tasks that benefit from a theory of mind extracted from visual data.

"Seem" is doing a lot of work here. So is the implicit claim that theory of mind in general can be demonstrated by current-gen foundation models, and only those aspects dependent on vision cannot.