Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Large language models lack deep insights or a theory of mind

arxiv.org

Post reply on HN
Print view View original

181–190 of 270 posts

Re: Large language models lack deep insights or a theory of mind

#181

Post by dragonwriter »

resters wrote: post #165
calf wrote: post #141

Earlier quoted context omitted.

So my observation is that we could embody an AI so that it learns theory of mind-body--but then we could remove the body. This gives a theory of mindful entity that does not need a body to exist. Then the next research step could be to study those properties so as to reconstruct/reproduce a theory of mind-body AI, without needing any embodiment process at all to obtain it. Is that, in principle, possible? It is uncle…

> we could embody an AI ... a hardware interface that generates a token stream from a living human's body would seem to enable this at some level. Not sure how it would work at scale. Maybe something much simpler like phones with built-in VOC sensors that can detect nuances of the user's perspiration, combined with real time emotion sensing via gait, voice, along with metadata that is already available would be suffi…

> ... a hardware interface that generates a token stream from a living human's body would seem to enable this at some level.

A hardware interface that generated a datastream from sensors monitoring the status and surroundings of the hardware the LLM was running on would be more to the point.

Re: Large language models lack deep insights or a theory of mind

#182

Post by tivert »

famouswaffles wrote: post #173
tivert wrote: post #120

Earlier quoted context omitted.

> A better eval is to use actual scientifically tested psychology test on text (the native and strongest domain for LLMs), for example the sort of scenarios used to gauge when children develop theory of mind (“Alice puts her keys on the table then leaves the room. Bob moves the keys to the drawer. Alice returns. Where does she think the keys are?”) which GPT-4 can handle easily; it is very clear from this that GPT ha…

How do I know you have theory of mind or are concious if not the "right" response to a test ? As far as I'm concerned, the only person I can be certain is concious is me. It doesn't have to be a "scientifically tested psychology test" Construct your own story with multiple characters of varying knowledge and beliefs and see how it does.

You're missing the point.

With prior knowledge of the test, even something that verifiably lacks the cognitive capability to legitimately pass the test (e.g. FizzBuzz level stuff) can pass by cheating.

Re: Large language models lack deep insights or a theory of mind

#183

Post by mr_toad »

I_Am_Nous wrote: post #104
Androider wrote: post #41

Earlier quoted context omitted.

The equivalent for a human would be an reflexive response to a question, the kind you could immediately answer after being woken up at 3am in the morning. That type of answer has been deeply trained into the human networks and also requires no deep insight. But if a human is allowed time and internal reasoning iterations, so should the LLM when determining if it has deep insight. Right now we're simply observing inpu…

This reminds me of The Last Question by Isaac Asimov. I also think if we stopped expecting all LLMs to have an immediate answer, it would be relatively easy to shim some kind of "conscience" to direct the output in different ways. Similar to the safeties already in place in LLMs, but instead of it just saying "NO DON'T SAY THAT" it can dialog internally to change what the output is until it reaches what it believes t…

> I also think if we stopped expecting all LLMs to have an immediate answer, it would be relatively easy to shim some kind of "conscience" to direct the output in different ways.

If the shim was just another AI, then how do you align that AI? Who watches the watchers? But if it was a deterministic algorithm it would probably fail for the same reasons that algorithmic AI never went anywhere.

Re: Large language models lack deep insights or a theory of mind

#184

Post by TeMPOraL »

thewakalix wrote: post #70
swatcoder wrote: post #21

Earlier quoted context omitted.

The underlying problem is that "intelligence" is itself a crappy, poorly defined word with a fraught and inconsistent history. It doesn't appear until the early 20th century, in the shadow of compulsory education and the challenges it presented, first as a technical label for attempts to sort students -- and later soldiers -- into the tracks in which they're most likely to succeed, and then being haphazardly asserted…

> and it wants to kill everyone It wouldn't have to want to kill everyone. As long as it doesn't want to not kill everyone, the side effects of it getting what it wants could be catastrophic. > and we don't notice How well do we understand what's going on inside ChatGPT? How well will we understand the next? > and forget to shut it off Earlier I would have argued that sufficiently advanced AI could prevent itself fro…

> Earlier I would have argued that sufficiently advanced AI could prevent itself from being shut off via Things You Didn't Expect

There's a good argument along these lines that I keep reposting when someone asks if we can't just shut the AI off. "All you gotta do is push a button, sir?"

https://www.youtube.com/watch?v=ld-AKg9-xpM&t=30s

Re: Large language models lack deep insights or a theory of mind

#185

Post by astrange »

NoMoreNicksLeft wrote: post #178
fnordpiglet wrote: post #151

Earlier quoted context omitted.

Are you saying every thought you’ve ever had is a logical consequence of all prior thoughts with a definite traceable lineage? (Assuming your mind was too dumb as some point and a thought emerged that originated all future thoughts)

I'm saying that I'm extremely confident that... 1. Most people (including Buddhists) think that their internal monologue comprises their thoughts, in majority or even in total. 2. Can't conceive of the possibility of a thought existing other than expressed in their spoken language 3. Find it difficult or impossible to suppress their inner monologue. 4. When successful at suppressing it believe that their thoughts hav…

I didn't say anything mystical.

> Most people (including Buddhists) think that their internal monologue comprises their thoughts

And they don't think this, in fact they think the opposite.

You seem to be offended I used the expression "from nowhere" instead of "from your unconscious" or something.

Re: Large language models lack deep insights or a theory of mind

#186

Post by tim333 »

tivert wrote: post #179
tim333 wrote: post #175

Earlier quoted context omitted.

AGI as in computer intelligence that out does humans would be a huge deal in practical terms. Chat GPT and similar are kind of like handy toys. With proper AGI you could link it to a robot body and tell it to go off, design a better version of itself, make a billion more robots and take over the world. It's a different category of thing. And if you think that's just sci-fi I think you'll get a surprise at some point…

> With proper AGI you could link it to a robot body and tell it to go off, design a better version of itself, make a billion more robots and take over the world. ^^^^^ literally a plot ripped from the pages of science fiction used to reason about the real world. > And if you think that's just sci-fi I think you'll get a surprise at some point during your life. Such faith that fantasy can be made real. Wake me up when…

I've never actually read that in fiction. It's just logical really. I mean I'm sure it is in fiction somewhere because the idea is obvious.

Re: Large language models lack deep insights or a theory of mind

#187

Post by tivert »

tim333 wrote: post #186
tivert wrote: post #179

Earlier quoted context omitted.

> With proper AGI you could link it to a robot body and tell it to go off, design a better version of itself, make a billion more robots and take over the world. ^^^^^ literally a plot ripped from the pages of science fiction used to reason about the real world. > And if you think that's just sci-fi I think you'll get a surprise at some point during your life. Such faith that fantasy can be made real. Wake me up when…

I've never actually read that in fiction. It's just logical really. I mean I'm sure it is in fiction somewhere because the idea is obvious.

> I've never actually read that in fiction.

I find that hard to believe. Ever watch Terminator?

But even if true, that science-fictional plot is so pervasive it would be easy to pick up from the millions who have the software engineer's blurry line between fantasy and reality.

> It's just logical really.

OK, then. You're a GI, go off and build an army of better yous and take over the world.

Re: Large language models lack deep insights or a theory of mind

#188

Post by TeMPOraL »

fredliu wrote: post #42

I have small kids, toddlers, who can already speak the language but still developing their "sense of the world" or "theory of mind" if you will. Maybe it's just me, but talking to toddlers often reminds me of interacting with LLMs, where you would have this realization from time to time "oh, they don't get this, need to break down more to explain". Of course LLM has more elaborate language skills due to its exposure…

> Maybe it's just me, but talking to toddlers often reminds me of interacting with LLMs

It's not just you. It hit me almost a year ago, when I realized my then 3.5yo daughter has a noticeable context window of about 30 seconds - whenever she went on her random rant/story, anything she didn't repeat within 30 seconds would permanently fall out of the story and never be mentioned again.

It also made me realize why small kids talk so repetitively - what they don't repeat they soon forget, and what they feel like repeating remains, so over the course of couple minutes, their story kind of knots itself in a loop, being mostly made of the thoughts they feel compelled to carry forward.

Re: Large language models lack deep insights or a theory of mind

#189

Post by TeMPOraL »

tivert wrote: post #127
poulsbohemian wrote: post #74

Earlier quoted context omitted.

Completely agree, and while we are at it... look I'm just a guy, not an expert, but I can't understand why there's so much focus on AGI. It feels like there are so many niche areas where we could apply some kind of analytical augmentation and by solving problems in the small, might learn something that would help figure the larger question of intelligence. I don't need the AI to replace everything I do, I need it to…

> ...but I can't understand why there's so much focus on AGI. Lots of software engineers have spent their lives reading sci-fi that features AGI, and they're excited by/lost in that fantasy. It's interesting to see that in people who often view themselves as hyper-rational.

> It's interesting to see that in people who often view themselves as hyper-rational.

It's perhaps because they are rational enough to realize, thanks to the same knowledge/skill that put them on the software engineer careers, that AGI isn't a fantasy but a possibility and a potentially very big deal.

Re: Large language models lack deep insights or a theory of mind

#190

Post by vidarh »

twobitshifter wrote: post #139
tinco wrote: post #130

Earlier quoted context omitted.

I feel the "let's think about it step by step" is a bit of a hack. To circumvent the fact that there's no external loop you use the fact that it gets re-run on every token so you can store a bit of state in the tokens that it's already generated. Or am I misunderstanding something about that technique?

You are right, it’s sometimes called zero shot chain of thought, but it’s a way of getting the type of thing you are describing to happen. The LLMs somehow process things in a perceived step by step to get a much improved answer. Whether the external loop or an llm imposed internal loop, does it matter? Are our own minds looping or just adding tokens?

The main thing that may matter, at least in the short run, is that an external loop allows us to inject additional steps by applying heuristics and allowing tool use. E.g. we can let the LLM "realise" there errors in its code and have it continue from an injected "thought" about making sure to fix the errors from the compiler before presenting it's output, or "remembering" that it needs test cases etc.

We can also potentially add longer term memory - summarise the context, and judge which parts are important and stuff them in a vector store, and now and again swap in similar pieces of past context.

But of course it's not either or - better prompting to get the LLMs to do better from the start doesn't compete with then feeding that into an external loop as well.

Post reply on HN