Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Large language models lack deep insights or a theory of mind

arxiv.org

Post reply on HN
Print view View original

231–240 of 270 posts

Re: Large language models lack deep insights or a theory of mind

#231

Post by nprateem »

corethree wrote: post #146
joduplessis wrote: post #66

For me, the entire AGI conversation is hyperbolic / hype. How can we infer intelligence to something when we, ourselves, have such a poor (none) grasp of what makes us conscience? I'm associating intelligence with consciousness - because it seems correlated. Are we really ready to associate "AGI" with solving math problems ("new Q algo.")? That seems incredibly naive & reinforces my opinion that LLM's are much more l…

It's not hype. It's a language problem that makes people like you think this way. The problem is consciousness is a vocabulary word that establishes a hard boundary where such a boundary doesn't exit. The language makes you think either something is conscious or it is not when the reality is that these two concepts are actually extreme endpoints on a gradient. The vocabulary makes the concept seem binary and makes it…

> A rock is not conscious. That's obvious.

But it's not obvious at all. It may possess consciousness in a way we can't relate to or communicate with.

This is the whole problem with consciousness and has been discussed by philosophers for centuries. We each appear to be conscious but can't be certain anything else is or isn't.

Re: Large language models lack deep insights or a theory of mind

#232

Post by mistermann »

TeMPOraL wrote: post #192
tivert wrote: post #187

Earlier quoted context omitted.

> I've never actually read that in fiction. I find that hard to believe. Ever watch Terminator? But even if true, that science-fictional plot is so pervasive it would be easy to pick up from the millions who have the software engineer's blurry line between fantasy and reality. > It's just logical really. OK, then. You're a GI, go off and build an army of better yous and take over the world.

The idea is indeed logical and stupidly obvious, once you learn the basics of what "optimization" means, or what "recursion" is. > I find that hard to believe. Ever watched Terminator? Terminator has fuck all to do with recursive self-improvement. Don't confuse people who grew up on sci-fi with people who casually went to see Terminator or some other pop-culture artifact featuring some kind of "AI". > OK, then. You'r…

It is not technically logical to think one can see the future, but it is colloquially logical.

Judging reality by how it appears is a bad strategy, this should be common knowledge by now.

What's concerning to me is that I suspect LLM's will be able to learn and remember thousands of basic facts like this, and ~reason on top of them. Perhaps they won't figure this out on their own, but what if all it takes is one individual to point them in this direction? I bet there are numerous people who know much more about this than me working for our various three letter agencies.

Re: Large language models lack deep insights or a theory of mind

#233

Post by nprateem »

danenania wrote: post #23
fnordpiglet wrote: post #19

In Buddhism there’s the idea that our core self is awareness, which is silent - it doesn’t think in a perceptible way, it doesn’t feel in a visceral way, but it underpins thought and feeling, and is greatly impacted by it. A large part of meditation and “release of suffering” is learning to let your awareness lead your thinking rather than your thinking lead your awareness. To be clear, I think this is in fact a corr…

Another idea from Buddhism is that this core of awareness you're talking about is nothingness. So when you stop all thought (if such a thing is really possible), you temporarily cease to exist as an individual consciousness. "Awareness" is when the thoughts come back online and you think "whoa, I was just gone for a bit". If that's how it works, then the "soul" is more like an emergent phenomenon created by the inter…

You've misunderstood. Awareness absolutely is not "when thoughts come back online".

This is simple to experience for yourself since it'd mean we stop being aware when listening so intently thoughts stop. Obviously we don't cease to be aware at such times.

You've also misunderstood what is meant by nothingness ("no thingness").

Re: Large language models lack deep insights or a theory of mind

#234

Post by two_in_one »

chii wrote: post #226
two_in_one wrote: post #99

Earlier quoted context omitted.

Just note that loop doesn't have to be visible from outside. It can be internal, with another driving thread asking right questions. Inner monologue. Then the summary is given back to user. This will give the model space for 'thinking' with internally generated text much large than the visible prompt + output. This way multi-step logic can be implemented.

> with another driving thread asking right questions. Inner monologue. spoilers warning: Is that basically the plot to westworld?

Inner monologue idea was around for a while. That's interesting that it actually materializes. It was thought that AGI will be operating with some abstracts, logic rules, probability calculations, etc. Not thinking in plain English.

Re: Large language models lack deep insights or a theory of mind

#235

Post by mistermann »

FrustratedMonky wrote: post #157
upghost wrote: post #93

Earlier quoted context omitted.

Couldn't agree more. How about this -- I think we've already reached AGI. Let me know if this tracks: Pick a set of tasks that can be considered AGI tasks. Provided the task sequences can be compared as closer to AGI or further from AGI, we can create a reward model using the same techniques as were used by ChatGPT via RLHF. Thus, for any definition of AGI that is meaningful and selectable, even if subjectively selec…

Exactly. 5 years ago, we would have said what GPT-4 is doing now would be AGI. Now it is here and it's like "No, what we really meant is it has to be the next Einstein". People are forgetting how stupid people are. GPT is already better than average human. Most people can't do what we claim GPT must be capable of to qualify as AGI. The only logical conclusion is that many people are also not conscious and don't quali…

> The only logical conclusion is that many people are also not conscious and don't qualify as being able to reason.

Most humans aren't "able to" juggle 3 balls, but most humans are physically and mentally capable of learning to be able to juggle 3 balls, it's just not a common thing for most people to learn. The same used to be true of reading and basic math, but look where we are now with some good planning and hard work!

Re: Large language models lack deep insights or a theory of mind

#236

Post by AnimalMuppet »

TeMPOraL wrote: post #197
AnimalMuppet wrote: post #113

Earlier quoted context omitted.

That's not the hard part about building a working airplane, though...

Yes, but as the saying goes, anything can fly if you strap a powerful enough engine to it... so we've demonstrated basic capability, and the rest is now optimizing its performance a couple orders of magnitude.

Not at all. "Anything can fly", but not everything can fly without killing you. The fundamental problem is three-axis control with stability. A source of power does exactly nothing to solve that.

Re: Large language models lack deep insights or a theory of mind

#237

Post by mistermann »

Tadpole9181 wrote: post #52
dsr_ wrote: post #20

Earlier quoted context omitted.

There is no reason to believe (evidence) that any meaning ascribed to an LLM's utterances comes from the LLM rather than being pareidolia. If you've found some, please let everyone know.

There is no reason to believe (evidence) that any meaning ascribed to anyone but me 's utterances comes from the person rather than being pareidolia. If you've found some, please let everyone know.

It's worse: there is technically no way to know if there is in fact no evidence, it is a colloquial phrase that people cannot think twice about.

Re: Large language models lack deep insights or a theory of mind

#238

Post by corethree »

nprateem wrote: post #231
corethree wrote: post #146

Earlier quoted context omitted.

It's not hype. It's a language problem that makes people like you think this way. The problem is consciousness is a vocabulary word that establishes a hard boundary where such a boundary doesn't exit. The language makes you think either something is conscious or it is not when the reality is that these two concepts are actually extreme endpoints on a gradient. The vocabulary makes the concept seem binary and makes it…

> A rock is not conscious. That's obvious. But it's not obvious at all. It may possess consciousness in a way we can't relate to or communicate with. This is the whole problem with consciousness and has been discussed by philosophers for centuries. We each appear to be conscious but can't be certain anything else is or isn't.

You're not reading my argument. You responded to one part of it. As a whole my argument is not about the problem you're describing.

My argument is saying that the problem is a sham. An illusion. The problem doesn't even exist. Read my whole write up.

Re: Large language models lack deep insights or a theory of mind

#239

Post by crdrost »

munificent wrote: post #145
jacobsimon wrote: post #48

Earlier quoted context omitted.

You might find this book interesting! This is essentially the theory put forward. https://www.google.com/books/edition/Consciousness_and_the_S...

Ah, that looks perfect! Thank you! I knew other people smarter than me must have stumbled onto this idea as well.

FWIW if you're looking for book recommendations this is similar to the thesis of Gödel, Escher, Bach: An Eternal Golden Braid.

Re: Large language models lack deep insights or a theory of mind

#240

Post by danenania »

nprateem wrote: post #233
danenania wrote: post #23

Earlier quoted context omitted.

Another idea from Buddhism is that this core of awareness you're talking about is nothingness. So when you stop all thought (if such a thing is really possible), you temporarily cease to exist as an individual consciousness. "Awareness" is when the thoughts come back online and you think "whoa, I was just gone for a bit". If that's how it works, then the "soul" is more like an emergent phenomenon created by the inter…

You've misunderstood. Awareness absolutely is not "when thoughts come back online". This is simple to experience for yourself since it'd mean we stop being aware when listening so intently thoughts stop. Obviously we don't cease to be aware at such times. You've also misunderstood what is meant by nothingness ("no thingness").

Ok? I’m not an expert, but I’ve read enough to know that there are many differing takes on these topics within Buddhist thought. Your appeals to dogma are not convincing.
Post reply on HN