Live data from Hacker News

Large language models lack deep insights or a theory of mind

arxiv.org

221–230 of 270 posts

Re: Large language models lack deep insights or a theory of mind

#221
post #156

> A chief goal of artificial intelligence is to build machines that think like people. Maybe that's their goal. But for many users of AI, the goal is to have easy and affordable access to a machine that, for some input (perhaps in a tightly constrained domain), gives us the output that we would expect from a high-functioning human being. When I use ChatGPT as a coding helper, I really don't care about its "theory of…

Look this is the only time I'll engage in this sort of discussion on HN[1], but first Donald Knuth is a real Human and it's extremely weird to position world class experts as something otherworldly. Second, suppose you got what you wished for (you used the "us" pronoun), is that not a sentient mind that you're forcing to do your labour? Does that not raise a ton of red flags in your ethics? [1] normally I find HN dis…

> is that not a sentient mind that you're forcing to do your labour? Does that not raise a ton of red flags in your ethics?

Maybe and maybe not. Being sentient or even sapient doesn’t mean it has human emotions, feelings or motivations. You have to be very careful when ascribing naturally evolved emotions and motivations to something that is not a naturally evolved intellect.

Re: Large language models lack deep insights or a theory of mind

#222

Earlier quoted context omitted.

Are you saying every thought you’ve ever had is a logical consequence of all prior thoughts with a definite traceable lineage? (Assuming your mind was too dumb as some point and a thought emerged that originated all future thoughts)

I'm saying that I'm extremely confident that... 1. Most people (including Buddhists) think that their internal monologue comprises their thoughts, in majority or even in total. 2. Can't conceive of the possibility of a thought existing other than expressed in their spoken language 3. Find it difficult or impossible to suppress their inner monologue. 4. When successful at suppressing it believe that their thoughts hav…

This looks like AI-buddhism, a "fork" of buddhism rewritten by an uber-LLM of the future, in which it will discard everything it cannot relate to. To such an AI, the only internal process is a sequence of words, and nothing is thinkable outside such a sequence.

However, we can imagine images, shapes and sounds, indescribable in words, and even higher there are abstract ideas that can't be expressed with images or sounds, let along with words.

The "thoughts from without" may appear for a multitude of reasons, because our minds aren't isolated systems. The source may be some sensory input that triggered a memory, and I wouldn't even reject the possibility, that brains can react to background electromagnetic noise and form thoughts based on it.

Re: Large language models lack deep insights or a theory of mind

#223
"Large language models lack deep insights or a theory of mind"

Funnily enough, this statement also applies to people that are scared of AI.

Maybe a bit off topic but does anyone else have that friend who sends them fear mongering AI videos with captions like "shocking AI" that are blatantly unimpressive or completely fake?

What is the best way to subdue this kind of fear in a friend, sending them written articles from high level researchers like Brooks does not work.

Re: Large language models lack deep insights or a theory of mind

#224
post #204

Earlier quoted context omitted.

>>> I've never actually read that in fiction. >> I find that hard to believe. Ever watched Terminator? > Terminator has fuck all to do with recursive self-improvement. Don't confuse people who grew up on sci-fi with people who casually went to see Terminator or some other pop-culture artifact featuring some kind of "AI". You're not following the thread. The future timeline in Terminator does involve something like an…

> The future timeline in Terminator does involve something like an AI making "a billion more robots [to] take over the world." Yes. That's distinctly different from Skynet iterating on itself a billion times to make itself smarter , which AFAIK (I'm not up to date with full Terminatorverse, but then, most people aren't either), isn't something that happened in that story. > The popularity of that and similar sci-fi m…

[deleted]

Re: Large language models lack deep insights or a theory of mind

#225
post #148
post #133

Earlier quoted context omitted.

Ideally a recursive execution would also be a pure function - maybe a better way to put it about current LLMs is that they are a single mathematical expression being built up from a fix number of nodes and only addition and multiplication.

yes, except the "reasoning" process should also be able to look up facts (retrieval) and invoke external tools, making it non-pure.

Agree. I am not an expert on AGI, but I suspect this route is a dead end because I believe true AGI requires interaction with an external environment to learn eg there needs to be a feedback loop to external things that it can interact with and learn from.

“You learn best from your mistakes” comes to mind here.

And not just shallow feedback loops but bigger, more profound ones eg “I didn’t go to work for a month and so I was fired and and now I have no money and it will now be hard to be hired again”.

Re: Large language models lack deep insights or a theory of mind

#226
post #30

I think that if they would, that would be very surprising and indicative of a lot of wastefulness inside the model architecture. All these tests are simple single prompt experiments, so the LLM's get no chance to reason about their responses. They're just system 1 thinking, the equivalent of putting a gun to someone's head and asking them to solve a large division in 2 seconds. I bet a lot of these experiments would…

Just note that loop doesn't have to be visible from outside. It can be internal, with another driving thread asking right questions. Inner monologue. Then the summary is given back to user. This will give the model space for 'thinking' with internally generated text much large than the visible prompt + output. This way multi-step logic can be implemented.

> with another driving thread asking right questions. Inner monologue.

spoilers warning:

Is that basically the plot to westworld?

Re: Large language models lack deep insights or a theory of mind

#227

Earlier quoted context omitted.

Fundamentally, this "awareness" isn't defined by quieting the thinking. It is a description of fundamental reality. No individual should be able to experience it, and the "glimpses" are just forms of brain dysfunction. Meditation techniques that focus on breath or the body are an attempt to make you do the breathing/sensing consciously. If you film yourself and later look at what you did, you'll notice you aren't bre…

I think this is entirely incorrect. Vipassana meditation, the type focused on breathing, require intense awareness of your breathing and physical body. It’s a similar state to when you intensely focus on what’s around you and everything gets brighter and more vibrant and you pick out a lot of details you normally don’t notice because you’re distracted by your thoughts. If you’re doing it the way intended you would 10…

> It’s observing passively.

This is a byproduct of the process. You cannot observe passively if you still have a very strong imprint of a doer. When you start observing the tingles and the sensations, you still observe them from the center and there's a distance from the center and the sensations. The only way this center disappears is if the regions of the brain that create imprints lower their activity.

I'd say it is very easy to accomplish it when you're sleepy (especially after you wake up during the night and go back to sleep). There's no need to shut anything down through high focus, brain does it naturally to paralyze the body and lose the waking state.

"My Stroke of Insight" by Jill Bolte Taylor goes through many side-effects experienced by the meditators, but through a lens of a brain stroke.

> Thought ceases because it interferes with being aware, not because you are experiencing brain death.

Not sure about that. The verbal/visual thinking can be a strong feedback loop to the underlying unconscious processes. The only way I can see that it stops is if it is suppressed to unconscious, you've learned to not have it in focus. Also, many people have no visual and verbal thoughts. They don't have an imprint of standard modes of thinking at all. I'd say the definition of thought needs to expand to sensing. Any sensation is a thought. Focus on verbal/visual thoughts is misplaced.

I do agree that people learn to eventually do this without the side effects of practice. They can probably transition to a different brain activity without dealing with the breath ever again.

I think a much better standard for meditators is to ask them if they feel like their whole body is alien, or if they have something like alien hand syndrome. Losing thoughts, yet still being the body (or even the space around the body) is just another form of thoughts.

The imprint of a doer gives you the feeling of someone controlling the body or doing the thinking. For example, I can get lost in thoughts so hard that I no longer have imprints of controlling the body. Somehow I leave the shower and dress myself yet I know I don't remember doing any of it.

To also move the conclusion further, it sounds like the ultimate goal is to feel like you're not separate but that you're whole, so in that case it must be that this body, from that perspective, is fully without individual imprint, behaving completely like an input/output machine, no inner subjective experience present.

Re: Large language models lack deep insights or a theory of mind

#228

No LLMs don't think like people, they're architecturally incapable of doing so. They have, physically unlike humans no access to their own internal state and they're, save for a small context window, static systems. They also have no insights. There's a hilarious video about LLM Jailbreaks by Karpathy[1] from a week ago, where he shows how you can break model responses by asking the same question with a base64 string…

> What they've proven is that you can build functioning systems that perform intelligent tasks purely at the level of syntax. But there is no (or very little) understanding of semantics. That's not how LLMs work though, and I'm increasingly convinced that "syntax" and "semantics" are turning into annoyingly useless ideas people forget are descriptive , in the same way grammar books and dictionaries are descriptive .…

> big enough

Not even slightly. See Cantor's diagonal argument.

See also Plato's cave for syntax vs semantics.

Re: Large language models lack deep insights or a theory of mind

#229
post #156

> A chief goal of artificial intelligence is to build machines that think like people. Maybe that's their goal. But for many users of AI, the goal is to have easy and affordable access to a machine that, for some input (perhaps in a tightly constrained domain), gives us the output that we would expect from a high-functioning human being. When I use ChatGPT as a coding helper, I really don't care about its "theory of…

Telling it to write like (or outright that it is) Knuth might just get it to write more efficient algorithms as it were.

IIRC one of the prompting techniques developed this year was to ask the model who are some world class experts in a field and then have it write as if it was that group collaborating on the topic.

Re: Large language models lack deep insights or a theory of mind

#230
post #9

I appreciate this paper for relatively clearly stating what "human-like" might entail, which in this case involves "reasoning about the causes behind other people's behavior" which is "critical to navigate the social world" as outlined in this citation: https://www.sciencedirect.com/science/article/abs/pii/S00100... I get frustrated often when people argue "well, it isn't really intelligent" and then give examples th…

> I get frustrated often when people argue "well, it isn't really intelligent" and then give examples that are clearly dependent on our brain's chemical state and our bodies' existence in-the-physical-world.

A big part of the problem is that "is" has a wide variety of inconsistent meanings, and that this fact is sub-perceptual, and that it is culturally very inappropriate to comment on aspects of our culture like this, preventing knowledge of the problem from spreading.

/u/Swatcoder makes essentially the same point but in much more detail, though regarding less important words.

Post reply on HN