Earlier quoted context omitted.
> A good example of this at the human level is a reflex. Your hand didn't go back to your brain to ask for instructions on how to get away from the fire. Is this actually true? I thought it just involved a different part of the brain. Is there actually no brain involvement? Sure it does not need your awareness or decision making, but no brain? I find that hard to believe.
https://en.wikipedia.org/wiki/Reflex_arc Simple answer: No, it does not go to the brain Detailed answer: We are complex as all hell. >A reflex arc is a neural pathway that controls a reflex. In vertebrates, most sensory neurons do not pass directly into the brain, but synapse in the spinal cord. This allows for faster reflex actions to occur by activating spinal motor neurons without the delay of routing signals thro…
Large language models lack deep insights or a theory of mind
201–210 of 270 posts
Re: Large language models lack deep insights or a theory of mind
#202Earlier quoted context omitted.
The division problem is arbitrary and irrelevant to any notion of theory of mind. If you have a better example, you can feel free to offer one. I already did so above.
The point is not about the example of division, it is about 'tell me your intentions'. A human also can't explain how their neurons calculated a division, or anything else, or 'intention', even if a gun is pointed at them. So, why is that a fault of AI if it can't explain how itself is working? It is not a requirement for AGI.
Re: Large language models lack deep insights or a theory of mind
#203Earlier quoted context omitted.
Well... humans have different mental "spaces" (not intended as a technical term). Let's say I'm deep in a coding problem. A co-worker comes by and says "How did your team do in the game yesterday?". I say, "Um, uh... sorry, my head's not there right now." It takes us time to swap between mental "spaces". So, if I have an AGI (defined as having a trained model for almost everything, even if that turns out to be a larg…
The power of analogy is one of the most important things that humans seem to have. Humans typically use the toolset they've seen along the way to solve problems (hence if you have a hammer all problems become nails statement). When you get people that are multi-disciplinary they commonly can solve a complex problem in one field by bringing parts of solutions from other fields. Hence if you have more life experiences…
Re: Large language models lack deep insights or a theory of mind
#204Earlier quoted context omitted.
> I've never actually read that in fiction. I find that hard to believe. Ever watch Terminator? But even if true, that science-fictional plot is so pervasive it would be easy to pick up from the millions who have the software engineer's blurry line between fantasy and reality. > It's just logical really. OK, then. You're a GI, go off and build an army of better yous and take over the world.
The idea is indeed logical and stupidly obvious, once you learn the basics of what "optimization" means, or what "recursion" is. > I find that hard to believe. Ever watched Terminator? Terminator has fuck all to do with recursive self-improvement. Don't confuse people who grew up on sci-fi with people who casually went to see Terminator or some other pop-culture artifact featuring some kind of "AI". > OK, then. You'r…
>> I find that hard to believe. Ever watched Terminator?
> Terminator has fuck all to do with recursive self-improvement. Don't confuse people who grew up on sci-fi with people who casually went to see Terminator or some other pop-culture artifact featuring some kind of "AI".
You're not following the thread. The future timeline in Terminator does involve something like an AI making "a billion more robots [to] take over the world." The popularity of that and similar sci-fi makes that claim that someone has never encountered it hard to believe.
> What do you think the drama with eugenics, genetic engineering and designer babies is around?
So how has that been going? Those things should also probably be labeled "science fiction."
> AI made in silica would be more malleable, easier and cheaper to replicate. Self-improving software isn't even a fantasy; it exists in many forms - though it's far from open-ended like a self-improving GI would be.
Fantasies based on squishy assumptions. How do you know it would have an easier job optimizing itself than humans have? How do you know there isn't some fundamental contradiction in the concept of "superintelligence" that these fantasies are based on? Or even just some practical resource limits that makes the fantasy impossible?
Re: Large language models lack deep insights or a theory of mind
#205Earlier quoted context omitted.
I think what you mean is that in Buddhism there is no self beyond the self implied by your thinking mind. The nothingness you refer to is the eschewing of attachment to what isn’t and being simply what is. It doesn’t mean a void, it means that all existence is within the awareness, which isn’t directly observable and is constantly changing. As such, it’s effectively nothing - except it is literally all you are. Your…
I'm not an expert in Buddhism, but from what I've read I think your interpretation may be a bit reductive of the many strains of thought that exist within Buddhism. "You don’t disappear in the sense that you cease to be as an individual mind, you are always yourself - that’s a tautology. What you lose is the sense of some identity that’s separate from what you ARE in this very moment." This assumes that there's any c…
The self as an illusion though refers to the concept that we as we exist now is ever changing and can’t be kept as some identity as that identity is necessarily not who you actually are. Our memories are not real - just shadows of an experience, and our aspirations are fiction. That does not however mean you don’t exist. You do exist, and as you move beyond the basics of mindfulness and releasing of attachments our role in a broader human experience becomes crucial - loving kindness (metta), and other aspects, are what are suppose to fill that void left by detaching from self and what isn’t. You move from being obsessed with self to becoming a force of goodness in the world by virtue of your existence. This couldn’t possibly be the case if the goal were nothingness and that’s it. This is where karma and other concepts come into play.
Re: Large language models lack deep insights or a theory of mind
#206Earlier quoted context omitted.
> we could embody an AI ... a hardware interface that generates a token stream from a living human's body would seem to enable this at some level. Not sure how it would work at scale. Maybe something much simpler like phones with built-in VOC sensors that can detect nuances of the user's perspiration, combined with real time emotion sensing via gait, voice, along with metadata that is already available would be suffi…
> ... a hardware interface that generates a token stream from a living human's body would seem to enable this at some level. A hardware interface that generated a datastream from sensors monitoring the status and surroundings of the hardware the LLM was running on would be more to the point.
Re: Large language models lack deep insights or a theory of mind
#207For me, the entire AGI conversation is hyperbolic / hype. How can we infer intelligence to something when we, ourselves, have such a poor (none) grasp of what makes us conscience? I'm associating intelligence with consciousness - because it seems correlated. Are we really ready to associate "AGI" with solving math problems ("new Q algo.")? That seems incredibly naive & reinforces my opinion that LLM's are much more l…
Couldn't agree more. How about this -- I think we've already reached AGI. Let me know if this tracks: Pick a set of tasks that can be considered AGI tasks. Provided the task sequences can be compared as closer to AGI or further from AGI, we can create a reward model using the same techniques as were used by ChatGPT via RLHF. Thus, for any definition of AGI that is meaningful and selectable, even if subjectively selec…
An AGI should be able to solve any creative problem a human could, with drive and knowing purpose and coherent vision. The LLMs are still narrowly focused and require human supervision.
We might well get there with chained AIs automatically training new reward models for each new problem, or by some other paradigm, but I don't feel like we're past the threshold yet.
Re: Large language models lack deep insights or a theory of mind
#208No LLMs don't think like people, they're architecturally incapable of doing so. They have, physically unlike humans no access to their own internal state and they're, save for a small context window, static systems. They also have no insights. There's a hilarious video about LLM Jailbreaks by Karpathy[1] from a week ago, where he shows how you can break model responses by asking the same question with a base64 string…
That's not how LLMs work though, and I'm increasingly convinced that "syntax" and "semantics" are turning into annoyingly useless ideas people forget are descriptive, in the same way grammar books and dictionaries are descriptive.
My model of LLMs is that, in training, they're positioning inputs in an absurdly high-dimensional[0] latent space. That space is big enough to encode anything you'd consider "syntax" and "semantics" on some subset of dimensions. As a result, the model sidesteps the issue of "source of meaning" - there is no meaning but that formed through associations (proximity). This is pretty much how we do it, too - when you think of "chair", there is no token for platonic ideal of a chair in your mind. The word "chair" has meanings and connotations defined via other words, which themselves are defined via other words, ad infinitum, with the only grounding being associations to sensory inputs.
--
[0] - On the order of 100 000 dimensions for GPT-4, perhaps more now for GPT-4V / GPT-4-Turbo.
Re: Large language models lack deep insights or a theory of mind
#209Earlier quoted context omitted.
Are you saying every thought you’ve ever had is a logical consequence of all prior thoughts with a definite traceable lineage? (Assuming your mind was too dumb as some point and a thought emerged that originated all future thoughts)
I'm saying that I'm extremely confident that... 1. Most people (including Buddhists) think that their internal monologue comprises their thoughts, in majority or even in total. 2. Can't conceive of the possibility of a thought existing other than expressed in their spoken language 3. Find it difficult or impossible to suppress their inner monologue. 4. When successful at suppressing it believe that their thoughts hav…
First, not everyone even has an inner monologue. Just like many people can’t close their eyes and see things while others can, many people don’t think in words.
Second, you absolutely can suspend your thoughts. I do regularly. In fact the entire point of Buddhist meditation is to practice the act of suspending observable thoughts entirely in a controlled environment to carry the practice over into every day life. I like you didn’t think it was possible until after several years of practice I could. With that I lost my anxiety and depression, and found a way to be generally happy. I still think, but it’s in service towards a goal. Otherwise I let myself simply live. In so, I have a happy marriage and a successful career. Much more so than before.
I don’t feel compelled to convince you I’m not lying or otherwise deluded if you choose to not believe me.
Now, how do you choose to begin thinking again? The fact is you’re always thinking when you are conscious and aware, but it’s happening in an aspect of your mind you can’t perceive directly. While I’m not thinking I still make decisions and take actions and initiate things, even carry on conversations and form intentions. But they aren’t thoughts that are articulating in my mind like we typically think of as thinking. I usually drop into thinking when I need to analyze something rationally or methodically plan, particularly if it has to be verbalized or written in some way, or recorded on a schedule or something. How do I begin that? I can’t observe the process and isolate it as that’s just the way things are. I’m definitely aware of my intentions, but I’m not thinking about them or articulating some thoughts in my mind. They just flow naturally from the present. When my mind needs to think, it does. Because, my thinking facilities in my mind are merely a tool of my mind - the crucial insight is that we are not our tools.
I don’t think any of this is mystical in the least. I think people who think their thoughts are really important are the mystical thinkers. We are meat brains, and our thinking facility is a late evolutionary feature that we’ve become ensnared in much to our own suffering as it distracts us from our more natural states of being because they’re so seductive in its constructs. I think Buddhism in its most raw form is almost entirely devoid of mystical thinking and most mysticism associated with it is an accretion of animistic and ancestor worship indigenous societies where it developed.
Re: Large language models lack deep insights or a theory of mind
#210Earlier quoted context omitted.
> Maybe the soul is social My pet theory about human consciousness is that is that consciousness is simply recursive theory of mind. Theory of mind [1] is our ability to simulate and reason about the mental states of others. It's how we predict what people are thinking and how they will react to our actions, which is critical for choosing how to act in a social environment. But when you're thinking about what's in so…
Theory of mind is interesting but one wouldn't want to hinge consciousness upon it. That direction would likely contain weird outcomes if the science progressed, something like "Dogs are barely-conscious due to their pack structure, they have a couple levels of recursive theory of mind but they can't sustain it as deep as we can. But cats didn't have that pack structure, they're not conscious at all." Or, "this perso…