I was having a drunken discussion with the philosophy lecturer a few weeks back. He was making a very similar point. I kept saying it does it really matter? Lacking a theory of mind and deep insights describes 90% of all perfectly normal people. And perhaps training will be able to "fake it" (he went off on bold tangents about the definitions of this and that), or the language model will be an adjunct to some other model which does have these insights encoded or deducible, much like the human mind does. He wasn't convinced and I was too drunk. But it was basically feeling like: You can't feed carrots to a car like you can a horse, therefore cars are worthless.
Large language models lack deep insights or a theory of mind
161–170 of 270 posts
Re: Large language models lack deep insights or a theory of mind
#162> A chief goal of artificial intelligence is to build machines that think like people. Maybe that's their goal. But for many users of AI, the goal is to have easy and affordable access to a machine that, for some input (perhaps in a tightly constrained domain), gives us the output that we would expect from a high-functioning human being. When I use ChatGPT as a coding helper, I really don't care about its "theory of…
> insights are already as deep (actually more deep) as I get from most humans I ask for help
This was my thought as well. But then I figured if I can't get someone to give me thoughtful feedback, I might have bigger problems to solve.
Re: Large language models lack deep insights or a theory of mind
#163Earlier quoted context omitted.
I feel the "let's think about it step by step" is a bit of a hack. To circumvent the fact that there's no external loop you use the fact that it gets re-run on every token so you can store a bit of state in the tokens that it's already generated. Or am I misunderstanding something about that technique?
You are right, it’s sometimes called zero shot chain of thought, but it’s a way of getting the type of thing you are describing to happen. The LLMs somehow process things in a perceived step by step to get a much improved answer. Whether the external loop or an llm imposed internal loop, does it matter? Are our own minds looping or just adding tokens?
Yeah that's true, and I do believe there's a good chance our minds are perpetually adding tokens. But our minds also have an efficient/effective way of dealing with the context cut off. We don't have (or we don't experience) a hard cut off of our memory context. Instead the tokens are increasingly lossily compressed as they age out of our memory, the lossiness amount being based both on time passed but also on some fancy value function. And that combined with a "system" (or trained/fine-tuned in) prompt that motivates the LLM to reason in a way that is conducive to working with that kind of memory would be a sort of single-shot AGI system. Where single-shot is lying a bit because it's just infinitely looping.
I guess from that perspective it might make sense to test if such a thing is already happening in current LLM's and my dismissive attitude stems from the fact that I've played with them enough to know that they currently don't.
Re: Large language models lack deep insights or a theory of mind
#164> A chief goal of artificial intelligence is to build machines that think like people. Maybe that's their goal. But for many users of AI, the goal is to have easy and affordable access to a machine that, for some input (perhaps in a tightly constrained domain), gives us the output that we would expect from a high-functioning human being. When I use ChatGPT as a coding helper, I really don't care about its "theory of…
> insights are already as deep (actually more deep) as I get from most humans I ask for help This was my thought as well. But then I figured if I can't get someone to give me thoughtful feedback, I might have bigger problems to solve.
Or rather 30 different people on 30 different topics
Re: Large language models lack deep insights or a theory of mind
#165Here's my theory: Consider a typical LLM token vector used to train and interact with an LLM. Now imagine that other aspects of being human (sensory input, emotional input, physical body sensation, gut feelings, etc.) could be added as metadata to the the token stream, along with some kind of attention function that amplified or diminished the importance of those at any given time period -- all still represented as a…
So my observation is that we could embody an AI so that it learns theory of mind-body--but then we could remove the body. This gives a theory of mindful entity that does not need a body to exist. Then the next research step could be to study those properties so as to reconstruct/reproduce a theory of mind-body AI, without needing any embodiment process at all to obtain it. Is that, in principle, possible? It is uncle…
> we could embody an AI
... a hardware interface that generates a token stream from a living human's body would seem to enable this at some level.
Not sure how it would work at scale. Maybe something much simpler like phones with built-in VOC sensors that can detect nuances of the user's perspiration, combined with real time emotion sensing via gait, voice, along with metadata that is already available would be sufficient to produce such a token stream... who knows.
Re: Large language models lack deep insights or a theory of mind
#166I was having a drunken discussion with the philosophy lecturer a few weeks back. He was making a very similar point. I kept saying it does it really matter? Lacking a theory of mind and deep insights describes 90% of all perfectly normal people. And perhaps training will be able to "fake it" (he went off on bold tangents about the definitions of this and that), or the language model will be an adjunct to some other m…
To get into this analogy: this doesn't mean cars are worthless; it just means they're a poor approximation of a horse. Maybe you don't want to approximate a horse. But, if you do want to approximate a horse, don't try to do it with a car.
Similarly, if you want to approximate a human, an LLM may be the best we can do right now, but it's hardly a good approximation.
Re: Large language models lack deep insights or a theory of mind
#167> A chief goal of artificial intelligence is to build machines that think like people. Maybe that's their goal. But for many users of AI, the goal is to have easy and affordable access to a machine that, for some input (perhaps in a tightly constrained domain), gives us the output that we would expect from a high-functioning human being. When I use ChatGPT as a coding helper, I really don't care about its "theory of…
Look this is the only time I'll engage in this sort of discussion on HN[1], but first Donald Knuth is a real Human and it's extremely weird to position world class experts as something otherworldly. Second, suppose you got what you wished for (you used the "us" pronoun), is that not a sentient mind that you're forcing to do your labour? Does that not raise a ton of red flags in your ethics?
[1] normally I find HN discussions about what if chatGPT is human or "humans are just autocompletes" to be highschool-level scifi and cringe respectively
Re: Large language models lack deep insights or a theory of mind
#168I was having a drunken discussion with the philosophy lecturer a few weeks back. He was making a very similar point. I kept saying it does it really matter? Lacking a theory of mind and deep insights describes 90% of all perfectly normal people. And perhaps training will be able to "fake it" (he went off on bold tangents about the definitions of this and that), or the language model will be an adjunct to some other m…
To get into this analogy: this doesn't mean cars are worthless; it just means they're a poor approximation of a horse. Maybe you don't want to approximate a horse. But, if you do want to approximate a horse, don't try to do it with a car. Similarly, if you want to approximate a human, an LLM may be the best we can do right now, but it's hardly a good approximation.
Well the analogy was more at the introduction of the automobile the people who were familiar with horses were able to point out all the ways that horses were better than cars by some measure. Cars ended up being used in entirely different and arguably more powerful ways. You didn't even need to contradict the people who held the horses in higher reguard. Horses just became irrelevant. It's an incremental value proposition. AI will keep hitting various plateaus, but it's already pretty fucking amazing. It's not going to get worse. And pointing out specifically how it differs from the human mind to me honestly feels like clinging to the wreckage.
Re: Large language models lack deep insights or a theory of mind
#169Re: Large language models lack deep insights or a theory of mind
#170Earlier quoted context omitted.
This is a profound question but I also wonder if this non-thinking “awareness” you’re referring to is largely defined by quieting the thinking mind and listening to the senses more directly. A lot of meditation is about tuning out thoughts and focusing on proprioception like breathing, the feelings of the body, etc.
Fundamentally, this "awareness" isn't defined by quieting the thinking. It is a description of fundamental reality. No individual should be able to experience it, and the "glimpses" are just forms of brain dysfunction. Meditation techniques that focus on breath or the body are an attempt to make you do the breathing/sensing consciously. If you film yourself and later look at what you did, you'll notice you aren't bre…
I think this is entirely incorrect. Vipassana meditation, the type focused on breathing, require intense awareness of your breathing and physical body. It’s a similar state to when you intensely focus on what’s around you and everything gets brighter and more vibrant and you pick out a lot of details you normally don’t notice because you’re distracted by your thoughts.
If you’re doing it the way intended you would 100% be aware of your irregular breath or pausing. In fact beginners vipassana often advises counting the breaths individually in a cycle 1..10, and resetting the count when you lose track of your breathing. You intensely focus on the sensation of the air moving through your nostrils, the muscles contracting, your clothing shifting.
However it’s not about controlling your breathing, so it’s not the same as breathing consciously. It’s observing passively. Often you’ll notice that you are breathing irregularly, not because of the meditation, but because you typically are stressed and tight in your musculature due to the way you’re thinking. You can then loosen and reset your patterns of breath to be more natural, deep, and complete.
A goal isn’t to stop with observing the breath though, and you work towards having a total awareness of the entire body at once, shifting your center of existence from your head to the rest of your body. You then incorporate sounds and events in your environment. This requires an intense amount of mental power, and is entirely different from your description of oxygen deprivation. Thought ceases because it interferes with being aware, not because you are experiencing brain death.