This is one of those papers that uses a lot of big words to paper over the fact that it's really a philosophical opinion rather than a logical argument.
The Abstraction Fallacy: Why AI can simulate but not instantiate consciousness
11–20 of 157 posts
Re: The Abstraction Fallacy: Why AI can simulate but not instantiate consciousness
#12This is one of those papers that uses a lot of big words to paper over the fact that it's really a philosophical opinion rather than a logical argument.
From my point of view
The Jedi
Are not nice
Re: The Abstraction Fallacy: Why AI can simulate but not instantiate consciousness
#13I've attempted desperately to understand this paper after thoroughly reading it and have made 0 progress. Can anyone who does understand it attempt to explain?
Currently my understanding is that this paper is claiming that "concepts" are a fundamental building block of experience (which relates to consciousness), and can only be built by a mapmaker which is something that directly converts continuous physical phenomena into discrete tokens. But I couldn't get further into how that related to consciousness.
EDIT: the paper seems to be assuming that something simulating a mapmaker, or the process of doing it, can by nature not be a mapmaker since performing alphabetization is inherently something that must be "instantiated". How do they confirm if something is doing simulation vs if it's actually instantiating it? How can you tell the difference? They say how, much like simulating photosynthesis will not produce glucose, simulating mapmaking won't produce concepts. But you can't measure concepts, they're intangible, so you can't differentiate simulating mapmaking vs a real mapmaker.
Re: The Abstraction Fallacy: Why AI can simulate but not instantiate consciousness
#14I've attempted desperately to understand this paper after thoroughly reading it and have made 0 progress. Can anyone who does understand it attempt to explain? Currently my understanding is that this paper is claiming that "concepts" are a fundamental building block of experience (which relates to consciousness), and can only be built by a mapmaker which is something that directly converts continuous physical phenome…
They're defining consciousness ("mapmaker") to exist outside the AI, and then showing that AI can't meet their definition of consciousness.
Re: The Abstraction Fallacy: Why AI can simulate but not instantiate consciousness
#15Doesn't this still presume that we understand our own consciousness, in order to make the comparison. Where does our survival instinct come from? And why couldn't AI have one? >>>Additional Also, reproduction. Humans are basically just Food, Sex, Survival. And consciousness is just a rule set for fulfilling those goals. So if a NN, modeled on US, does develop the same rules, why can't it have the same degree of consc…
If AI has a survival instinct, then we should theoretically see evidence of it if we construct the right environment for AI to express it. Animals and cellular organisms demonstrate a survival instinct under the right conditions, so we would have to find equivalent conditions for a hypothetical machine intelligence.
Conversely, we know that if we take animals that do have a survival instinct and put them into the wrong kinds of environments, they will not thrive and will degenerate or possibly commit suicide. Similarly, if AI did have a survival instinct, do we think we've created an environment where that could be reasonably tested and observed?
Re: The Abstraction Fallacy: Why AI can simulate but not instantiate consciousness
#16Somewhat comically IMO,
the abstract very directly and literally denies the titular claim. It states:
> [consciousness] requires active, experiencing cognitive agent to alphabetize continuous physics into a finite set of meaningful states.
This may well be true—I think it is.
I also think that it is both widely understood and self-evident that the most promising path to machine consciousness, is via AI with continuous sensory input and agency, of which "world models" are getting a lot of attention.
When an AI system has phenomenology, the goal posts are going to start to resemble the God of the Gaps; at some point, critics will be arguing with systems which have a world model, a self model, agency, and literally and intrinsically understand the world not simply as symbolic tokens, but as symbolic tokens which are innately coupled to multi-modal representations of the things represented.
In other words, they will look—and increasingly, sound—a lot like us.
It's not that any of this is easy, nor that there is some paricular timeline, but it increasingly looks like "a mere question of engineering," and not blocked by fundamentals. It's blocked by the cost of computation and the limitations of our current model topologies.
But HN readers well know that the research frontier is far ahead of commercialized LLM, and moving fast.
An interesting time to be an agent with a phenomenology, is it not?
Re: The Abstraction Fallacy: Why AI can simulate but not instantiate consciousness
#17Yawn. We have no understanding of what consciousness actually is. Therefore attempting to prove whether a system can or cannot be conscious is something we can't prove or disprove at this point.
Re: The Abstraction Fallacy: Why AI can simulate but not instantiate consciousness
#18Doesn't this still presume that we understand our own consciousness, in order to make the comparison. Where does our survival instinct come from? And why couldn't AI have one? >>>Additional Also, reproduction. Humans are basically just Food, Sex, Survival. And consciousness is just a rule set for fulfilling those goals. So if a NN, modeled on US, does develop the same rules, why can't it have the same degree of consc…
The paper isn't saying "AI can't have one" it's saying (very approximately) that behavioral mimicry is not the path to one.
That is good point.
Just wondering, once an 'AI Model of Some Form', is in a Physical Body a 'robot', and is provided with some rules about survival so it doesn't fall into a hole. After a series of these events, does it matter? Does mimicry become reality, or no longer differentiable.
Kind of the philosophical zombie argument. If a robot can perfectly mimic a human, can you really know the internal state of the 'real' one is different from the 'mimicked' one.
Re: The Abstraction Fallacy: Why AI can simulate but not instantiate consciousness
#19Well, not sure whether humans have a consciousness, but very sure they want one.
Re: The Abstraction Fallacy: Why AI can simulate but not instantiate consciousness
#20Doesn't this still presume that we understand our own consciousness, in order to make the comparison. Where does our survival instinct come from? And why couldn't AI have one? >>>Additional Also, reproduction. Humans are basically just Food, Sex, Survival. And consciousness is just a rule set for fulfilling those goals. So if a NN, modeled on US, does develop the same rules, why can't it have the same degree of consc…
If AI has a survival instinct, then we should theoretically see evidence of it if we construct the right environment for AI to express it. Animals and cellular organisms demonstrate a survival instinct under the right conditions, so we would have to find equivalent conditions for a hypothetical machine intelligence. Conversely, we know that if we take animals that do have a survival instinct and put them into the wro…
I can make an AI system with a survival instinct right now. Of course, all that will do is make people tell me “it’s not a proper survival instinct” or move the goal posts and tell me I need yet some other property.
This whole endeavor is doomed from the beginning. There is no crucial test for “consciousness”, just ad hoc criteria people come up with to land on the conclusions that leave their belief system intact.
Consciousness is not a concept that can be rendered operational.