The New AI Consciousness Paper

(Also, we definitely and obviously are a machine. But ignore that for now.)

- We can't even prove/disprove humans are consciousness

- Yes but we assume they are because very bad things happen when we don't

- Okay but we can extend that to other beings. See: factory farming (~80B caged animals per year).

- The best we can hope for is reasoning by analogy. "If human (mind) shaped, why not conscious?"

This paper is basically taking that to its logical conclusion. We assume humans are conscious, then we study their shape (neural structures), then we say "this is the shape that makes consciousness." Nevermind octopi evolved eyes independently, let alone intelligence. We'd have to study their structures too, right?

My question here is... why do people do bad things to the Sims? If people accepted solipsism ("only I am conscious"), would they start treating other people as badly as they do in The Sims? Is that what we're already doing with AIs?

So if we're eschewing the inelegance / "spooky magic" of dualism (and fair enough), we either have to start with subjectivity as primitive (idealism/pan-psychism), deriving matter as emergent (also spooky magic); or, try to concoct a monist model in which subjectivity can emerge from non-subjective building blocks. And while the latter very well might be the case, it's hard to imagine it could be falsifiable: if we constructed an AI or algo which exhibits verifiable evidence of subjectivity, how would we distinguish that from imitating such evidence? (`while (true) print "I am alive please don't shut me down"`).

If any conceivable imitation is necessarily also conscious, we arrive at IIT, that it is like something to be a thermostat. If that's the case, it's not exactly satisfying, and implies a level of spooky magic almost indistinguishable from idealism.

It sounds absurd to modern western ears, to think of Mind as a primitive to the Universe. But it's also just as magical and absurd that there exists anything at all, let alone a material reality so vast and ordered. We're left trying to reconcile two magics, both of whose existences would beggar belief, if not for the incontrovertible evidence of our subjectivity.

However, the point I'm making is that even assuming an agent/thing is capable of achieving consciousness, it would have to have a suitably complex environment and the capability of forming an independent feedback loop with that environment to even begin to display conscious capability.

If the agent/thing is capable of achieving consciousness but is not in a suitable environment, then we'd likely never see it doing things that resemble consciousness as we understand it. Which is something we have seen occur in the real world many times.

Re: qualia. Let’s put it aside briefly. It isn’t inconceivable that a system could construct representations that don’t correspond to an “objective” reality, i.e. a sort of reality hologram, as a tool to guide system behavior.

The key question to ask is: “construct representations for whom?”, or, to put the challenge directly, “it’s not surprising that an observer can be fooled. It’s surprising that there is an observer to fool”.

The world, in the standard understanding of physics, should be completely devoid of observers, even in cases where it instantiates performers, i.e. the I/O philosophical zombie most people know well by now.

To circle back around on why this is difficult: you have in front of you a HUD of constant perceived experience (which is meaningful even if you’re being fooled, i.e. cogito ergo sum). This has, through acculturation, become very mundane to you. But, given how we understand the rules of the world, if you direct your rationality onto the very lens through which you constantly perceive, you will find a very dark void of understanding that seems to defy the systems that otherwise serve you exceptionally well. This void is the hard problem.