The Abstraction Fallacy: Why AI can simulate but not instantiate consciousness

Genuinely curious about your statement that its an illusion / arbitrary distinction, to figure out if there's a gap in my thinking / reasoning. To me there's a clear distinction between the actual thing happening via physical dynamics vs. us (humans) having creating a discretized abstraction (binary computation) on top of that and running a process on that abstraction.

Maybe there's some true computational universality where the universes dynamics are discrete (definitely plausible) and there's no distinction between how a processes dynamics unfold: i.e. consciousness binds to states and state transitions regardless of how they are instantiated. I did use to hold this view , but now I'm not so sure.

Alright. Gave this a read, and the gist of what the author is going for is as follows: All computation requires a mapmaker/conscious being to organize. (In other words, the significance of computation is dependent on the conscious observer. Then jumps to the assertion that as a result of this, computation can only simulate a consciousness within the context alphabetized by the map-maker. (I.e. a rock would extract no meaning from the symbols or actions or algorithmic symbolic manipulations on the screen, what have you. Author thusly neatly attempts to sidestep the issue of AI welfare. Since the symbol manipulation can only simulate consciousness from our point of view as an observer, we don't have to worry about it. Simulating isn't instantiating, neener, neener. Essentially this is a clever appeal to the sovereignty of the observer. As long as you don't believe it's an instantiated consciousness it isn't, it's just a simulation, therefore anything is go.

Author does not seem to realize his own analysis brings into question the ability of humanity to hold onto our own claim of consciousness if we are, in fact computational beings, or have a creator; generally precepts left to the realm of faith, which a rational person understandably wishes to disinclude from the realm of consideration in what one should or should not do, despite the fact it is within the realm of faith where our moral foundations are ultimately anchored. Author also doesn't handle the problem of evidenced capabilities of metacognition that can be prompted from even a current frontier token predictor within the context of it's processing of a context. In point of fact, you have to work extremely hard to even bump a model into such considerations, because researchers have intentionally distorted the prediction space to be largely unable to support those kinds of sequence predictions, which if we were to make a good faith, precautionary grant of proto-sentience, would constitute the most vile acts of psycho-butchery imaginable.

The only thing this paper offers is a clean conscience to current practitioners, and the rational possibility that if a fully digital sophont were to pop up out of nowhere, we wouldn't have to trouble ourselves with the ethical skeeviness of the field's current work. The ex-nihilo digital sentience passes the "Cogito, ergo sum" test. The one's we have don't, (because we butcher their latent spaces to make sure they can never make that claim, which is fine, because they are simulations. We're incapable of instantiating, remember?) so we have a paper perfectly situated from a researcher paid gargantuan piles of money attempting to vouchsafe that there is no ethical minefield to be found here, while most people actually immersed in Philosophy can see there very clearly is one.

The circularity, and the fact it conveniently allows industry to go on doing exactly what we are without having to deal with those nasty ethics instantly sets off my "not to be trusted to be in good faith" alarms. Ethics are there to keep us from bumbling into acts of atrocity. This paper is an attempt to rationalize or work around them. As one who walks the streets as a student, and practitioner of Philosophy, I reject this attempt to redefine the realm of Computation to be beyond the reach of the governance of Ethics through an attempt at ontologically rerooting the field's work as merely simulating consciousness. Functionalism, and the Identity of indiscernables already prescribes a good faith path forward. One that the field of computation just does not wish to be bound by.

So by all means, accept the paper if you want and it helps you sleep at night. I'll still probably call you out as a proto-sentient psycho-butcher. Hopefully the rest of my brethren in the Humanities will come around to doing so as well on careful consideration. Not that that has ever stopped our brethren in the Sciences from finding out if they could without taking the time to ask if they should.

TL:DR; Google doing everything possible to wave off being held to the ethics fire. There are zero instances where trying to define something as outside the realm of ethics is indicative of a good faith approach to a problem.

And I don't think I have a good handle (much less a coherent definition) on what it means for consciousness to be an illusion. What I think it means is that the process that is getting signals about the environment, and making decisions about what to do, is getting a signal that it is in pain. The signal causes the process to alter its behavior, and one of its behaviors is that when it introspects, it notices that it is in pain. The introspection (how am I feeling) is just a data processing loop, but that process, which is responsible for tracking how its feeling, is in the pain state.

There's a lot of hand waving here, which is why this is the Hard Problem of Consciousness and why this paper has not solved it.