Artificial intelligence is not conscious – Ted Chiang

The philosophical arguments about what it means to be concious are so cagey. Are we more than our thoughts? Is being concious more than being a state machine being fed inputs and generating outputs? Are we more than a feedback machine? What types of animal nervous systems qualify?

"It can't be concious because we understand that it is just reacting in a simplistic way from simplistic inputs." So do other simple creatures. Some just react to light.

I can appreciate his comment that he sees it as more possible when they have inputs of their own (like emotions!). Perhaps his concern is that the entirety of the LLM model is frozen. It has no ability to have a subjective experience of its own. (he does literally say this in the article) It can be copied from one place to another, and (ignoring the nuance of operational details) -- it is largely the same "thing", and has no ability to change, which is definitely in the definition of alive, to say nothing of concious.

I think folks get hung up on "prediction". The prediction aspect is what is enabling emulation. How it does it is irrelvant. If something emulates human perfectly (or better, more human than human!) -- then it is probably concious. (but I agree that the inability to change and have a subjective experience are a pretty good argument against

Probably, Dijkstra would be right to say, "LLMs are no more concious than a submarine can swim." But I think he'd still be wrongfully dismissive of the larger question.

https://www.cs.utexas.edu/~EWD/transcriptions/EWD08xx/EWD867...

The fact that a LLM is essentially immutable would be my biggest argument against consciousness or self-awareness.

It's a big file with a bunch of coordinates describing spatial relationships between tokens. When you give it a prompt, it uses those relationships to generate a string of tokens that is a statistically likely response to that prompt, then it stops. It's not changed by the experience. It doesn't remember anything. It doesn't sit around thinking on its own.

Even if the model itself were extremely complex, it's hard to imagine a definition of consciousness that includes something that doesn't remember and can't change.

"I feel very strongly that I'm unique, therefore you are wrong" is a bad argument.

Consciousness is an extremely confusing, ambiguous topic, and no one has a good way to establish it, or even define it. But it seems to demand people make very strong statements about what is and isn't conscious, entirely driven by convenience and emotionalism. (Curiously, very few people who think that bags of chemicals and action potentials give an entity a conscious soul are eager to extend that to other animals, with broadly similar hardware.)

I've seen papers claim that there are anywhere from 12 to 40 competing definitions (https://philpapers.org/rec/VIMMAT), or, more accurately, there are something like 12 to 40 different aspects which all relate to "consciousness", which is very clearly a family resemblance category.

"Is X conscious or not" is an entirely unserious question today, unless this is just a headline followed by actual (and explicit) examination of the various aspects of consciousness being discussed. But, even still, LLMs are probably only conscious in like 2-3 or so ways, most of those meaning broadly "intelligence", i.e. reasoning, problem-solving, etc. When it comes to anything experiential or embodied, AI might eventually get more of these, but LLMs based on recursively applied linear algebra are clearly missing too many core aspects of consciousness to be considered conscious in any broad sense.

> "Moreover, it must be confessed that perception and that which depends upon it are inexplicable on mechanical grounds, that is to say, by means of figures and motions. And supposing there were a machine, so constructed as to think, feel, and have perception, it might be conceived as increased in size, while keeping the same proportions, so that one might go into it as into a mill. That being so, we should, on examining its interior, find only parts which work one upon another, and never anything by which to explain a perception."

- Monadology, Section 17

Conscious self-awareness is neither scale invariant nor independent of substrate. Computational theories will never account for it b/c computational abstractions are both scale invariant & substrate independent.