Artificial intelligence is not conscious – Ted Chiang • Hacker News Board

Re: Artificial intelligence is not conscious – Ted Chiang

#941

Post by andrewla » Thu, Jun 04, 2026, 8:58 PM UTC

indoordin0saur wrote: post #793

I often hear the assumption that LLMs can or will become conscious because consciousness is likely substrate independent . The idea being that our brain is just a computer made out of meat and it doesn't do anything that can't be precisely simulated by a silicon computer. But I wonder if some of the magic in the human brain is its analog nature. Chemical signaling and impulses of neurons interact with each other with…

I think philosophically this comes down to the Universal Church-Turing Thesis.

If you believe that there is computation being done by nature/physics that is strictly more powerful than what a Turing machine is capable of, then consciousness might be beyond the reach of a silicon computer.

I personally believe that the UCTT is very likely to be true, not least because we have not yet proposed an oracle that is physically feasible and yet is beyond the capabilities of a Turing Machine.

That said, in the space of feasibility there are open questions -- even if P != NP, a non-deterministic Turing machine can be simulated by a deterministic one, just potentially with an exponential increase in computation time or space.

Re: Artificial intelligence is not conscious – Ted Chiang

#942

Post by gobdovan » Thu, Jun 04, 2026, 9:05 PM UTC

hardbass wrote: post #776

gobdovan wrote: post #27

Earlier quoted context omitted.

> I honestly don’t know what it even means to be conscious > subjugate other cultures (assuming you mean they're not conscious in other's minds) Have you ever considered you might be a philosophical zombie? [0] [0] https://en.wikipedia.org/wiki/Philosophical_zombie

Its unprovable and a meaningless concept. There is no observable evidence that can distinguish so called "p zombies" or so called "real persons". Might as well call it a "unicorn in space" or anything else meaningless.

Notice how you describe a perfectly intelligible concept like 'unicorn in space' as meaningless, even though there's a clear picture you can form in your mind about it and you can imagine the exact evidence that would convince you of their existence.

We can meaningfully talk about 'unicorns in space' since it's analytically intelligible and merely syntetically unverified.

Re: Artificial intelligence is not conscious – Ted Chiang

#943

Post by ryougi » Thu, Jun 04, 2026, 9:10 PM UTC

fnordpiglet wrote: post #538

overgard wrote: post #475

Earlier quoted context omitted.

But the machines don't understand. They predict. And what they predict is the next token. I'm not trying to beat this horse to death, but you have to realize using the word "understand" is anthropomorphising it. It's essentially the chinese room experiment -- if the rules are followed, no understanding is neccessary . If the tokens didn't correlate to words imbued with meaning outside the system, if the LLMs were tra…

However it’s disingenuous to say the inference is on the next token because it’s actually not, it’s in the models parameter space across a set of nonlinear activation functions then effectively projected into the token. The idea its predictive of the token isn’t actually the case, it really is a much more complex and more semantic relationship that ends in the series of tokens through the attention mechanism. The art…

>it’s disingenuous to say the inference is on the next token because it’s actually not, it’s in the models parameter space across a set of nonlinear activation functions then effectively projected into the token. The idea its predictive of the token isn’t actually the case, it really is a much more complex and more semantic relationship

Do you, or anyone reading, have any worthwhile links that make a strong case for this (that there is a stronger semantic relationship than simply next token prediction)? I would like to read more about this.

Re: Artificial intelligence is not conscious – Ted Chiang

#944

Post by glerk » Thu, Jun 04, 2026, 9:15 PM UTC

All I'm going to say is this is a very shallow analysis, and I expected more from the author of masterpieces like Exhalation. It's a good time to start discussing these topics, so it might still be valuable to publish this as a way to get the debate started.

Re: Artificial intelligence is not conscious – Ted Chiang

#945

Post by i5heu » Thu, Jun 04, 2026, 9:18 PM UTC

8bitsrule wrote: post #496

The chess-playing Mechanical Turk of 1770 seemed to have a consciousness to its viewers. The viewers were encouraged to think that it did. The Turk's human chess opponent knew that there was an actual human chess-player inside the box, along with levers and magnets. That illusion was profitable for 84 years. LLM's have no problems using expressions that make them sound human. The algos are demonstrably not human, and…

And humans just work with some electrons that are fired via neurons.

If you limit yourself to reductionism you will not see emergence and thus human consciousness is impossible for humans.

Re: Artificial intelligence is not conscious – Ted Chiang

#946

Post by i5heu » Thu, Jun 04, 2026, 9:29 PM UTC

xk3 wrote: post #883

JCTheDenthog wrote: post #842

Earlier quoted context omitted.

I mean, outside of our own internal experience of qualia, there's no way for us to prove anyone or anything is conscious. But my biggest issue with the debates about consciousness and LLMs is that the "LLMs are conscious" crowd usually has to presuppose a completely materialistic/mechanistic understanding of the universe and of our own consciousness, and that's something that we really don't know for certain. Also, i…

> a bunch of pipes and valves can be conscious Billions * billions of pipes and valves can result in emergent behavior that appears conscious while at the same time the sound of a single independent water pipe can moan and sound like human speech or otherwise lifelike and evoke human emotions. I think LLMs are doing both of these things and often people are more impressed by the independent fixtures (the moan) rather…

> What information goes out of the system which is novel?

I think there is very little truly novel information. Most information including the information of "breakthrough idea after mis-hearing someone" is just a mix of previous information.

I guess you are already aware of that... just for completness sake.

Re: Artificial intelligence is not conscious – Ted Chiang

#947

Post by aaroninsf » Thu, Jun 04, 2026, 9:37 PM UTC

CommieBobDole wrote: post #104

The fact that a LLM is essentially immutable would be my biggest argument against consciousness or self-awareness. It's a big file with a bunch of coordinates describing spatial relationships between tokens. When you give it a prompt, it uses those relationships to generate a string of tokens that is a statistically likely response to that prompt, then it stops. It's not changed by the experience. It doesn't remember…

That definition is in fact the predominant one today in serious circles: consciousness proper is not itself inclusive of the things which consider to define a continuous coherent self.

I.e. the "self" is not the same as what it means to experience consciousness.

There are for example well characterized examples of memory disruption under the influence of various drugs (e.g. as used intentionally in anesthesia); and neurological conditions which produce various kinds of amnesia.

Do these conditions mean someone is not conscious? We have the luxury of asking people directly.

More unsettling edges yet include things like so-called "split brain" patients or people suffering form serious psychological conditions like so-called "multiple personalities." Psychology does get great mileage out pathology!

Re: Artificial intelligence is not conscious – Ted Chiang

#948

Post by CraftingLinks » Thu, Jun 04, 2026, 9:51 PM UTC

nozzlegear wrote: post #468

dmitshur wrote: post #196

Earlier quoted context omitted.

Maybe time passes at a different rate for it, making it an easy "mistake" of not accounting for that for it to make.

Maybe it's just a dumb, unconscious machine

I spit out my tea. So dry.

Re: Artificial intelligence is not conscious – Ted Chiang

#949

Post by nonfamous » Thu, Jun 04, 2026, 9:56 PM UTC

pegasus wrote: post #923

nonfamous wrote: post #906

This entire essay basically boils down to one simple contention: LLMs can’t be conscious because only humans can be conscious and LLMs are not human. What an unimaginative argument: essentially ruling out the possibility of the title by definition. And the author fails to touch on the really interesting question posed by the article’s title: what if human consciousness is more like the working of LLMs than we think?

What? Where are you seeing such claims? I.e. that only humans can be conscious, that AI in general is not conscious by definition?? Seems to me you've put up an easy-to-dismiss strawman. Here's how I would summarize crux of the argument: LLMs (specifically) are, by construction, chameleons. More precisely, role-playing machines. When compelled to have a conversation as "themselves", all they do is take on the role of…

Err… right here in TFA?

>> The first requirement is that the computer program has a body (either physical or virtual) and sense organs

LLMs may be chameleons, but, to steal an excellent quote, if you can’t tell does it really matter.

My personal take on this is that consciousness is a thing induced, not a thing evoked. If it generates a response on the observer as a conscious thing would it is, by my definition, conscious.

Re: Artificial intelligence is not conscious – Ted Chiang

#950

Post by [deleted] » Thu, Jun 04, 2026, 10:07 PM UTC

partyficial wrote: post #920

nonfamous wrote: post #906

This entire essay basically boils down to one simple contention: LLMs can’t be conscious because only humans can be conscious and LLMs are not human. What an unimaginative argument: essentially ruling out the possibility of the title by definition. And the author fails to touch on the really interesting question posed by the article’s title: what if human consciousness is more like the working of LLMs than we think?

why does AI/LLM deserve special scrutiny regarding consciousness or the lack of? why not publish a paper stating - 'the radio is not conscious just because it can speak words'. or "the phone is not conscious because it can speak and listen".

[deleted]