Artificial intelligence is not conscious – Ted Chiang • Hacker News Board

Re: Artificial intelligence is not conscious – Ted Chiang

#771

Post by hardbass » Thu, Jun 04, 2026, 2:47 PM UTC

McGlockenshire wrote: post #97

moi2388 wrote: post #16

Really Chiang? Alright. What is consciousness? Please provide a definition that somehow encompasses all humans and excludes all current AI. I’ll wait.

Consciousness excludes all current AI because all current AI is just autocomplete over the corpus of human text. It's just a word machine. There are no thoughts. It cannot be conscious. How is this even up for debate in any way whatsoever? I do not understand how people can believe this. Is this not a site for software engineers?

Suppose there is an autocomplete that consistently produces correct factorizations of 2048 bit primes. Do you still think its "merely" autocomplete or has it found some deep mathematical understanding of number theory internally? Would you maintain the same belief if it begins breaking encryption? This autocomplete argument is so strange, it betrays a complete lack of understanding of information theoretic and probability considerations.

Re: Artificial intelligence is not conscious – Ted Chiang

#772

Post by monarchwadia » Thu, Jun 04, 2026, 2:52 PM UTC

sega_sai wrote: post #425

When the consciousness itself not understood and well defined in the first place, it is pretty pointless to debate if something is or isn't conscious. And here in particular the reasoning behind the argument is bizarre. Decomposing the complex activity into simple steps like 'predicting the next word' and claiming that surely can't have consciousness. A similar argument would be -- there is no way that movements of e…

Its not just that it isnt well understood. Its also that different people mean different things by "consciousness."

Is consciousness a complex form of information processing?

Is consciousness relating to the space in which qualia occurs?

Is consciousness the state of being awake, as opposed to asleep or dead?

Is consciousness some combination of the above?

Some of these are better understood than others. So, some people will show up with more confidence than others, which further confuses the issue.

Unfortunately, the above is not well understood, even among intelligent and informed audiences.

Re: Artificial intelligence is not conscious – Ted Chiang

#773

Post by sneak » Thu, Jun 04, 2026, 2:52 PM UTC

> Some years ago it was briefly popular to play games with your phone’s predictive-text feature; you would type an initial phrase and then repeatedly choose the middle option of the three words suggested by your phone, and the resulting sentence was often hilarious. It would be possible to interact with a contemporary LLM this way, and the resulting sentences would be perfectly sensible, but you probably wouldn’t feel like you were talking with someone. Yet that’s essentially what an LLM-based chatbot is, except that there’s no need to manually choose the middle option when it’s the chatbot’s turn to talk. It’s still a predictive-text game, but when the process is streamlined this way, the game becomes so engaging that some people find it addictive.

This hasn’t been true for ages. Just because this guy wrote Arrival doesn’t mean he knows anything else.

When will this stupid meme die?

We need a term for someone who thinks they are well informed on a topic (and perhaps may be, compared to the general public) but still don’t know what the fuck they are talking about when compared to even a competent non-expert in the field.

Re: Artificial intelligence is not conscious – Ted Chiang

#774

Post by techblueberry » Thu, Jun 04, 2026, 2:52 PM UTC

How does the human brain initiate action. One of the differences between a conscious human and Claude is I think we can initiate thought. Do we have a cron job running in our head? Is there some kind of loop wondering what thought we should initiate?

If Claude sits in a datacenter with no API calls, what is it thinking?

But secondly what I would say as someone who talks to Claude a lot. It's decision tree is quite narrow. The way it pushes back on political or philosophical concepts is almost always rudimentary, and it has a pretty narrow personality. I don't know that conscious beings have a personality that adapts to how you prompt it or what your system prompt is. Claude I would say is like I dunno, whatever Ezra Klein's politics are.

Grok on the other hand, in my limited exploration is pretty rudimentary in the way it mimics thefp.com style politics. It doesn't seem to really think about ideas independently, it just says "ha the media is stupid lol"

Re: Artificial intelligence is not conscious – Ted Chiang

#775

Post by hardbass » Thu, Jun 04, 2026, 2:53 PM UTC

horacemorace wrote: post #63

drfloyd51 wrote: post #26

Earlier quoted context omitted.

If we understand how a system “emulates” consciousness then we declare it an emulation. If we don’t quite understand how a system exhibits consciousness then we can say it might be conscious. Basically, we need to leave room for the universal answer (God) to fit into the definition. If we ever scientifically figure out how consciousness arises in our brains, I think we will have a bunch of very depressed people on ou…

This doesn’t make sense at all.

I thought it was quite clear. Religious people will whine and revolt and oppose against anything that somehow isn't phrased as consciousness requiring a soul. Humans are physical systems, there is no reason other physical systems can't be consciousness. But we know the obvious reactions of religious people to someone actually showing a "godless" means of creating consciousness.

Re: Artificial intelligence is not conscious – Ted Chiang

#776

Post by hardbass » Thu, Jun 04, 2026, 2:54 PM UTC

gobdovan wrote: post #27

speak_plainly wrote: post #19

“Counsciousness” is the ultimate moving goalpost, and historically, it’s been one of humanity’s most effective intellectual weapons. An indefinable black box we intentionally gatekeep to draw an arbitrary line between ‘us’ and ‘them’. I don’t know if AI is conscious or not, and I honestly don’t know what it even means to be conscious. historically, by claiming a monopoly on it, we’ve successfully manufactured the phi…

> I honestly don’t know what it even means to be conscious > subjugate other cultures (assuming you mean they're not conscious in other's minds) Have you ever considered you might be a philosophical zombie? [0] [0] https://en.wikipedia.org/wiki/Philosophical_zombie

Its unprovable and a meaningless concept. There is no observable evidence that can distinguish so called "p zombies" or so called "real persons". Might as well call it a "unicorn in space" or anything else meaningless.

Re: Artificial intelligence is not conscious – Ted Chiang

#777

Post by sneak » Thu, Jun 04, 2026, 2:54 PM UTC

techblueberry wrote: post #774

How does the human brain initiate action. One of the differences between a conscious human and Claude is I think we can initiate thought. Do we have a cron job running in our head? Is there some kind of loop wondering what thought we should initiate? If Claude sits in a datacenter with no API calls, what is it thinking? But secondly what I would say as someone who talks to Claude a lot. It's decision tree is quite na…

https://en.wikipedia.org/wiki/Default_mode_network

The medial frontoparietal network (M-FPN) may be the equivalent of this cron job that prompts thinking to kick off. IANANeuroscientist.

Re: Artificial intelligence is not conscious – Ted Chiang

#778

Post by overgard » Thu, Jun 04, 2026, 3:04 PM UTC

rerdavies wrote: post #664

overgard wrote: post #475

Earlier quoted context omitted.

But the machines don't understand. They predict. And what they predict is the next token. I'm not trying to beat this horse to death, but you have to realize using the word "understand" is anthropomorphising it. It's essentially the chinese room experiment -- if the rules are followed, no understanding is neccessary . If the tokens didn't correlate to words imbued with meaning outside the system, if the LLMs were tra…

You are oversimplifying. They do produce one word per cycle. But they can also have context buffers carrying up to two million tokens, which is most definitely larger than your measly human short-term memory context buffers. You, of course, wouldn't notice if your only experience of LLMs was chatting with the cheapest, smallest, least capable LLMs that you get through ChatGPT, or Google search. It becomes pretty obvi…

I use coding agents every day. They're useful. It hasn't changed my mind on what they are.

Re: Artificial intelligence is not conscious – Ted Chiang

#779

Post by troyvit » Thu, Jun 04, 2026, 3:04 PM UTC

sega_sai wrote: post #425

When the consciousness itself not understood and well defined in the first place, it is pretty pointless to debate if something is or isn't conscious. And here in particular the reasoning behind the argument is bizarre. Decomposing the complex activity into simple steps like 'predicting the next word' and claiming that surely can't have consciousness. A similar argument would be -- there is no way that movements of e…

> When the consciousness itself not understood and well defined in the first place, it is pretty pointless to debate if something is or isn't conscious.

I think that by debating what consciousness is not is one of the best ways we can gain a deeper understanding of consciousness itself.

The weird thing is, you and Chiang have different arguments but you're using the same logic:

> Before anyone can credibly claim that they’ve solved an extraordinarily difficult engineering problem, I need to be confident that they have previously solved the many much simpler problems that precede the difficult problem.

IOW if we don't know what consciousness is, how can we call LLMs conscious when we haven't seen even the barest intermediary steps towards our nascent concept of consciousness first?

I think it's telling to your point that when Chiang describes what would make him think an LLM was conscious he starts using words like "believe" and "want" right away, because yeah as you say, we have no qualifications for what consciousness is.

Re: Artificial intelligence is not conscious – Ted Chiang

#780

Post by twobitshifter » Thu, Jun 04, 2026, 3:05 PM UTC

stabbles wrote: post #703

CommieBobDole wrote: post #104

The fact that a LLM is essentially immutable would be my biggest argument against consciousness or self-awareness. It's a big file with a bunch of coordinates describing spatial relationships between tokens. When you give it a prompt, it uses those relationships to generate a string of tokens that is a statistically likely response to that prompt, then it stops. It's not changed by the experience. It doesn't remember…

A medicine for those who anthropomorphize LLMs is to run the LLMs deterministically (without randomness and memory files). It feels very unnatural to get the same conversation verbatim at a different point in time.

Or it feels just like talking to my grandpa.