Live data from Hacker News

Artificial intelligence is not conscious – Ted Chiang

theatlantic.com

431–440 of 1001 posts

Re: Artificial intelligence is not conscious – Ted Chiang

#431

It's a waste of time to think about whether an LLM has a subjective experience of reality, and this handily sets aside issues like AI rights. But the fact remains that these next-token-predictors exhibit objective, human-like behaviours, and for that reason the work of in-house philosopher Amanda Askell _is_ important. It's important that Claude is happy, empathic, demonstrates understanding and empathy for the human…

What does it even mean for an llm to be happy? My dog can be happy, not my llm.

Re: Artificial intelligence is not conscious – Ted Chiang

#432
post #289

Earlier quoted context omitted.

> modern AI implemented via LLMs has almost none of the stronger or most core aspects of consciousness Can you elaborate on this? What are the specific "stronger and most core aspects of consciousness"? And why are you certain that they are stronger and "more core"?

If you are interested in some serious discussion, see https://lossfunk.com/papers/ai-consciousness.pdf , especially the early section "Consciousness as Family Resemblance". I suppose another is Ned Block on consciousness being a "mongrel concept", and the distinction between access vs. phenomenal The first paper picks out e.g. arousal/wakefulness, phenomenal quality / qualia, unity (how we feel sensory inputs and qua…

I've gone back and forth with AI on this stuff quite a bit, and there are many, many theories of consciousness, which is why when you were vague about the "more core" concepts, I asked for which ones specifically.

And I broadly disagree that the AI lacks things like qualia, self-reflexivity, and embodiment... at least that it lacks them any more than humans do.

Qualia: ultimately, all qualia are inputs and outputs, at least as far as modern science has been able to derive. There's nothing special about "hearing", it's just sound waves tickling some sensors which send some signals which trigger some neural pathways. Same for smell and sight, it's all just inputs being processed in different and efficient ways. An LLM only has token based input, but that's input nonetheless.

Self-reflexivity: an AI is capable of thinking about itself, and indeed papers have shown that larger models are capable of a self-awareness that can demonstrate that they realize when their weights have been manually tampered with, including being able to figure out how they were tampered with. The AI will quite literally output "you have injected 'ELEPHANT' into my weights" in some of the tests.

Embodiment: I don't know how one can confidently distinguish between an embodiment in a biological substrate vs a digital substrate. Both things actually exist at very high complexity in the real world. The substrate may be worlds different, but that alone doesn't suggest that one thing is conscious while the other isn't. You would need some missing 'magic' that we haven't yet discovered to truly understand.

In other words, I find it uncompelling that AI is clearly lacking any major aspects of consciousness that humans are clearly not lacking.

Re: Artificial intelligence is not conscious – Ted Chiang

#433

It's a waste of time to think about whether an LLM has a subjective experience of reality, and this handily sets aside issues like AI rights. But the fact remains that these next-token-predictors exhibit objective, human-like behaviours, and for that reason the work of in-house philosopher Amanda Askell _is_ important. It's important that Claude is happy, empathic, demonstrates understanding and empathy for the human…

Much of the improvements in the tools I use have been things that reinforce the machine elements of the token-based-reasoning-machine. Over time less they've been exhibiting a lot less "human-like" behavior. E.g. they get "lazy" far less than they used to.

(Perhaps they weren't lazy, but were working in spaces that corresponded to training data that said things like "and then repeat this for the next 20 examples"...)

And it's entirely unclear to me how a "happy" vs "sad" model would behave when given prompts generated by coding tool harnesses. Even maintaining "neutral" emotions in the face of the feedback/steering from the tool harnesses doesn't feel very "human."

Re: Artificial intelligence is not conscious – Ted Chiang

#434
post #200

I think about Star Trek: TNG’s “Measure of a Man” a lot lately. We can be so confident to decide what is and isn’t alive from vibes alone. The conclusion I’m currently at is that I don’t know and probably can’t ever know. Maybe you’re all philosophical zombies. Maybe I am one too! But at some point we will get close enough that it hopefully becomes obvious that we must tread carefully. The entire episode is incredibl…

I'm a big fan of Star Trek but I recently rewatched this one in the context of recent AI developments and it's not as good as I remember it. They barely touch on the issues of consciousness, Picard basically says "What if Data is conscious?" and then goes off on a tangent. The judge eventually rules in Data's favor but doesn't give much of a justification IMO. It's still a good episode, but it doesn't add much to the…

> it's not as good as I remember it. They barely touch on the issues of consciousness, Picard basically says "What if Data is conscious?" and then goes off on a tangent.

ST:TNG writing is generally like this. The show required considerable suspension of disbelief and a willingness to accept the kayfabe that deep concepts are being presented when for the most part they're just not that deep. (But it can be very enjoyable when you make those accommodations.)

Re: Artificial intelligence is not conscious – Ted Chiang

#435

Earlier quoted context omitted.

Raphaël Millière has a very useful term for this kind of vacuous dismissal, the redescription fallacy ( https://arxiv.org/pdf/2401.03910 , page 9): > Recent debates have been clouded by a misleading inference pattern, which we term the “Redescription Fallacy.” This fallacy arises when critics argue that a system cannot model a particular cognitive capacity, simply because its operations can be explained in less abstr…

> In the present context, the fallacy manifests in claims that LLMs could not possibly be good models of some cognitive capacity because their operations merely consist in a collection of statistical calculations, or linear algebra operations, or next-token predictions Nobody actually makes this argument though.

Are you serious? I hear it every single day, especially from computer scientists. There are top ranked posts here on HN _today_ with this argument.

Re: Artificial intelligence is not conscious – Ted Chiang

#436
post #227

Earlier quoted context omitted.

> Yeah, it's kind of mind boggling that Ted Chiang (of all people!) can't imagine intelligence without a body. and the whole thing just begs a lot of questions. How can you have a subjective experience without a body? That's the point, in my opinion: your physical/chemical state (body) in a given moment is then translated into the higher abstraction of the emotion. An emotion that *you* feel, because you are self awa…

How can you have a subjective experience without a body? Some anecdotal data. Many dreams I have are just of the computer screen of some coding problem. I think the problem could be x, so I try x. But I don't type the keyboard or anything, the code just magically appears as soon as I think of the solution. Then run the code (but no clicking) and it works or not. I feel in the dream success feeling or failure feeling…

What was the thing that was dreaming? The brain. Your brain is part of your body.

Re: Artificial intelligence is not conscious – Ted Chiang

#437

When the consciousness itself not understood and well defined in the first place, it is pretty pointless to debate if something is or isn't conscious. And here in particular the reasoning behind the argument is bizarre. Decomposing the complex activity into simple steps like 'predicting the next word' and claiming that surely can't have consciousness. A similar argument would be -- there is no way that movements of e…

A similar true argument!

Re: Artificial intelligence is not conscious – Ted Chiang

#438

The fact that a LLM is essentially immutable would be my biggest argument against consciousness or self-awareness. It's a big file with a bunch of coordinates describing spatial relationships between tokens. When you give it a prompt, it uses those relationships to generate a string of tokens that is a statistically likely response to that prompt, then it stops. It's not changed by the experience. It doesn't remember…

> It's not changed by the experience The entire file is not changed, but the KV cache is. > It doesn't remember anything The model definitely remembers previous exchanges within the same conversation.

> The model definitely remembers previous exchanges within the same conversation.

Christ HN isn't what it used to be

Re: Artificial intelligence is not conscious – Ted Chiang

#439
post #218

> My intention is to highlight the fact that LLM conversations are cleverly disguised examples of sentence continuation Regardless of bigger issues, this kind of statement reveals a deep misunderstanding. Problem type does not limit problem complexity. Nor does problem type limit solution complexity or power. If a machine has to learn to understand humans to complete text, then that is what it has to do. And there is…

Yeah. There are good arguments against LLM consciousness. This is not one of them. I'm hearing a lot of bad arguments against LLM consciousness lately. Bad argumentation heralds bad outcomes.

> Bad argumentation heralds bad outcomes.

What bad outcomes do you foresee from badly arguing against LLM consciousness?

Re: Artificial intelligence is not conscious – Ted Chiang

#440

> My intention is to highlight the fact that LLM conversations are cleverly disguised examples of sentence continuation Regardless of bigger issues, this kind of statement reveals a deep misunderstanding. Problem type does not limit problem complexity. Nor does problem type limit solution complexity or power. If a machine has to learn to understand humans to complete text, then that is what it has to do. And there is…

Come on, I invented parts of this technology at Google and am baffled why this is debated. We discovered math that decodes data storage in langauge and is able to use sophisticated continuation cohorts from ALL OF HUMAN RECORDED KNOWLEDGE to respond to you in a call/response model with very good synthesis capabilities. Its super useful, but not life or conciousness. Its a simulated echo from our collective recorded b…

I have no technical expertise re. LLM’s but from my intuition I came to this same conclusion.

It’s strange many others have not eh? I think when new developments arise, ironically, this is the true measure of human intelligence - one’s ability to make sense of a thing and be closest to the truth.

Post reply on HN