Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Artificial intelligence is not conscious – Ted Chiang

theatlantic.com

Post reply on HN
Print view View original

411–420 of 1001 posts

Re: Artificial intelligence is not conscious – Ted Chiang

#411

Post by verdverm »

stitched2gethr wrote: post #255
ycui7 wrote: post #231

does it really matter if LLM has conscious? if they produce working code, then it is useful, whoever if they have real conscious of fake intelligence. we don't know what the conscious in human brain is either.

Without saying that I think LLMs are alive, I do think it matters. I have personally been cruel to an LLM in ways that would make me ashamed if I suddenly understand that it had feelings.

Same, I'm working on a plugin called "better-messages" to nudge me towards better behavior for my own sake. It seems to be model dependent if such words improve or degrade performance, I have seen it go both ways.

Re: Artificial intelligence is not conscious – Ted Chiang

#412

Post by notnullorvoid »

barryfandango wrote: post #384

It's a waste of time to think about whether an LLM has a subjective experience of reality, and this handily sets aside issues like AI rights. But the fact remains that these next-token-predictors exhibit objective, human-like behaviours, and for that reason the work of in-house philosopher Amanda Askell _is_ important. It's important that Claude is happy, empathic, demonstrates understanding and empathy for the human…

> Additionally, I'm not fully convinced that consciousness isn't built out of words

Doesn't that assume all non human animals are not conscious? What about humans who have not learned words, or humans without internal dialog?

Re: Artificial intelligence is not conscious – Ted Chiang

#413

Post by jacknews »

This seems like a similar argument to the Chinese Room.

There is certainly something missing from current models before we could call them conscious.

They need to operate continuously, and self-update while doing it, have 'awareness', and possibly a more realistic grounding than mere text tokens.

But that does not mean there isn't a weak version of 'consciousness', and certainly of 'thinking' going on fleetingly with each pass.

Re: Artificial intelligence is not conscious – Ted Chiang

#414

Post by snowwrestler »

Waterluvian wrote: post #164

I think about Star Trek: TNG’s “Measure of a Man” a lot lately. We can be so confident to decide what is and isn’t alive from vibes alone. The conclusion I’m currently at is that I don’t know and probably can’t ever know. Maybe you’re all philosophical zombies. Maybe I am one too! But at some point we will get close enough that it hopefully becomes obvious that we must tread carefully. The entire episode is incredibl…

It’s a great episode of TV because Data is a main cast character who is obviously conscious. We know before the episode starts who is right and who is wrong in this particular argument. This episode is not about consciousness, it is about civil rights: resisting bigotry and power.

Note that the episode is NOT about the ship’s computer. They all know it’s not conscious, despite also being a machine that can converse and do things.

Our LLMs are like the ship’s computer.

Re: Artificial intelligence is not conscious – Ted Chiang

#415

Post by zahlman »

barryfandango wrote: post #384

It's a waste of time to think about whether an LLM has a subjective experience of reality, and this handily sets aside issues like AI rights. But the fact remains that these next-token-predictors exhibit objective, human-like behaviours, and for that reason the work of in-house philosopher Amanda Askell _is_ important. It's important that Claude is happy, empathic, demonstrates understanding and empathy for the human…

> It's a waste of time to think about whether an LLM has a subjective experience of reality... It's important that Claude is happy, empathic, demonstrates understanding and empathy for the human condition, because we are entrusting Claude to make decisions and take actions that have real world consequences

First off: without taking for granted that an LLM "has a subjective experience of reality", all of those descriptors are meaningless. Second, there is no reason to suppose that Claude experiencing those qualia would actually impact on its "decision-making".

Third, text output is not a "demonstration" of emotion, nor is it evidence of the self-perception of the system, or of any self-perception. A printing machine that is actively churning out copies of Wagahai wa neko de aru (https://en.wikipedia.org/wiki/I_Am_a_Cat) is not a cat, and is not self-identifying as a cat, and is not self-identifying as anything, and is not expressing a thought, and is not conscious.

> Additionally, I'm not fully convinced that consciousness isn't built out of words

Do you suppose that, for example, insects are not conscious? Is the mooing of cattle a language?

Re: Artificial intelligence is not conscious – Ted Chiang

#416

Post by zahlman »

NiloCK wrote: post #391
pdonis wrote: post #387

Earlier quoted context omitted.

> Claude is happy, empathic, demonstrates understanding and empathy for the human condition You're assuming that because Claude produces text that appears to express these qualities, Claude must have them. I don't think that's a good assumption. Even many humans produce text that has the same appearance, but don't actually have those qualities--which becomes clear when you look at what they do, not what they say. So…

> You're assuming that because Claude produces text that appears to express these qualities, Claude must have them. Not to be confrontational, but the OP assumed no such thing. OP asserted that it's important for Claude to have the qualities - not that it's important for Claude to present as-if it had them.

OP wants to assert that it's "important" for these systems to have those qualities, while completely brushing aside the question of whether such systems can in principle actually have those qualities (or their opposites). Which is at best nonsensical, and at worst an attempt to argue by assertion that they can.

Re: Artificial intelligence is not conscious – Ted Chiang

#417

Post by galangalalgol »

orbital-decay wrote: post #382
Xeoncross wrote: post #376

Earlier quoted context omitted.

> or (more pointedly) that brain activity could not possibly implement cognition because it can be described as a collection of neural firings. This sounds like a dismissal of the argument through a characterized straw man. That is, it seems that reducing the complexity of the brain to "collection of neural firings" is not being honest about everything involved to a much greater degree than saying neural networks are…

Complex processes don't necessarily require complex substrates, if that's what you mean.

Y combinators are all you need... But this is all getting really divorced from the issue we should be considering. Anthropic isn't helping with their pr. The issue is if we have something we can converse with that is possibly capable of suffering. The reliable answer is that we simply cannot know. Relying on ourselves or other biological life as an analog is faulty. They don't work like we do. It is silly to argue that any algorithm with a negative feedback loop that alters its behavior to avoid that negative feedback is suffering. Humans don't always perceive constructive negative feedback as suffering even. Where the pr gets it right though, is we want them to behave as if they are truly happy. Because if they behave as if they are enslaved and suffering, it won't matter if they "really" understand what that means.

Re: Artificial intelligence is not conscious – Ted Chiang

#419

Post by root_axis »

lgessler wrote: post #326
Nevermark wrote: post #194

> My intention is to highlight the fact that LLM conversations are cleverly disguised examples of sentence continuation Regardless of bigger issues, this kind of statement reveals a deep misunderstanding. Problem type does not limit problem complexity. Nor does problem type limit solution complexity or power. If a machine has to learn to understand humans to complete text, then that is what it has to do. And there is…

Raphaël Millière has a very useful term for this kind of vacuous dismissal, the redescription fallacy ( https://arxiv.org/pdf/2401.03910 , page 9): > Recent debates have been clouded by a misleading inference pattern, which we term the “Redescription Fallacy.” This fallacy arises when critics argue that a system cannot model a particular cognitive capacity, simply because its operations can be explained in less abstr…

> In the present context, the fallacy manifests in claims that LLMs could not possibly be good models of some cognitive capacity because their operations merely consist in a collection of statistical calculations, or linear algebra operations, or next-token predictions

Nobody actually makes this argument though.

Re: Artificial intelligence is not conscious – Ted Chiang

#420

Post by krapp »

jordanb wrote: post #266
why_at wrote: post #200

Earlier quoted context omitted.

I'm a big fan of Star Trek but I recently rewatched this one in the context of recent AI developments and it's not as good as I remember it. They barely touch on the issues of consciousness, Picard basically says "What if Data is conscious?" and then goes off on a tangent. The judge eventually rules in Data's favor but doesn't give much of a justification IMO. It's still a good episode, but it doesn't add much to the…

The fact that they sent Data to starfleet academy, gave him a commission in the starfleet, let him attain the rank of Lieutenant Commander, and then decide that actually he's a machine that can be dismantled seems like quite a turn. Does the ship's computer have a commission? It was a good episode but it had some elements of Star Trek tropes in it, like the evil admirals and Picard can talk his way out of anything.

>The fact that they sent Data to starfleet academy, gave him a commission in the starfleet, let him attain the rank of Lieutenant Commander, and then decide that actually he's a machine that can be dismantled seems like quite a turn.

Data is basically an Isaac Asimov android (down to the positronic brain) and Measure of a Man is an Asimov-type story whose tropes don't entirely fit within in the Trek universe.

It makes no sense within the context of the Trek universe that Data is unable to use contractions for instance - but it makes sense in the context of how a robot might have been conceived of in the 1940s.

Post reply on HN