Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Artificial intelligence is not conscious – Ted Chiang

theatlantic.com

Post reply on HN
Print view View original

401–410 of 1001 posts

Re: Artificial intelligence is not conscious – Ted Chiang

#401

Post by DennisP »

barryfandango wrote: post #384

It's a waste of time to think about whether an LLM has a subjective experience of reality, and this handily sets aside issues like AI rights. But the fact remains that these next-token-predictors exhibit objective, human-like behaviours, and for that reason the work of in-house philosopher Amanda Askell _is_ important. It's important that Claude is happy, empathic, demonstrates understanding and empathy for the human…

So if I don't think any words for a few seconds, am I not conscious?

Suppose I'm an advanced meditator and maintain that state for hours?

Re: Artificial intelligence is not conscious – Ted Chiang

#402

Post by pdonis »

fjdjshsh wrote: post #390
pdonis wrote: post #387

Earlier quoted context omitted.

> Claude is happy, empathic, demonstrates understanding and empathy for the human condition You're assuming that because Claude produces text that appears to express these qualities, Claude must have them. I don't think that's a good assumption. Even many humans produce text that has the same appearance, but don't actually have those qualities--which becomes clear when you look at what they do, not what they say. So…

Read parent's post carefully. The post starts by saying that discussing whether they have subjective emotion is a waste of time, so the post is definitely NOT saying that Claude has emotions.

> The post starts by saying that discussing whether they have subjective emotion is a waste of time

"Empathy and understanding for the human condition" is not an emotion. As the post I responded to said, it's an objective thing, not subjective.

Re: Artificial intelligence is not conscious – Ted Chiang

#403

Post by Isamu »

Nevermark wrote: post #194

> My intention is to highlight the fact that LLM conversations are cleverly disguised examples of sentence continuation Regardless of bigger issues, this kind of statement reveals a deep misunderstanding. Problem type does not limit problem complexity. Nor does problem type limit solution complexity or power. If a machine has to learn to understand humans to complete text, then that is what it has to do. And there is…

>If a machine has to learn to understand humans to complete text, then that is what it has to do.

A language model completes text based on the overlapping patterns of the training data.

There absolutely was thinking involved… in the training data. Same as when you read a book, you engage with the thinking behind the text. The book isn’t thinking, and the author may be dead and gone, but there’s absolutely the traces of thinking in the text.

Language models produce mashups of texts they were trained on, and there’s absolutely the traces of thoughts behind those mashups.

Re: Artificial intelligence is not conscious – Ted Chiang

#404

Post by famouswaffles »

Terr_ wrote: post #385
cauch wrote: post #254

Earlier quoted context omitted.

I think, for me, the thing is that when you do basic ML, you discover that ML will very often find data pattern that fit the goal but does not correspond to a real mechanism. So, I think there is a flaw in the logic of saying that human text have a pattern of "consciousness mechanism" and therefore LLM will learn "consciousness mechanism" in order to return sentence continuation that is convincing. There is probably…

> a flaw in the logic [...] mechanism Similar to: "Birds fly, my spinning helical device flies, therefore we've started to replicate how birds fly." > without having to build elements that one expect on a conscious being One of the elements I expect in a conscious being is that you can't rewrite it by changing the introductory paragraph. When it comes to LLMs, almost every "mind" we humans perceive is a fictional cha…

>One of the elements I expect in a conscious being is that you can't rewrite it by changing the introductory paragraph.

If we had enough knowledge of the workings of the human brain, you could alter the perception of every single memory you've ever had. And limited versions of this already happen all the time. Human memory is notoriously unreliable for a reason.

Are you aware of the Recovered Memory Therapy Scandals of the 80s/90s ? Boy did that ruin a lot of lives. You can rewrite a human by changing their 'introductory paragraph'. It's just not as accessible.

Re: Artificial intelligence is not conscious – Ted Chiang

#405

Post by orbital-decay »

vmg12 wrote: post #88

People are constantly talking past eachother when they discuss this. Is there even a concrete definition of consciousness? When people talk about consciousness it's more than just self-awareness. It's self awareness + sensory stimulus + emotions + some level of intelligence. Now onto AI: I don't even think it's self aware. Notice how if you ask an AI to estimate how long a certain task will take, it estimates arbitra…

I agree with the first part but your framing still relies on ill-defined terms. What is your definition of self-awareness? Intelligence? Knowledge?

I suspect that if you attempt to rigorously define consciousness all the way down without handwaving, you might discover that it doesn't exist after all, or just decompose it into low-level abstractions while having the original meaning slip away (which is the same).

You may also want to look at functional equivalence analogies provided by mechinterp and functional anatomy of large models (not necessarily language ones). Evolutionary analogies as well.

Re: Artificial intelligence is not conscious – Ted Chiang

#406

Post by pdonis »

NiloCK wrote: post #391
pdonis wrote: post #387

Earlier quoted context omitted.

> Claude is happy, empathic, demonstrates understanding and empathy for the human condition You're assuming that because Claude produces text that appears to express these qualities, Claude must have them. I don't think that's a good assumption. Even many humans produce text that has the same appearance, but don't actually have those qualities--which becomes clear when you look at what they do, not what they say. So…

> You're assuming that because Claude produces text that appears to express these qualities, Claude must have them. Not to be confrontational, but the OP assumed no such thing. OP asserted that it's important for Claude to have the qualities - not that it's important for Claude to present as-if it had them.

The OP said Claude and similar LLMs "exhibit objective, human-like behaviours". That's a claim about what is true, not about what is important. That's the claim I'm disputing: we don't have evidence that Claude exhibits such behaviors, we only have evidence that it produces text that is similar to the text humans produce when they claim to exhibit such behaviors. Which is not good evidence, for the reasons I gave.

Re: Artificial intelligence is not conscious – Ted Chiang

#407

Post by sfn42 »

JoshTriplett wrote: post #278
gabrieledarrigo wrote: post #147

Earlier quoted context omitted.

> Yeah, it's kind of mind boggling that Ted Chiang (of all people!) can't imagine intelligence without a body. and the whole thing just begs a lot of questions. How can you have a subjective experience without a body? That's the point, in my opinion: your physical/chemical state (body) in a given moment is then translated into the higher abstraction of the emotion. An emotion that *you* feel, because you are self awa…

> How can you have a subjective experience without a body? If every neuron of your brain were simulated precisely on a sufficiently powerful computer, that simulation would have subjective experiences, without having a body.

Citation needed.

Also that's impossible. It is impossible to simulate reality exactly using digital computers. The best we can do is approximate. Doesn't matter how powerful it gets, it'll always just be an approximation.

Re: Artificial intelligence is not conscious – Ted Chiang

#408

Post by CommieBobDole »

in-silico wrote: post #366
CommieBobDole wrote: post #104

The fact that a LLM is essentially immutable would be my biggest argument against consciousness or self-awareness. It's a big file with a bunch of coordinates describing spatial relationships between tokens. When you give it a prompt, it uses those relationships to generate a string of tokens that is a statistically likely response to that prompt, then it stops. It's not changed by the experience. It doesn't remember…

> It's not changed by the experience The entire file is not changed, but the KV cache is. > It doesn't remember anything The model definitely remembers previous exchanges within the same conversation.

Right, but that's still external to the LLM, it's just a KV cache that's stored on the provider side for performance reasons, so that the client doesn't have to re-send the whole chat history with every subsequent call in the conversation.

It still generates every response using the model's pristine state with every new API call; whether the context is provided from the client or from a colocated cache server doesn't really change that.

Re: Artificial intelligence is not conscious – Ted Chiang

#409

Post by verdverm »

ycui7 wrote: post #231

does it really matter if LLM has conscious? if they produce working code, then it is useful, whoever if they have real conscious of fake intelligence. we don't know what the conscious in human brain is either.

What do we do when someone wants to marry their Ai companion?

If it produces words of love, those that stir emotion and make the human happy, should we let them live on in a delusion that the machine also feels love? Does the machine have different preferences? Do they get in fights?

Should such a machine have the same legal rights? Do they get more tax deductions? Can they adopt a human?

I think it important that we humans have some rational discussion on these Ai tools that are really quite good at simulating emotions and providing frictionless relationships. If you though social media was bad for people...

Re: Artificial intelligence is not conscious – Ted Chiang

#410

Post by hackinthebochs »

psvv wrote: post #283
empath75 wrote: post #157

Earlier quoted context omitted.

Claude once said to me: "After six months we have made no progress on this and I think we should reconsider another option" and I was like my dude we have been at this for only 2 hours.

I like this anecdote because it gets at how words to an LLM have no connection to their real concepts. To an LLM, words are simply numbers arranged in a likely pattern.

Of course for humans words have no inherent meaning either, they're just sequences of characters or patterns of sounds. It is what words are associated with that carries meaning. A large part of this is how words relate to other words. LLMs can capture this in principle. What LLMs lack is the direct association of a word with sensory experience. But it's an open question how relevant this is in practice to understanding.
Post reply on HN