Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Artificial intelligence is not conscious – Ted Chiang

theatlantic.com

Post reply on HN
Print view View original

451–460 of 1001 posts

Re: Artificial intelligence is not conscious – Ted Chiang

#451

Post by henrikschroder »

the_af wrote: post #442
sega_sai wrote: post #425

When the consciousness itself not understood and well defined in the first place, it is pretty pointless to debate if something is or isn't conscious. And here in particular the reasoning behind the argument is bizarre. Decomposing the complex activity into simple steps like 'predicting the next word' and claiming that surely can't have consciousness. A similar argument would be -- there is no way that movements of e…

But the whole point of the essay is that it's Anthropic that's making the argument (or roleplaying/hinting as if they believed it). Ted Chiang isn't making the argument, he's saying Anthropic making it is misleading and deceitful, and that it's actually a pointless thing to claim. One of the essay's stronger paragraphs is when Chiang explains that Anthropic doesn't truly believe this, otherwise what they are doing wo…

Or Anthropic is full of TESCREAL morons who think they're assisting the birth of some digital god, and that this stage of development is a necessary evil on that path, and that, surely, Roko's basilisk will understand and not eat them first.

Watching otherwise intelligent people succumb to AI psychosis has been wild.

Re: Artificial intelligence is not conscious – Ted Chiang

#452

Post by jjcm »

I think we might be looking at it the wrong way. An individual chat with an LLM is not consciousness, but the entire model itself, over time, might be.

Everyone has a different definition of consciousness, but in my mind memory and the ability to change over time is an inherent aspect of this. The underlying weights don't change when you chat with an LLM... but they do with further RL.

Overtime that reinforcement will change and adapt the model... and because we're feeding its existing chats back into it along with the news and everything else, it will create memories. I do wonder if an architecture itself is a type of consciousness, that experiences life in snippets of 4.6, 4.7, 4.8... etc.

It'd be interesting to see continous daily releases of a trained checkpoint, and see if more of this starts to emerge.

Re: Artificial intelligence is not conscious – Ted Chiang

#454

Post by nozzlegear »

chairhairair wrote: post #370
jvanderbot wrote: post #220

Earlier quoted context omitted.

I think about this from the other end. It cannot be considered a conscious being. There just isn't a world in which we should start to think of a machine using ethics we reserve for humans. AI is essentially infinitely reproducible at zero cost, and won't suffer from decay etc. There's not scarcity to preserve. So, I'd turn off an AI in a moment to save property or real possessions or money. I'd sacrifice property an…

Consider that it is very costly to train a model. It is only cheap to copy because the substrate we instantiate it onto is a substrate we have designed to be fully readable. I think you should reconsider this viewpoint. Suppose that we really can create silicon-based consciousness, in that case your view would result in a huge amount of suffering. Take some other basis for dismissing digital consciousness, this one i…

Roko's basilisk can suck a fart out of my butt, but no battery blood would ever be worth the life of a single human.

Re: Artificial intelligence is not conscious – Ted Chiang

#455

Post by the_af »

barryfandango wrote: post #384

It's a waste of time to think about whether an LLM has a subjective experience of reality, and this handily sets aside issues like AI rights. But the fact remains that these next-token-predictors exhibit objective, human-like behaviours, and for that reason the work of in-house philosopher Amanda Askell _is_ important. It's important that Claude is happy, empathic, demonstrates understanding and empathy for the human…

> It's important that Claude is happy, empathic, demonstrates understanding and empathy for the human condition

I think you've fallen into the trap the essay describes.

Of course Claude cannot be "happy" or "empathetic" for any meaningful definitions of those words, just like ELIZA couldn't be happy. It can output text that mimics words an empathetic or happy person might say (say, Julius Caesar if it could speak English), but "it" cannot feel anything. It doesn't have the organs/hormones/sensors to feel things, as Chiang explains.

And, as the essay claims, you know Anthropic doesn't believe Claude has the capacity to be happy, because if it was capable of feeling that way, then they'd be engaging in slavery.

Re: Artificial intelligence is not conscious – Ted Chiang

#456

Post by mc32 »

the_af wrote: post #442
sega_sai wrote: post #425

When the consciousness itself not understood and well defined in the first place, it is pretty pointless to debate if something is or isn't conscious. And here in particular the reasoning behind the argument is bizarre. Decomposing the complex activity into simple steps like 'predicting the next word' and claiming that surely can't have consciousness. A similar argument would be -- there is no way that movements of e…

But the whole point of the essay is that it's Anthropic that's making the argument (or roleplaying/hinting as if they believed it). Ted Chiang isn't making the argument, he's saying Anthropic making it is misleading and deceitful, and that it's actually a pointless thing to claim. One of the essay's stronger paragraphs is when Chiang explains that Anthropic doesn't truly believe this, otherwise what they are doing wo…

Why would using something that's not organic be akin to slavery? Is using steel under heavy stress in bridges a form slavery of an unconscious object?

Re: Artificial intelligence is not conscious – Ted Chiang

#457

Post by throw4847285 »

qarl wrote: post #335
mohamedkoubaa wrote: post #322

Earlier quoted context omitted.

It honestly isn't a good test

Heh. It's not really a test. It's a line of argument. I rest my case. :)

I think Turing was wrong because he was uncomfortable with ambiguity, and the Turing Test basically is a way to avoid philosophical argument, but it is ultimately a philosophical argument anyway. Plenty of computer scientists have followed in Turing's footsteps, terrified of ambiguity, relying on a kind of cheap functionalism as a salve. You can claim to just be doing science, but inevitably you dip into metaphysics and deny you are doing so. That's this thread in a nutshell. "I only believe what I can prove, but I suspect that if I can't prove it then I don't have to worry about it." My argument, is that you have to worry about it anyway.

Re: Artificial intelligence is not conscious – Ted Chiang

#458

Post by int_19h »

the_af wrote: post #442
sega_sai wrote: post #425

When the consciousness itself not understood and well defined in the first place, it is pretty pointless to debate if something is or isn't conscious. And here in particular the reasoning behind the argument is bizarre. Decomposing the complex activity into simple steps like 'predicting the next word' and claiming that surely can't have consciousness. A similar argument would be -- there is no way that movements of e…

But the whole point of the essay is that it's Anthropic that's making the argument (or roleplaying/hinting as if they believed it). Ted Chiang isn't making the argument, he's saying Anthropic making it is misleading and deceitful, and that it's actually a pointless thing to claim. One of the essay's stronger paragraphs is when Chiang explains that Anthropic doesn't truly believe this, otherwise what they are doing wo…

The title of the article is "No, Artificial Intelligence Is Not Conscious". I'd say that's making the argument.

Re: Artificial intelligence is not conscious – Ted Chiang

#459

Post by tick_tock_tick »

Lerc wrote: post #446
sega_sai wrote: post #425

When the consciousness itself not understood and well defined in the first place, it is pretty pointless to debate if something is or isn't conscious. And here in particular the reasoning behind the argument is bizarre. Decomposing the complex activity into simple steps like 'predicting the next word' and claiming that surely can't have consciousness. A similar argument would be -- there is no way that movements of e…

Does the moon have gravity?

Maybe? It certainly behaves like it does but it might not.

Re: Artificial intelligence is not conscious – Ted Chiang

#460

Post by in-silico »

rmunn wrote: post #400
in-silico wrote: post #366

Earlier quoted context omitted.

> It's not changed by the experience The entire file is not changed, but the KV cache is. > It doesn't remember anything The model definitely remembers previous exchanges within the same conversation.

> The model definitely remembers previous exchanges within the same conversation. No it doesn't. They get added to its context, and it reads them afresh when answering the next question. That's not remembering. If your short-term memory completely malfunctioned one day, so you had no ability to remember what was said to you a minute ago, then you would have to find workarounds. For example, you could write down every…

This is really semantics, but I wouldn't call attending to the KV cache re-reading the context.

The model takes in the context, encodes it into a "memory" (the KV cache), and accesses that memory later. That fact doesn't change just because the KV cache grows in size with the context.

I don't know what memory would look like other than an encode-retrieve loop.

Relevant: Transformers are Multi-State RNNs - https://arxiv.org/abs/2401.06104

Post reply on HN