Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Artificial intelligence is not conscious – Ted Chiang

theatlantic.com

Post reply on HN
Print view View original

731–740 of 1001 posts

Re: Artificial intelligence is not conscious – Ted Chiang

#731

Post by curation »

jmull wrote: post #728

We'd need a good definition of consciousness to get anywhere on this. I suspect such a definition would include agency, which includes desires and goals for the self. LLMs don't seem to have agency, and seem unlikely to get it since they are specifically engineered to do as told. No doubt someone is trying, as we speak, to do just this. But I doubt the effort will be large -- LLMs are engineered to do as told because…

LLMs do not have an unconscious, the negative dimension from which a subject can question what they. LLMs do not have desire because they are thoughts without a thinker. The problem is not LLMS but rather, that subjectivity itself is not popular in discussion. I suggest reading Freud, Lacan, Hegel for a start.

Re: Artificial intelligence is not conscious – Ted Chiang

#735

Post by Zigurd »

sega_sai wrote: post #425

When the consciousness itself not understood and well defined in the first place, it is pretty pointless to debate if something is or isn't conscious. And here in particular the reasoning behind the argument is bizarre. Decomposing the complex activity into simple steps like 'predicting the next word' and claiming that surely can't have consciousness. A similar argument would be -- there is no way that movements of e…

It's perfectly rational for AI researchers to think consciousness could emerge in a software neural network. If what goes on between your ears isn't consciousness emerging from a network of neurons, it must be magic. So anyone claiming it's impossible to make machine consciousness should hedge that bet. There's a chance it could arrive tomorrow.

On the other hand we've got no idea whether that chance is so close to zero as to be negligible, or whether it's imminent. Just the fact we don't know indicates it's closer to zero. Not having a theory of consciousness is kind of a big architectural risk. It's like not having a theory of accounting when you're making an ERP product. Sure, consciousness in bio neural networks wasn't designed. But it took a few few product revisions to get there.

Re: Artificial intelligence is not conscious – Ted Chiang

#736

Post by bondarchuk »

moron4hire wrote: post #715
patrickscoleman wrote: post #535

Earlier quoted context omitted.

I've been digging into Thomas Metzinger[1] recently and here's a tentative component by component definition of human consciousness based on his ideas: - a model of your environment - desires - a process for modeling yourself in that environment (in time & space) - the ability to take action - the perception of yourself having agency - persistence of these processes even without input - unawareness of these processes…

I don't think any of this comports with how chat completion systems work, even if I were to accept Metzinger's framing. Everything an LLM "knows" had to be told to it. It does not have a process for understanding itself in its environment. It does not have any sensing organ that would allow it to detect conditions and come to a conclusion that it must be in such an environment. It doesn't even "know" when it's done s…

>Everything an LLM "knows" had to be told to it.

Why does that matter for whether or not it is conscious? Many things I "know" I also know because someone told me.

>It does not have a process for understanding itself in its environment.

It has a process for understanding, namely outputting tokens by evaluating the neural network iteratively. It can apply this understanding to its environment insofar as the context window and weights include a model of this environment.

>It does not have any sensing organ that would allow it to detect conditions and come to a conclusion that it must be in such an environment.

It does have a sensing organ, namely the input into the context window. Possibly it even has closed-loop sensors by doing tool calls. The conclusion could be incorrect or inaccurate but that doesn't in itself prove there is no consciousness (Plato's cave etc..).

>It doesn't even "know" when it's done something if you don't tell it about what it just did.

It does know, it's in the context window.

>It doesn't have an awareness of the tool calls it can perform; again it has to be told and even then it gets it wrong sometimes.

Same as above, as long as it's in the context window it could have "awareness" of having done it.

>And it can't actually execute any of those tools.

It can execute tools by outputting certain tokens in the right environment.

>It still relies on you to pattern recognize that its output should be a "tool call" and then perform the execution yourself.

Tool calls are not dependent on the human chat user executing them, right? They can happen automatically through the surrounding software.

>The "model" of its environment that an LLM has is 100% a construct of what a human told it and there isn't any way for it to differentiate between that which is real and that which is fantasy.

Again why would the fact that a human told it mean it can't result in consciousness? Why would lack of ability to differentiate between real and fantasy mean it can't be conscious? On my view, it would then be conscious of the fantasy.

Re: Artificial intelligence is not conscious – Ted Chiang

#737

Post by ooloncoloophid »

My background is in cognitive science and psycholinguistics. I spent more than ten years talking to first year psychology undergraduates about whether AIs could be conscious; also did some research on (extremely tiny) AIs in modelling language behaviours.

There is a great deal of good thinking on Chiang's topic by professional philosophers, and there's much to be said for reading them. I won't rehearse their ideas here. Chiang's arguments might be correct; but I suspect they probably aren't, and his error may well stem from characterising human thought as something in its own class, which is probably a cognitive bias that humans have. He might also - I'm speculating - be arriving at his conclusion based on his feelings, which the final paragraph suggests (the comment about the models being based on morally dubious actions).

Speculation aside, we are not, I believe, in a position to make points like he does with any certainty.

Re: Artificial intelligence is not conscious – Ted Chiang

#738

Post by wvbdmp »

jmull wrote: post #724
MrScruff wrote: post #609

Earlier quoted context omitted.

Do you believe consciousness to be an emergent property of the laws of physics?

Regardless, the question still stands: "What does computers getting more intelligent has to do with it getting conscious?" Just because consciousness emerged for we humans and other animals through one mechanism doesn't mean consciousness has/will/can emerge from current LLM technology. For this extraordinary claim, I think the burden is firmly on those who are arguing that it has/will/can.

The other point of view is that the burden is on those who suggest that biological consciousness is somehow special. What makes it so, and if the answer isn’t metaphysical, what’s stopping us from constructing it artificially?

Re: Artificial intelligence is not conscious – Ted Chiang

#739

Post by msabalau »

brcmthrowaway wrote: post #191
Waterluvian wrote: post #164

I think about Star Trek: TNG’s “Measure of a Man” a lot lately. We can be so confident to decide what is and isn’t alive from vibes alone. The conclusion I’m currently at is that I don’t know and probably can’t ever know. Maybe you’re all philosophical zombies. Maybe I am one too! But at some point we will get close enough that it hopefully becomes obvious that we must tread carefully. The entire episode is incredibl…

I wonder if modern Star Trek could make this episode.

Even if it could, it probably ought not makes this precise episode, because the technological, philosophical and social context is remarkably different.

A similar episode, actually informed by what we know or can forsee about AI and LLMs, and addressing our hopes and fears about what they mean would be interesting though.

Post reply on HN