Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Artificial intelligence is not conscious – Ted Chiang

theatlantic.com

Post reply on HN
Print view View original

321–330 of 1001 posts

Re: Artificial intelligence is not conscious – Ted Chiang

#321

Post by glaslong »

eberkund wrote: post #124
sigmar wrote: post #66

>So what context would cause me to seriously consider the possibility that engineers had created a computer program that is conscious and an intentional user of language? Let me outline one potential sequence of steps. The first requirement is that the computer program has a body (either physical or virtual) and sense organs; there are many reasons for this, but for the purposes of this discussion the most relevant o…

I'm more interested in what a virtual body would entail. To me the root of this idea is around persistent state which is something that currently LLMs do not have. Imagine if somehow your brain lacked long-term memory forming capabilities and instead each day when you woke up you had to read a notebook with (markdown formatted) instructions that you wrote the day before? I wouldn't be surprised if such a person lacke…

Yes it seems that is the crux of the embodiment section in the article. That whether physical or virtual, the "AI" needs minimally: persistence in its environment, sensory signals of that environment, and some feedback loop of continuing to try to exist in that environment; having a subjective experience.

And that that is the baseline before we can really even consider that it has consciousness of its own subjective experience, versus being a worm that happens to output text as its digestion process.

And then the further question only after that is established, is what are its needs? What moral patienthood do we have to acknowledge in terms of meeting those needs? And finally, with all the other prerequisites checked, what is the AI's moral agency in what it chooses to do.

Re: Artificial intelligence is not conscious – Ted Chiang

#323

Post by felipeerias »

throwawayk7h wrote: post #204

Regardless of whether LLMs are conscious or not, they have no known mechanism for experiencing pain and suffering, and there's no reason to believe they have one (such as a limbic system). So why worry about it?

Mistreatment and abuse, even when directed at a machine, make you a worse person.

Even if you are only interested in getting good results out of them, LLMs tend to work better when they are immersed in a narrative of open collaboration.

Re: Artificial intelligence is not conscious – Ted Chiang

#324

Post by hn_acc1 »

calf wrote: post #292
Nevermark wrote: post #194

> My intention is to highlight the fact that LLM conversations are cleverly disguised examples of sentence continuation Regardless of bigger issues, this kind of statement reveals a deep misunderstanding. Problem type does not limit problem complexity. Nor does problem type limit solution complexity or power. If a machine has to learn to understand humans to complete text, then that is what it has to do. And there is…

His intention is irrelevant, as is "trying to highlight a fact" as if it were the final say: all Chiang is doing here is using fancy white-collar words to argue the same argument leveled against Hinton and others regarding next-token prediction. And his audience, who have even less technical understanding, lap it all up unawares. Chiang is a writer and needs to stay his own lane, not RP as an expert; or, if he wants…

Chiang has, in fact, written on this topic before - see "The Lifecycle of Software Objects", and has speculated about sentience in AI, etc. This is not a "one-off", "I need money" type of article. I dare say he has thought about this much more than most people here.

From Wikipedia: In 2023, Chiang was named one of Time's 100 most influential people in AI.

Re: Artificial intelligence is not conscious – Ted Chiang

#326

Post by lgessler »

Nevermark wrote: post #194

> My intention is to highlight the fact that LLM conversations are cleverly disguised examples of sentence continuation Regardless of bigger issues, this kind of statement reveals a deep misunderstanding. Problem type does not limit problem complexity. Nor does problem type limit solution complexity or power. If a machine has to learn to understand humans to complete text, then that is what it has to do. And there is…

Raphaël Millière has a very useful term for this kind of vacuous dismissal, the redescription fallacy (https://arxiv.org/pdf/2401.03910, page 9):

> Recent debates have been clouded by a misleading inference pattern, which we term the “Redescription Fallacy.” This fallacy arises when critics argue that a system cannot model a particular cognitive capacity, simply because its operations can be explained in less abstract and more deflationary terms. In the present context, the fallacy manifests in claims that LLMs could not possibly be good models of some cognitive capacity because their operations merely consist in a collection of statistical calculations, or linear algebra operations, or next-token predictions. Such arguments are only valid if accompanied by evidence demonstrating that a system, defined in these terms, is inherently incapable of implementing . To illustrate, consider the flawed logic in asserting that a piano could not possibly produce harmony because it can be described as a collection of hammers striking strings, or (more pointedly) that brain activity could not possibly implement cognition because it can be described as a collection of neural firings. The critical question is not whether the operations of an LLM can be simplistically described in non-mental terms, but whether these operations, when appropriately organized, can implement the same processes or algorithms as the mind, when described at an appropriate level of computational abstraction.

Re: Artificial intelligence is not conscious – Ted Chiang

#327

Post by D-Machine »

Taek wrote: post #289
D-Machine wrote: post #94

Embarrassingly incompetent article. Given that one can observe up to 40 definitions of consciousness ( https://philpapers.org/rec/VIMMAT - also many definitions are unrelated at all), "consciousness" is almost certainly just a family resemblance category at best, and talk about whether or not something is "conscious" without providing definitions is simply completely unserious. To make progress, you have to talk abou…

> modern AI implemented via LLMs has almost none of the stronger or most core aspects of consciousness Can you elaborate on this? What are the specific "stronger and most core aspects of consciousness"? And why are you certain that they are stronger and "more core"?

If you are interested in some serious discussion, see https://lossfunk.com/papers/ai-consciousness.pdf, especially the early section "Consciousness as Family Resemblance". I suppose another is Ned Block on consciousness being a "mongrel concept", and the distinction between access vs. phenomenal

The first paper picks out e.g. arousal/wakefulness, phenomenal quality / qualia, unity (how we feel sensory inputs and qualia as a unified scene), access consciousness (instrumental self-observation and modification broadly), meta-cognition and self-modeling, emotional valence (e.g. pain/pleasure).

One might also include intelligence (abstract reasoning / argument, information integration and abstraction, attention) broadly, and also agency / desires / drives / will. Insofar as these are aspects of consciousness, yes, AI (and simpler algorithms and mechanistic structures) demonstrate aspects of consciousness. But insofar as embodiment, self-reflexivity and qualia (phenomenal consciousness) are the more mysterious and more obviously unique aspects of consciousness, current LLMs very clearly are lacking these things in most ways (whereas animals are much less clearly lacking, especially when you get to mammals and primates).

Seriously, just ask an AI this stuff, you'll get very detailed responses, nothing I am saying here is new or obscure.

Re: Artificial intelligence is not conscious – Ted Chiang

#328

Post by supern0va »

slashdave wrote: post #315
Nevermark wrote: post #194

> My intention is to highlight the fact that LLM conversations are cleverly disguised examples of sentence continuation Regardless of bigger issues, this kind of statement reveals a deep misunderstanding. Problem type does not limit problem complexity. Nor does problem type limit solution complexity or power. If a machine has to learn to understand humans to complete text, then that is what it has to do. And there is…

> of the form of original data streaming in and out. Except this is not consciousness.

I will say, I find it fascinating that there are some philosophers and consciousness researchers who seem to be less certain. I just listened to Chris Hayes interview David Chalmers this week, whose position seemed to be that it's probably not conscious, but that we can't be certain. And more than that: he seemed open to the idea that they may become conscious under further scaling/training/advancements.

It's a great interview, if you're interested: https://www.youtube.com/watch?v=NgDIG8u1-CA

Re: Artificial intelligence is not conscious – Ted Chiang

#329

Post by mohamedkoubaa »

ViktorRay wrote: post #3

Ted Chiang is brilliant. His novella “Anxiety is the Dizziness of Freedom” altered the course of my life. It changed the way I looked back at certain pivotal moments in my life and taught me to think about those pivotal moments differently than how I was thinking about them. Similar to what happens to one of the characters in the story who ends up changing their perception of a key moment in their life. I won’t go in…

Seconded. That was one of the best stories I ever read

Re: Artificial intelligence is not conscious – Ted Chiang

#330

Post by K0balt »

darepublic wrote: post #325

I don't know what consciousness is exactly but I doubt we create it by accident, without fully understanding our own

lol. All consciousness has been created by accident, so empirically there’s not much support for that position. But I want to feel the same way.
Post reply on HN