Live data from Hacker News

Artificial intelligence is not conscious – Ted Chiang

theatlantic.com

91–100 of 1001 posts

Re: Artificial intelligence is not conscious – Ted Chiang

#91
sociopaths sometimes study people to learn how to emulate emotion

that's exactly the state of "AI" right now, it's cold, mathematical emulation

btw there are some fascinating papers on the concept that consciousness in humans is actually a quantum effect

brilliant Roger Penrose proposed it (and they thought he was nuts) but recent discoveries about microtubules make it plausible

so who knows, maybe a dozen exponential improvements in quantum computers could make "AI" really conscious next century

* https://www.youtube.com/watch?v=xa2Kpkksf3k

Re: Artificial intelligence is not conscious – Ted Chiang

#92

We do not know if Claude is conscious, and we will almost certainly never know. Any strong claim either way is over confident.

Fancypants autocomplete cannot be conscious. It's just echoing previous human experiences, which make it sound like a person. It is not a person. It is an algorithm. There is no mechanism by which it can obtain consciousness.

People believing otherwise are fools. People debating this are idiots. I realize these words are harsh, but it's the truth.

Re: Artificial intelligence is not conscious – Ted Chiang

#93

I think Chiang is right about this, but there is a related philosophical mystery. The trend from Deep Blue to AlphaGo to LLMs solving Erdos problems suggests that Peter Watts was onto something when he wrote Blindsight. Reasoning ability is apparently independent of consciousness? We haven't really come to grips with that yet. What does it mean if nothing we write proves anything about anyone's consciousness?

We don't have anything to measure consciousness generally. We don't even have a broad consensus on what consciousness is . And because of that we can't discern whether X is conscious for most values of X , including but not limited to LLMs.

Well, at the very least we have a base to distinct different kinds then.

Probably the main problem of people implying LLM consciousness is that they imply LLM have human-kind of consciousness. Judging only on "how they speak", generally, they insist on using the same word that labels human consciousness (exclusively), etc.

But there are so many instrinic differences that such claim is not feasible despite similar "talking abilities".

Re: Artificial intelligence is not conscious – Ted Chiang

#94
Embarrassingly incompetent article. Given that one can observe up to 40 definitions of consciousness (https://philpapers.org/rec/VIMMAT - also many definitions are unrelated at all), "consciousness" is almost certainly just a family resemblance category at best, and talk about whether or not something is "conscious" without providing definitions is simply completely unserious.

To make progress, you have to talk about kinds / aspects of consciousness. AI does and will share some of these aspects with humans, but it will not and does not share others. It is really that simple. For the most part, modern AI implemented via LLMs has almost none of the stronger or most core aspects of consciousness.

For huge parts of the article "intelligence" and "consciousness" are conflated, which is mostly extremely unhelpful, as this is not generally a core feature of most aspects of "consciousness".

The moral arguments are also incompetent, i.e. claiming "Moral reasoning is [...] is necessarily subjective" is just clearly empirically wrong, as in fact LLMs can produce moral reasoning (i.e. verbalized moral arguments that are coherent), as can p-zombies (i.e. there is nothing 'necessary', in the philosophical sense of the term, about subjectivity here). The only way the argument holds is if you tautologically define moral reasoning as requiring that reasoning be produced by a consciousness, but this is question-begging.

Re: Artificial intelligence is not conscious – Ted Chiang

#95
post #59

Earlier quoted context omitted.

>When exactly would a bunch of graphics cards become conscious? When does an embryo become conscious? Unless you can answer that precisely then it seems futile to speculate about non-human consciousness.

I mean, there is obviously a point - we can argue whether that point is 20 weeks in, 40 weeks in, or after birth - but there is obviously a point where a human being goes from a collection of cells to a conscious being. I don't really see a need to answer this precisely to be able to say that a token predictor is not conscious?

No, the point is we literally cannot meaningfully argue that.

There is no actual definition of consciousness and there is no way to test it's existence. Let alone understanding the properties of consciousness, such as if it's binary or a gradient; or if it requires a meat substrate or not; and why would that possibly matter since meat is just a lot of the same stuff but highly processed and wet? A solipsist may not even believe you are conscious, despite being made of similar meat.

No matter how much you want to hand-wave it, there's absolutely nothing "obvious" about it. Many have a preconceived notion and are simply asserting it as undeniable fact.

Re: Artificial intelligence is not conscious – Ted Chiang

#96
I always find the minimizing view of consciousness a bit uninspiring. Like we need to be unique.

I've yet to find a reason why it couldn't be the opposite, way more things are conscious than we've been led to believe. What if consciousness appears out of any system that is actively persisting through effects caused by itself? That might be a forest, or outside the realm of the living, a company. An ant colony, or a planet.

Complex chemical reactions, layered upon each other such that tiny blocks make up large entities. Individual bits combined such that they make up something new intelligible by us.

I think the strongest argument against AI being conscious is that it does not persist, it resets, but that does not seem unchangeable.

Re: Artificial intelligence is not conscious – Ted Chiang

#97
post #16

Really Chiang? Alright. What is consciousness? Please provide a definition that somehow encompasses all humans and excludes all current AI. I’ll wait.

Consciousness excludes all current AI because all current AI is just autocomplete over the corpus of human text.

It's just a word machine. There are no thoughts. It cannot be conscious. How is this even up for debate in any way whatsoever? I do not understand how people can believe this. Is this not a site for software engineers?

Re: Artificial intelligence is not conscious – Ted Chiang

#98
>I would argue that it is fundamentally dishonest to have a machine emit many categories of sentences, including any sentences using first-person pronouns.

Finally I said something before Ted rather than the other way round!

https://techcrunch.com/2023/12/21/against-pseudanthropy/

While I agree with the premise here, I do think that it's easy for an arguer to move the goalposts such that the Caesar-Khan example no longer matters. The characters don't have to be conscious for the thing that created them (as in the case of the user doing it) to be so. So the argument would be that the creator of the characters is itself conscious, but not them. This feels like a kind of inverted no-true-scotsman type thing, but it does allow someone to retreat in some semblance of rationality.

Re: Artificial intelligence is not conscious – Ted Chiang

#99
post #15

A person can certainly be conscious, but can they also be not-conscious? I think that most of our cognitive time is spent in activities that don't require consciousness and consciousness itself isn't needed for the majority of activities that people do. I would go so far as the a non-trivial part of people's time is spent in a not-conscious state.

That doesn't jive with normal definitions of consciousness. The word we use for "not-conscious" humans is "unconscious".

Regardless of the word we use, occasionally there have been times where I was awake but so absorbed in an activity or in my thoughts that I didn’t have a self-experience, for a certain duration (tenths of minutes). My mind was focused solely on the activity, and not on itself whatsoever. It’s a surprising feeling to notice that while you have memory of what you had just been doing, you have no memory of your mind experiencing itself doing it. I would be inclined to say that I wasn’t conscious during that stretch of time.

Re: Artificial intelligence is not conscious – Ted Chiang

#100
post #96

I always find the minimizing view of consciousness a bit uninspiring. Like we need to be unique. I've yet to find a reason why it couldn't be the opposite, way more things are conscious than we've been led to believe. What if consciousness appears out of any system that is actively persisting through effects caused by itself? That might be a forest, or outside the realm of the living, a company. An ant colony, or a p…

Maybe we're the ones who are not conscious?
Post reply on HN