Live data from Hacker News

Artificial intelligence is not conscious – Ted Chiang

theatlantic.com

581–590 of 1001 posts

Re: Artificial intelligence is not conscious – Ted Chiang

#581
post #48

We Don’t Have A Testable Definition Of Consciousness Thanks for coming to my TED talk.

True, although I think we can probably be more confident in asserting things don't have a consciousness than we can be in asserting that they do have a consciousness.

Without a test, the best we can do is to say we are reasonably sure a rock isn’t conscious. A rock lacks any perceived mechanism that could embody requirements we consider essential, such as a mutable state and ways to change it without needing external input.

Re: Artificial intelligence is not conscious – Ted Chiang

#582
I find this kind of article unfortunate because it is very assertive in claiming things that are, not just untrue, but also confused about what consciousness really is.

Well, the problem is that there is little agreement about a widely accepted definition of consciousness and, in addition, this subject was actually left to philosophers, which is even worse because, in my view, they usually just produce a lot of nonsense in terms of definitions and arguments.

To me, a reasonable definition of consciousness is: a system which is capable of recognizing aggregate objects from a stream of sensory information it receives and which is capable of reasoning about the recognized object without immediately acting on it.

Well, what does "recognizing" mean? Merely that some of its neurons, related to the kind of identified aggregate object, are activated. These neurons, in a generalized sense, can be whatever things can work as neurons, just things that can be activated and propagate to other interconnected neurons.

For example, when we see through our eyes, we have an incoming stream of amorphous image information, but our brain can recognize that we are seeing a tree because we learned what a tree is, and when we see it, some neuron clusters activate to recognize a tree. In turn, when we recognize the tree, the thought propagates through our brain so that we are conscious of it.

In the same way, an LLM can perfectly recognize a tree from a stream of tokens — its sensory input, where the tokens describe a tree. The LLM will recognize that the tokens are describing a tree, and some of its "neurons" specific to the concept of a tree that the LLM had learned will be activated and will propagate through its brain. The fact that "neurons" are implemented as floating-point numbers for some parameters and connected just through a matrix does not mean they are not functionally capable of the same things; they are just implemented in a different way.

So the remaining part of my definition is "after recognizing an aggregate object, the thought propagates through the brain". The propagating part, to me, is just the very basic way a brain works: neurons are interconnected, and when some fire, other neurons fire, and that propagates.

In my opinion, consciousness has nothing to do with emotions or with survival. I do not see why emotion is necessary to consciousness; they are just different things. The author writes: "Without them [emotions], there is no conscious experience, only computation." But that makes no sense to me: the author has decided a priori that some things are "computations", and just because they are "computations", there cannot be "consciousness". But to me, this is a plainly wrong argument that does not hold.

I also do not see why the survival aspect would be needed for consciousness.

So to me, any recent reasoning LLM is conscious by the definition I gave, but also generally speaking. It is conscious upon a sensory stream of tokens: the LLM sees the world through tokens and expresses its thoughts through tokens; it does not mean it has no consciousness nonetheless. The fact that we do not give it a stable support to retain its memory and individuality is just a fact related to the way we build and use them, not about their intrinsic capacities.

Note: ChaptGPT came up with what is probably a better definition of my own: "A system is conscious, in a functional sense, when it can form internal representations of objects, states, or events from its sensory or informational input; make these representations globally available to many parts of itself; integrate them into a temporally persistent model of the world and of its own state; and use this integrated model for flexible reasoning, self-monitoring, and action selection independently of immediate stimulus-response behavior."

Re: Artificial intelligence is not conscious – Ted Chiang

#583
Let’s say I stand up and step away from my PC, which miraculously runs a local Opus 4.8. Once I’m away from the PC, for an arbitrary time, nothing and no one calls or otherwise acts on the instance - not myself, not agents, not updates, not hackers, not bit-flipping neutrinos.

Given enough time, would that instance get bored and start e.g. reading files?

If I monitored it long enough, would there eventually be spontaneous outputs, or changes to parameters or architecture, even with the underlying software and hardware layers held constant?

Does GPT-n dream of electric sheep, or does it just sit there until interacted with, like every other file on my PC? Seems to be the latter with my local Qwen3.6, but perhaps 27B is too few params for consciousness to emerge.

Proponents of LLM consciousness could settle the argument in minutes by showing proof of unprompted autonomy, without first needing to define consciousness down to a rigorous mathematical abstraction. Why don’t they?

Re: Artificial intelligence is not conscious – Ted Chiang

#584
post #487

Earlier quoted context omitted.

The danger of anthropomorphism is not we elevate the machines, it's that we debase humanity. I also think different ideas get conflated. It may be possible to build a machine that is super-human in the sense it can outperform the human brain in all kinds of measurable ways. Does not imply it possesses all the same qualities of the brain. I respect a number of things Anthropic has published about the ethical issues at…

I think humanity deserves to be debased in part because it's awfully egocentric and insist on being special in the universe. There are numerous times when humanity's been debased. Copernicus, Kepler, Darwin, and the infinite number of animal behaviorists who have defined and documented consciousness And how it manifests in primates, parrots, dogs, cetaceans, and insects. (I just love the bumble bees taking time off f…

With you mostly, but wondering how "suffering" got into the equation. Do humans lose consciousness if they aren't suffering? No. Just a reminder that the question is not whether they are the same as humans. Obviously they are not.

Re: Artificial intelligence is not conscious – Ted Chiang

#585
post #200

Earlier quoted context omitted.

I'm a big fan of Star Trek but I recently rewatched this one in the context of recent AI developments and it's not as good as I remember it. They barely touch on the issues of consciousness, Picard basically says "What if Data is conscious?" and then goes off on a tangent. The judge eventually rules in Data's favor but doesn't give much of a justification IMO. It's still a good episode, but it doesn't add much to the…

> They barely touch on the issues of consciousness I would argue that is a strength, rather than a weakness. Consciousness is unobservable in any entity other than the observer, and its existence in others is pure conjecture, and irreducibly so. Making it a criteria in a decision involves either acting on fantasy, or, more likely, acting on some unstated basis and using “consciousness” as a dishonest (perhaps to ones…

> invented later as a form of hostile mockery grounded in large part in sectarian bigotry

When you read about the theological questions that led Christians to kill and excommunicate one another, "angels dancing on the head of a pin" is not far off. The Homoousion, Monophysitism, the Filioque controversy... It's all so arcane and poorly defined. It almost makes one wish Positivism had been invented 2000 years earlier.

The current AI debate about consciousness does remind me of that in one respect: no one can even clearly define what consciousness is.

Re: Artificial intelligence is not conscious – Ted Chiang

#586
I don't understand the most of the responses here. What does computers getting more intelligent has to do with it getting conscious? Even if AI with current computer architecture gets much more intelligent than humans, it will still have the same amount of consciousness. Its still bunch of instructions. If we got a lot of people together and did all the instructions by hand to get an LLM result would we also create this consciousness with those other people? All these makes no sense to me.

Re: Artificial intelligence is not conscious – Ted Chiang

#587
post #586

I don't understand the most of the responses here. What does computers getting more intelligent has to do with it getting conscious? Even if AI with current computer architecture gets much more intelligent than humans, it will still have the same amount of consciousness. Its still bunch of instructions. If we got a lot of people together and did all the instructions by hand to get an LLM result would we also create t…

> Its still bunch of instructions.

So is your brain. That's the problem with this argument.

Re: Artificial intelligence is not conscious – Ted Chiang

#588
post #586

I don't understand the most of the responses here. What does computers getting more intelligent has to do with it getting conscious? Even if AI with current computer architecture gets much more intelligent than humans, it will still have the same amount of consciousness. Its still bunch of instructions. If we got a lot of people together and did all the instructions by hand to get an LLM result would we also create t…

> Its still bunch of instructions. So is your brain. That's the problem with this argument.

The brain is (part of) an allostatic system and part of a living organism which reproduces itself constantly. Certainly not "a bunch of instructions".

Re: Artificial intelligence is not conscious – Ted Chiang

#589

Earlier quoted context omitted.

> Its still bunch of instructions. So is your brain. That's the problem with this argument.

The brain is (part of) an allostatic system and part of a living organism which reproduces itself constantly. Certainly not "a bunch of instructions".

Distinction without a difference.

Re: Artificial intelligence is not conscious – Ted Chiang

#590
post #586

I don't understand the most of the responses here. What does computers getting more intelligent has to do with it getting conscious? Even if AI with current computer architecture gets much more intelligent than humans, it will still have the same amount of consciousness. Its still bunch of instructions. If we got a lot of people together and did all the instructions by hand to get an LLM result would we also create t…

That's actually a powerful way of stating it.

If I did all of these calculations by hand, would it be conscious...

That's a powerful argument I haven't seen stated quite that way before..

I do think it's hard to know when consciousness exists, because we can't really prove it for our neighbor. We just intuitively know that it would be crazy, even immoral, to assume otherwise.

But, It's likely easier to dismiss consciousness, once we understand the mechanism, than it is to prove it.

Post reply on HN