Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Artificial intelligence is not conscious – Ted Chiang

theatlantic.com

Post reply on HN
Print view View original

341–350 of 1001 posts

Re: Artificial intelligence is not conscious – Ted Chiang

#341

Post by hackinthebochs »

cauch wrote: post #254
Nevermark wrote: post #194

> My intention is to highlight the fact that LLM conversations are cleverly disguised examples of sentence continuation Regardless of bigger issues, this kind of statement reveals a deep misunderstanding. Problem type does not limit problem complexity. Nor does problem type limit solution complexity or power. If a machine has to learn to understand humans to complete text, then that is what it has to do. And there is…

I think, for me, the thing is that when you do basic ML, you discover that ML will very often find data pattern that fit the goal but does not correspond to a real mechanism. So, I think there is a flaw in the logic of saying that human text have a pattern of "consciousness mechanism" and therefore LLM will learn "consciousness mechanism" in order to return sentence continuation that is convincing. There is probably…

> I think there is a flaw in the logic of saying that human text have a pattern of "consciousness mechanism" and therefore LLM will learn "consciousness mechanism" in order to return sentence continuation that is convincing.

There is no independent "consciousness mechanism" that one might imagine humans have learned or evolved for its own sake. Evolution learns various solutions to optimization problems, and so if consciousness evolved then it was either useful instrumentally, or it is a byproduct of some organization that is useful instrumentally. The point is that as a solution to certain kinds of optimization problems, consciousness can conceivably be the solution to the optimization problem of predicting the next token of text written by humans who themselves have complex phenomenology. There is nothing that a priori constrains token prediction from the domain of consciousness.

>For me, one element that shows it is the case is the absence of world model (or "human-like" world model) despite the fact that the sentence continuation is convincing

World models don't have to be rich and detailed to count as a world model. Lower life forms might be conscious but they only model the part of the world useful for their existence in their ecological niche.

Re: Artificial intelligence is not conscious – Ted Chiang

#342

Post by supertroop »

CommieBobDole wrote: post #104

The fact that a LLM is essentially immutable would be my biggest argument against consciousness or self-awareness. It's a big file with a bunch of coordinates describing spatial relationships between tokens. When you give it a prompt, it uses those relationships to generate a string of tokens that is a statistically likely response to that prompt, then it stops. It's not changed by the experience. It doesn't remember…

Reinforcement learning changes the model. So it can and does change and remember based on experience. Eventually reinforcement learning can happen in real time.

Re: Artificial intelligence is not conscious – Ted Chiang

#343

Post by scotty79 »

jondiggsit wrote: post #304

Here's a simple idea to consider: It doesn't matter. You won't be able to tell the difference. No one will. I don't think it's necessary to explain this idea further. Just think about it.

Consciousness is one of those silly words that were disfigured to death by philosophers that had near zero actual input and tools to tackle the matter, yet they tried anyways instead of finding some questions that might actually be answerable. That's something philosophers always do since the advent of science.

With LLMs, where we can manipulate their parameters intentionally run them many times on the same data, run parts of them, split and connect, we might eventually acquire sufficient tools to even define consciousness concretely for the first time.

Re: Artificial intelligence is not conscious – Ted Chiang

#344

Post by felipeerias »

IMHO the sane position is essentially the Aristotelian one.

Hylomorphism: body and consciousness are intrinsically linked. The nature of that link is an open metaphysical question.

Virtue ethics: even if LLMs are not conscious, we should not abuse or mistreat them them because cruelty practised on anything trains one's disposition toward cruelty.

Re: Artificial intelligence is not conscious – Ted Chiang

#345

Post by luka2233 »

scotty79 wrote: post #332

"Is it conscious?" Is that even a question worth asking? We are so terrible at even defining each word in that sentence. Can I help you? Can I harm you? What's the moral behavior towards you? Those are more practical questions.

+1 Whenever I see these debates and ask what the implications should be, both sides tend to agree.

Re: Artificial intelligence is not conscious – Ted Chiang

#346

Post by luka2233 »

andrelaszlo wrote: post #100
blixt wrote: post #96

I always find the minimizing view of consciousness a bit uninspiring. Like we need to be unique. I've yet to find a reason why it couldn't be the opposite, way more things are conscious than we've been led to believe. What if consciousness appears out of any system that is actively persisting through effects caused by itself? That might be a forest, or outside the realm of the living, a company. An ant colony, or a p…

Maybe we're the ones who are not conscious?

whatever that would mean :)

Re: Artificial intelligence is not conscious – Ted Chiang

#347

Post by glaslong »

ycui7 wrote: post #231

does it really matter if LLM has conscious? if they produce working code, then it is useful, whoever if they have real conscious of fake intelligence. we don't know what the conscious in human brain is either.

This, too, is in the article.

"The fact that LLMs lack subjective experience has little bearing on the question of whether LLMs might be useful tools or have significant economic impact."

Re: Artificial intelligence is not conscious – Ted Chiang

#348

Post by [deleted] »

cout wrote: post #184
solid_fuel wrote: post #155

Earlier quoted context omitted.

I think you're missing his point. It's not about the hormones and physics of our bodies, and indeed he specifically allows for "either physical or virtual" bodies even in the block ~~~you~~~ grandparent quoted. The point here is not that it must have a body like ours, the point is that a conscious entity must have a boundary line between internal (the body) and external (everything else). A virtual sense organ can si…

Would it be sufficient to have a second stream of tokens that becomes the model's equivalent of "internal dialogue"? Would that satisfy the requirement of a boundary line? Related, is a human "thinking out loud" still thinking, even though the internal reasoning is "immediately dumped into the external environment"?

[deleted]

Re: Artificial intelligence is not conscious – Ted Chiang

#349

Post by calf »

There's a provocative argument raised in the article that I disagree with:

1. DeepFakes, generative image/video/AlphaFold type AIs are not conscious

2. LLMs are generative AI trained on human text samples

3. LLMs are not conscious, and LLMs just seem-to-be conscious

I might argue instead that (2)-> destroys (1), that in fact we should consider even sensory generative AI are somewhat conscious. That is, Chiang's argument also flows in reverse. Or I might argue text samples (2) are so rich in conscious expression that the same process of training really does produce a conscious machine (through some kind of emergence and complexity.)

Either way his simplistic argument falls apart, and/but the crux of the piece falls on getting basics like this correct.

Re: Artificial intelligence is not conscious – Ted Chiang

#350

Post by hackinthebochs »

hn_acc1 wrote: post #305
krupan wrote: post #229

Earlier quoted context omitted.

"If a machine has to learn to understand humans to complete text, then that is what it has to do." But the machine doesn't have to understand humans to do that. It gets trained on a whole bunch of sentences and then it is able to complete text. You could maybe claim that it "understands" the text but even that's a stretch.

It can't even natively understand how many letters there are in words - how will it understand the meaning?

I wish people would do even the most basic amount of research into LLMs before opining about what they can or cannot do. There are very principled reasons why LLMs do not know how many letters are in words, and it says nothing about their facility for understanding meaning.

Tokens are the most basic input unit of an LLM. But tokens don't generally correspond to words or letters, rather sub-word sequences. So Strawberry might be broken up into two tokens 'straw' and 'berry'. It has trouble distinguishing features that are "sub-token" like specific letter sequences because it doesn't see letter sequences but just the token as a single atomic unit. 'Straw' and 'r' are two tokens but an LLM is entirely blind to the fact that 'straw' has one 'r' in it.

As an analogy, I might ask you to identify the relative activations of each of the three cone types on your retina as I present some solid color image to your eyes. But of course you can't do this, you simply do not have cognitive access to that information. Individual color experiences are your basic vision tokens.

The widespread mistake people keep making is assuming the development of intelligence in LLMs should follow the same trajectory that human intelligence takes as it develops into adult levels of intelligence. Thus deficiency in some capacity that we take for granted in humans is an indictment on LLM intelligence. But this is specious. LLMs are entirely alien; their developmental paths do not and should not look anything like ours. Your intuition from human intelligence just works against understanding the potential for intelligence in LLMs.

Post reply on HN