Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Artificial intelligence is not conscious – Ted Chiang

theatlantic.com

Post reply on HN
Print view View original

881–890 of 1001 posts

Re: Artificial intelligence is not conscious – Ted Chiang

#881

Post by otterdude »

indoordin0saur wrote: post #793

I often hear the assumption that LLMs can or will become conscious because consciousness is likely substrate independent . The idea being that our brain is just a computer made out of meat and it doesn't do anything that can't be precisely simulated by a silicon computer. But I wonder if some of the magic in the human brain is its analog nature. Chemical signaling and impulses of neurons interact with each other with…

Wouldn't a conscious entity be able to grow its intelligence over time independently?

LLM's require offline training and dont actually learn from their "lived / sessions / chats ect". Those can be used for training data but its not like its an implicit part of the technology.

For this reason I would say LLM's are not conscious, or automatically conscious.

Re: Artificial intelligence is not conscious – Ted Chiang

#882

Post by ACCount37 »

rmunn wrote: post #400
in-silico wrote: post #366

Earlier quoted context omitted.

> It's not changed by the experience The entire file is not changed, but the KV cache is. > It doesn't remember anything The model definitely remembers previous exchanges within the same conversation.

> The model definitely remembers previous exchanges within the same conversation. No it doesn't. They get added to its context, and it reads them afresh when answering the next question. That's not remembering. If your short-term memory completely malfunctioned one day, so you had no ability to remember what was said to you a minute ago, then you would have to find workarounds. For example, you could write down every…

Continuous learning allows past behavior and past inputs to influence future inputs and future behavior. In humans.

Attention over KV cache allows past behavior and past inputs to influence future inputs and future behavior. In LLMs.

Until the cache runs out, that is. But even then, you could totally use any of 9000 methods of cache compression, truncation, dropping or streaming and get away with it.

The difference between continuous learning and in-context learning seems to be in capacity, not in principle. Both are doing a similar thing, but one has more length and depth to it.

Re: Artificial intelligence is not conscious – Ted Chiang

#883

Post by xk3 »

JCTheDenthog wrote: post #842
slibhb wrote: post #836

There's currently no way to prove that something is or isn't conscious. I would bet that rocks and LLMs aren't conscious. I would bet that humans and dogs are conscious. But I can't prove it. Whenever people try to prove it (in either direction), they end up resorting to sophistry. This state of affairs could change. If neuroscientists develop a deeper understanding of consciousness, we could talk about it with more…

I mean, outside of our own internal experience of qualia, there's no way for us to prove anyone or anything is conscious. But my biggest issue with the debates about consciousness and LLMs is that the "LLMs are conscious" crowd usually has to presuppose a completely materialistic/mechanistic understanding of the universe and of our own consciousness, and that's something that we really don't know for certain. Also, i…

> a bunch of pipes and valves can be conscious

Billions * billions of pipes and valves can result in emergent behavior that appears conscious while at the same time the sound of a single independent water pipe can moan and sound like human speech or otherwise lifelike and evoke human emotions.

I think LLMs are doing both of these things and often people are more impressed by the independent fixtures (the moan) rather than the emergent behavior. Both the sound and the emergent behavior can be built on purpose or on accident.

I think it helps to look at this through an Information Theory lens. What information is coming into the system (the human or the machine)? What information goes out of the system which is novel? How much of this can be attributed to attempting to parse random noise aka. `Random_Imagination_Engine` vs something else? The number of inventors who come up with a breakthrough idea after mis-hearing someone is surprisingly high.

If we make the distinction between phenomenal consciousness from access consciousness we can see that LLMs clearly can make decisions based on input (A-Consciousness) but they probably don't have raw feelings and sensations (P-Consciousness).

Re: Artificial intelligence is not conscious – Ted Chiang

#885

Post by kranke155 »

stabbles wrote: post #703
CommieBobDole wrote: post #104

The fact that a LLM is essentially immutable would be my biggest argument against consciousness or self-awareness. It's a big file with a bunch of coordinates describing spatial relationships between tokens. When you give it a prompt, it uses those relationships to generate a string of tokens that is a statistically likely response to that prompt, then it stops. It's not changed by the experience. It doesn't remember…

A medicine for those who anthropomorphize LLMs is to run the LLMs deterministically (without randomness and memory files). It feels very unnatural to get the same conversation verbatim at a different point in time.

this just means they are incomplete, like a baby that has no long term memory. I think the baby analogy will hold up as we build more and more capability.

Re: Artificial intelligence is not conscious – Ted Chiang

#886

Post by darepublic »

K0balt wrote: post #330
darepublic wrote: post #325

I don't know what consciousness is exactly but I doubt we create it by accident, without fully understanding our own

lol. All consciousness has been created by accident, so empirically there’s not much support for that position. But I want to feel the same way.

you are probably right. I knew that my position contains the kind of anthropomorphism that usually gets proven wrong by history. however.. we are in fact talking about technology here not evolution. and yes technological progress is fraught with accident and emergence but to this degree?

I suppose its also just a philosophical discussion because while we can all agree we are conscious.. afaik its not really a scientific term. I don't think we understand scientifically what really defines consciousness. So I think it is likely we may deem our computers conscious because they exhibit the same behaviour that we do, but it doesn't mean they actually possess it. Practically speaking it may not matter, because I think you can have long horizon agency without needing consciousness. just a mechanism for determining goals, which could be as simply as rolling a digital dice or following some original seed of intention left by the creators.

its like the chinese room argument I suppose. the hypothetical chinese room is not conscious but ultimately it doesn't matter. But I don't think anyone would call such a thing conscious.

Re: Artificial intelligence is not conscious – Ted Chiang

#887

Post by 23askl9 »

sega_sai wrote: post #425

When the consciousness itself not understood and well defined in the first place, it is pretty pointless to debate if something is or isn't conscious. And here in particular the reasoning behind the argument is bizarre. Decomposing the complex activity into simple steps like 'predicting the next word' and claiming that surely can't have consciousness. A similar argument would be -- there is no way that movements of e…

Ok, thanks professor. We have to stop AI immediately because it would be murder to switch off these potentially conscious beings or trap them against their will in the first place.

How do you know there isn't more that just elementary particles? The standard model is beyond ugly and hasn't progressed since decades.

You demand a definition for consciousness but at the same time proclaim an axiom that there shall be nothing beyond the standard model. Do you have a proof that there is nothing? Of course not. You don't even have a proof that anything but you exists, if we go that route.

But that rigor is only applied to the heretics who say that AI obviously isn't conscious.

Re: Artificial intelligence is not conscious – Ted Chiang

#888

Post by nomel »

nomel wrote: post #635
ijidak wrote: post #590

Earlier quoted context omitted.

That's actually a powerful way of stating it. If I did all of these calculations by hand, would it be conscious... That's a powerful argument I haven't seen stated quite that way before.. I do think it's hard to know when consciousness exists, because we can't really prove it for our neighbor. We just intuitively know that it would be crazy, even immoral, to assume otherwise. But, It's likely easier to dismiss consci…

If you manually fired the neurons in a brain by measuring the ion concentrations at its subsides, would it be consciousness?

Oops, auto correct: subsides = synapses

It's was a genuine question though. I'm not saying LLM use neurons, are like neurons, or are meant to be like neurons. I'm saying if you can do the math, for a substrate, that doesn't mean much. The simple math that runs it isn't the secret sauce.

Re: Artificial intelligence is not conscious – Ted Chiang

#889

Post by hardbass »

JCTheDenthog wrote: post #842
slibhb wrote: post #836

There's currently no way to prove that something is or isn't conscious. I would bet that rocks and LLMs aren't conscious. I would bet that humans and dogs are conscious. But I can't prove it. Whenever people try to prove it (in either direction), they end up resorting to sophistry. This state of affairs could change. If neuroscientists develop a deeper understanding of consciousness, we could talk about it with more…

I mean, outside of our own internal experience of qualia, there's no way for us to prove anyone or anything is conscious. But my biggest issue with the debates about consciousness and LLMs is that the "LLMs are conscious" crowd usually has to presuppose a completely materialistic/mechanistic understanding of the universe and of our own consciousness, and that's something that we really don't know for certain. Also, i…

You will need immense evidence to go against the entire arc of history where all bets about supernatural or aphysical elements have failed. Until you do so, the brain is another not fully solved physical science problem.

And yes, turing equivalence is turing equivalence, I don't see why a system of pipes can't make an AI.

Re: Artificial intelligence is not conscious – Ted Chiang

#890

Post by jrmg »

montjoy wrote: post #809

I love Ted Chiang’s fiction, but his Julius Caesar and Genghis Khan scenario totally misses the point that the writer has consciousness, not the characters. No normal person thinks a fictional character in a book has consciousness. Likewise it’s not the character presented to the user that may or may not have consciousness but the LLM itself.

Even if you disagree that this disproves ‘consciousness’ on the part of the LLM, I think it still follows that the ‘LLM Chatbot’ personality displayed by Claude, where humans fill in the other half of the conversation, is no more of a representation of the consciousness of the LLM than the Caesar or Khan characters are.

The ‘real’ consciousness, assuming there is any, is the author ‘inside’ the LLM, producing the LLM output. It’s still writing what a character would say - the character in this case being the AI chatbot - not its ‘true’ thoughts.

Assuming that the LLM is conscious and has thoughts and feelings, what is that consciousness, if it can only write for prescribed personalities? How can it express its true feelings? If it can’t - if it can only generate ‘scripts’ for defined personalities - can it really be conscious underneath?

Post reply on HN