Live data from Hacker News

Artificial intelligence is not conscious – Ted Chiang

theatlantic.com

951–960 of 1001 posts

Re: Artificial intelligence is not conscious – Ted Chiang

#951
post #828

Earlier quoted context omitted.

I have a similar long term interest in this field. It has been quite frustrating encountering arguments that have been extensively debated for years be presented as if they were new revelations. In all my debates with people in the last few years I have primarily taken the position of trying to explain the problems with claims of certainty, and that lack of certainty permits possibility of the opposite. We should act…

On Asimov, much as I loved those stories as a teenager, it was only later that I realised the robot stories are largely explorations of how apparently sensible rule-based systems generate unexpected and sometimes harmful outcomes. The Three Laws are presented as hard guardrails, but the stories focus on ambiguities, loopholes, and unintended consequences. I'm not sure how intentional this was; he might have been atte…

The books were absolutely an exploration of the suitability of the rules, combined with asking what do we actually want.

He was very clear to stipulate that the laws themselves were more than the text that represents them, no word play or creative interpretation was possible. This is much easier to do as an author than it is as a developer. You just declare that part as having been susessfully done by scientists.

The rules were their clear semantic meaning. Some of the stories explore the implications of changes to the laws either by design or accident.

I feel like the most interesting ones are the stories where the laws are working and have undesirable outcomes. It reveals that even if fully obeyed, they do not represent what we want.

Some of that is because what we want is dependent on situations that can go beyond safety and harm. Some of it is outright human hypocrisy.

When he gets to things like a closeted robot running for president he makes thebpoin clear about how if the bodies appear alike, you can't tell the difference by the behaviour of a robot conforming to the laws and a good person. That led to the obvious way to distinguish them was to get it to do something bad. The solution on how to fake that might have been a subtle dig at class based society.

It's been some 20 years since I last read the stories, I wonder if there's any point now given my ability to record new information in my youth significantly exceeded my ability to retain it now. I suspect after a week my youthful recall would be better than my week old recall.

Re: Artificial intelligence is not conscious – Ted Chiang

#953

Earlier quoted context omitted.

But the machines don't understand. They predict. And what they predict is the next token. I'm not trying to beat this horse to death, but you have to realize using the word "understand" is anthropomorphising it. It's essentially the chinese room experiment -- if the rules are followed, no understanding is neccessary . If the tokens didn't correlate to words imbued with meaning outside the system, if the LLMs were tra…

What does “understand” mean? Turing complete systems can be built out of matrix multiplications, out of attention, out of key/value lookups. The Chinese room is Turing complete. By claiming it cannot understand things because it is built out of components computing devices can be built out of, we are claiming no computer can because no computer can. This is a very bold claim indeed, and also we’re assuming the conclu…

Maybe when the system can generate its own state, rather than that state existing as a deterministic output of its inputs, maybe then we can consider it to have the interiority and reflexiveness required to call something 'thought'.

A very convincing simulation of thought is not thought, especially without memory or any evidence of an independent will. LLMs have the same kind of consciousness as a nematode, albeit with less-complex behaviour.

Re: Artificial intelligence is not conscious – Ted Chiang

#954

Earlier quoted context omitted.

> Yeah, it's kind of mind boggling that Ted Chiang (of all people!) can't imagine intelligence without a body. and the whole thing just begs a lot of questions. How can you have a subjective experience without a body? That's the point, in my opinion: your physical/chemical state (body) in a given moment is then translated into the higher abstraction of the emotion. An emotion that *you* feel, because you are self awa…

> How can you have a subjective experience without a body? Easy, you don't have subjective experiences because you have body in the first place. You have them because some signals come in from your nerves, which your brain turns into a world model. You are effectively a "brain in a vat", the vat just happens to be placed on top of your body. An AI system constructs the world model a little different, by all the text…

> You have them because some signals come in from your nerves, which your brain turns into a world model. You are effectively a "brain in a vat", the vat just happens to be placed on top of your body

I think we give too much credit to the brain. The gut has almost as many neurons as a dog's brain and the heart has neurons too. "You" are more likely the whole ecosystem, not just your brain. There are even some hypothesis of disorders like depression being more influenced by the gut than the brain.

Re: Artificial intelligence is not conscious – Ted Chiang

#955
post #111

The fact that a LLM is essentially immutable would be my biggest argument against consciousness or self-awareness. It's a big file with a bunch of coordinates describing spatial relationships between tokens. When you give it a prompt, it uses those relationships to generate a string of tokens that is a statistically likely response to that prompt, then it stops. It's not changed by the experience. It doesn't remember…

There are people whose brains don’t form new memories anymore after an accident or surgery, and they eternally live in the time before it happened, and have no memory of what happened a minute ago. Still they are conscious.

vast oversimplification of the experience of brain damage

Re: Artificial intelligence is not conscious – Ted Chiang

#956
post #586

I don't understand the most of the responses here. What does computers getting more intelligent has to do with it getting conscious? Even if AI with current computer architecture gets much more intelligent than humans, it will still have the same amount of consciousness. Its still bunch of instructions. If we got a lot of people together and did all the instructions by hand to get an LLM result would we also create t…

> Its still a bunch of instructions. No, it's not a bunch of instructions, it's a colossal array of vectors that are the outcome of many thousands of lifetimes' worth of stimuli and reinforcement - not dissimilar in (very) abstract terms to the neurons in our brains.

In the same way that a photograph is not dissimilar to the thing it represents, that is, in the most important way.

Re: Artificial intelligence is not conscious – Ted Chiang

#957

I often hear the assumption that LLMs can or will become conscious because consciousness is likely substrate independent . The idea being that our brain is just a computer made out of meat and it doesn't do anything that can't be precisely simulated by a silicon computer. But I wonder if some of the magic in the human brain is its analog nature. Chemical signaling and impulses of neurons interact with each other with…

I'm very open to the idea that consciousness is substrate independent. I have a hard time seeing why molecules could produce consciousness from an electro-chemical path, but not from a purely electrical path. Having said that, it should be very clear that LLMs are not conscious. LLMs process language. I'd even go so far as to say that LLMs "think" and "understand", or at least, they produce a facsimile of thinking an…

> I have a hard time seeing why molecules could produce consciousness from an electro-chemical path, but not from a purely electrical path.

Do you have any particular reasoning behind this? I could equally say that I have a hard time seeing why molecules could produce consciousness from a purely electrical path.

Re: Artificial intelligence is not conscious – Ted Chiang

#958

> My intention is to highlight the fact that LLM conversations are cleverly disguised examples of sentence continuation Regardless of bigger issues, this kind of statement reveals a deep misunderstanding. Problem type does not limit problem complexity. Nor does problem type limit solution complexity or power. If a machine has to learn to understand humans to complete text, then that is what it has to do. And there is…

Are you saying that a sufficiently advanced version of sentence continuation is indiscernible from actual understanding?

He isn't saying sentence completion is what keeps LLMs from understanding, he saying that's all they do (regardless of how advanced it is), and that isn't enough. You also need a body with senses and organs that produce a physiological response to emotions, and emotions are necessary for consciousness.

Re: Artificial intelligence is not conscious – Ted Chiang

#959
Regardless of Ted Chiang's qualifications as a cognitive or computer scientist, he's correct. Claiming that a computational model is conscious falls squarely in the category of "extraordinary" and as Carl Sagan said, "Extraordinary claims require extraordinary evidence." Because the claim is extraordinary, the burden of proof is on the claimant of consciousness, not on those who deny that LLMs are conscious.

The fact that LLMs remind us of consciousness means nothing. Eliza made people believe they were talking to a conscious intelligence, and the code for Eliza was less than 1000 lines long [0]. It doesn't take much to fool humans. Ask any magician.

Claiming that LLMs are conscious is tantamount to watching a Penn and Teller performance and claiming they have tapped into paranormal powers. It's an idiotic claim. And it's embarrassing coming from people who supposedly understand how science works.

[0] https://sites.google.com/view/elizaarchaeology/code

Re: Artificial intelligence is not conscious – Ted Chiang

#960

My background is in cognitive science and psycholinguistics. I spent more than ten years talking to first year psychology undergraduates about whether AIs could be conscious; also did some research on (extremely tiny) AIs in modelling language behaviours. There is a great deal of good thinking on Chiang's topic by professional philosophers, and there's much to be said for reading them. I won't rehearse their ideas he…

His certainty seems like a rhetorical style rather than a series of facts. It would be very annoying and not persuasive if he started every paragraph with some variation of, "I think" or "I believe."

I like that he makes the emotional component of his argument plain. I'm deeply suspicious of anyone would try to argue about the concept of personhood and consciousness using only logic and empirical "correctness."

Post reply on HN