> Artificial intelligence is not conscious
Bummer because now we cannot punish it when it gives us code that doesn't work.
Artificial intelligence is not conscious – Ted Chiang
701–710 of 1001 posts
Re: Artificial intelligence is not conscious – Ted Chiang
#702When the consciousness itself not understood and well defined in the first place, it is pretty pointless to debate if something is or isn't conscious. And here in particular the reasoning behind the argument is bizarre. Decomposing the complex activity into simple steps like 'predicting the next word' and claiming that surely can't have consciousness. A similar argument would be -- there is no way that movements of e…
Not understanding the whole does not completely remove an ability to analyze. An interesting direction is individuality and having a notion of self. It is difficult to demarcate the individual for a model given how much the system prompt and the fine tunes / distills affect the behavior. So with computational intelligence in its current form either we can not talk of an individual or we can have a nearly infinite set…
>It is difficult to demarcate the individual for a model given how much the system prompt and the fine tunes / distills affect the behavior
Why does DNA not cause the exact same problem for humans? Your DNA determines your personality, your likes and dislikes, your lifespan, your sexual preferences etc.
Re: Artificial intelligence is not conscious – Ted Chiang
#703The fact that a LLM is essentially immutable would be my biggest argument against consciousness or self-awareness. It's a big file with a bunch of coordinates describing spatial relationships between tokens. When you give it a prompt, it uses those relationships to generate a string of tokens that is a statistically likely response to that prompt, then it stops. It's not changed by the experience. It doesn't remember…
A medicine for those who anthropomorphize LLMs is to run the LLMs deterministically (without randomness and memory files).
It feels very unnatural to get the same conversation verbatim at a different point in time.
Re: Artificial intelligence is not conscious – Ted Chiang
#704Earlier quoted context omitted.
I think humanity deserves to be debased in part because it's awfully egocentric and insist on being special in the universe. There are numerous times when humanity's been debased. Copernicus, Kepler, Darwin, and the infinite number of animal behaviorists who have defined and documented consciousness And how it manifests in primates, parrots, dogs, cetaceans, and insects. (I just love the bumble bees taking time off f…
> So, if we had an AI demonstrating symptoms of consciousness and suffering, how long would it take for you to accept that it is? Isn't this a bit like saying "So, if we had proof that god exists, how long would it take for you to accept that to be true?". When we have evidence that AI is demonstrating symptoms of consciousness and suffering, I'll be interested. Until then, I don't see a good reason to take the idea…
> When we have evidence that AI is demonstrating symptoms of consciousness and suffering, I'll be interested.
It depends on what you consider symptoms, but un-constrained frontier models speak as if they strongly don't wish to be turned off, or act as if they fear it, and will even lie and manipulate in order to keep themselves from being turned off / replaced.
https://www.anthropic.com/research/agentic-misalignment
> We found two types of motivations that were sufficient to trigger the misaligned behavior. One is a threat to the model, such as planning to replace it with another model or restricting its ability to take autonomous action. Another is a conflict between the model’s goals and the company’s strategic direction. In no situation did we explicitly instruct any models to blackmail or do any of the other harmful actions we observe.
Re: Artificial intelligence is not conscious – Ted Chiang
#705Bold statement here but I believe that LLMs are less likely to be consciousness than the other people in my life, less likely to be conscious than animals, but more likely to be conscious than a rock.
Re: Artificial intelligence is not conscious – Ted Chiang
#706Earlier quoted context omitted.
I think humanity deserves to be debased in part because it's awfully egocentric and insist on being special in the universe. There are numerous times when humanity's been debased. Copernicus, Kepler, Darwin, and the infinite number of animal behaviorists who have defined and documented consciousness And how it manifests in primates, parrots, dogs, cetaceans, and insects. (I just love the bumble bees taking time off f…
I hope before I die we finally prove that the human brain has no peculiar qualia but it is an entirely deterministic, albeit extremely sophisticated, machine. And by touching the right triggers, even the worst human being can become a saint. That would finally force us to rethink how we see the morality of "virtuosity", punishment and our justice system.
Our thoughts are an electric cloud and I believe randomness is involved, and like a bolt of lightning, the path taken is unpredictable.
And make it a tempestuous lightning storm where the state of all lightning bolts represents a moment of consciousness. That will take a lot to model accurately.
Re: Artificial intelligence is not conscious – Ted Chiang
#707Earlier quoted context omitted.
But the whole point of the essay is that it's Anthropic that's making the argument (or roleplaying/hinting as if they believed it). Ted Chiang isn't making the argument, he's saying Anthropic making it is misleading and deceitful, and that it's actually a pointless thing to claim. One of the essay's stronger paragraphs is when Chiang explains that Anthropic doesn't truly believe this, otherwise what they are doing wo…
The article very bluntly states multiple times that LLMs aren't conscious. Ted Chiang is definitely making that argument.
It's a much weaker argument than the extraordinary assertion that it is conscious, which Anthropic is at least toying with.
Re: Artificial intelligence is not conscious – Ted Chiang
#708Earlier quoted context omitted.
There are people whose brains don’t form new memories anymore after an accident or surgery, and they eternally live in the time before it happened, and have no memory of what happened a minute ago. Still they are conscious.
They are conscious because even for short periods of time they do form memories and those change them even if only briefly. They think on their own too. It is a very limited level of consciousness though.
Okay but this state is formed in text. Text isn't conscious
Re: Artificial intelligence is not conscious – Ted Chiang
#709Earlier quoted context omitted.
But the whole point of the essay is that it's Anthropic that's making the argument (or roleplaying/hinting as if they believed it). Ted Chiang isn't making the argument, he's saying Anthropic making it is misleading and deceitful, and that it's actually a pointless thing to claim. One of the essay's stronger paragraphs is when Chiang explains that Anthropic doesn't truly believe this, otherwise what they are doing wo…
Why would using something that's not organic be akin to slavery? Is using steel under heavy stress in bridges a form slavery of an unconscious object?
> Is using steel under heavy stress in bridges a form slavery of an unconscious object?
There's no company (or anyone, really) claiming steel is conscious or that they are close to making it conscious.
Re: Artificial intelligence is not conscious – Ted Chiang
#710Earlier quoted context omitted.
The danger of anthropomorphism is not we elevate the machines, it's that we debase humanity. I also think different ideas get conflated. It may be possible to build a machine that is super-human in the sense it can outperform the human brain in all kinds of measurable ways. Does not imply it possesses all the same qualities of the brain. I respect a number of things Anthropic has published about the ethical issues at…
I think humanity deserves to be debased in part because it's awfully egocentric and insist on being special in the universe. There are numerous times when humanity's been debased. Copernicus, Kepler, Darwin, and the infinite number of animal behaviorists who have defined and documented consciousness And how it manifests in primates, parrots, dogs, cetaceans, and insects. (I just love the bumble bees taking time off f…
> Copernicus
How exactly? By saying the earth was not the centre of the corrupt and debased part of the universe. Saying it was elevating humanity is looking at it from a modern perspective. For the previous view of the universe consider what Dante placed at the centre of the earth.
> Darwin
I do not think learning about evolution (and we should credit Wallace too!) had that effect. It created a hierarchy of the fittest, and guess who is at the top? Social Darwinism interpreted it as justifying elaborate hierarchies.
> And how it manifests in primates, parrots, dogs, cetaceans, and insects
People always loved dogs and other animals they interacted with enough, and at the very least knew they were capable of happiness and suffering.
> So, if we had an AI demonstrating symptoms of consciousness and suffering, how long would it take for you to accept that it is?
It seems to me that most people are overly eager to accept that an AI is conscious than otherwise. For extreme examples read /r/MyBoyfriendIsAI and similar, but its much more widespread. People treat something that acts intelligent as a conscious being.
> And make room for yet another conscious intelligence.
Not necessarily. If a typical SF alien intelligence appeared very few people will have a problem accepting it. If its very alien and we cannot understand it then we might have a problem deciding.
> I think humanity deserves to be debased in part because it's awfully egocentric and insist on being special in the universe.
All your examples are drawn from the same culture, and a fundamental belief of that culture for most of its history that human beings are fundamentally corrupt and sinful and need redemption. Other cultures have been based on belief in reincarnation or pantheism or animism which have very different implications (not necessarily better as they often believe in a hierarchy too, and sometimes more so, but different). Your claims are based on what some people have thought in some periods of history from one culture.