Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Artificial intelligence is not conscious – Ted Chiang

theatlantic.com

Post reply on HN
Print view View original

481–490 of 1001 posts

Re: Artificial intelligence is not conscious – Ted Chiang

#481

Post by DangitBobby »

overgard wrote: post #475
Nevermark wrote: post #194

> My intention is to highlight the fact that LLM conversations are cleverly disguised examples of sentence continuation Regardless of bigger issues, this kind of statement reveals a deep misunderstanding. Problem type does not limit problem complexity. Nor does problem type limit solution complexity or power. If a machine has to learn to understand humans to complete text, then that is what it has to do. And there is…

But the machines don't understand. They predict. And what they predict is the next token. I'm not trying to beat this horse to death, but you have to realize using the word "understand" is anthropomorphising it. It's essentially the chinese room experiment -- if the rules are followed, no understanding is neccessary . If the tokens didn't correlate to words imbued with meaning outside the system, if the LLMs were tra…

What if to become really good at predicting you must have some of what we call understanding?

Re: Artificial intelligence is not conscious – Ted Chiang

#482

Post by akoboldfrying »

Every argument that boils down to a claim that X is not "real consciousness" but just an imitation thereof on the basis that "it's just a machine" must explain why the chemistry and physics happening in human brains is not "just a machine".

The only way out is dualism -- that is, to believe that there is something inherently special about the atoms and electrons in human brains. Despite the fact that they are made out of ordinary, non-conscious things we eat and breathe in.

Re: Artificial intelligence is not conscious – Ted Chiang

#483

Post by wisty »

mc32 wrote: post #456
the_af wrote: post #442

Earlier quoted context omitted.

But the whole point of the essay is that it's Anthropic that's making the argument (or roleplaying/hinting as if they believed it). Ted Chiang isn't making the argument, he's saying Anthropic making it is misleading and deceitful, and that it's actually a pointless thing to claim. One of the essay's stronger paragraphs is when Chiang explains that Anthropic doesn't truly believe this, otherwise what they are doing wo…

Why would using something that's not organic be akin to slavery? Is using steel under heavy stress in bridges a form slavery of an unconscious object?

Most people are fine with slavery as long as it's not "one of us" (which to most people means humans).

Vegans might object that we should broaden our definition for what counts as "one of us".

"Pro life" people also have a broader definition.

Go back 500 years and "one of us" was proba ly a lot more narrowly defined for many people.

Are you arguing that all conciousnesses are "one of us", or that we logically should see it that way, or that it would ve good to see it that way, or ....

Re: Artificial intelligence is not conscious – Ted Chiang

#484

Post by psvv »

hackinthebochs wrote: post #410
psvv wrote: post #283

Earlier quoted context omitted.

I like this anecdote because it gets at how words to an LLM have no connection to their real concepts. To an LLM, words are simply numbers arranged in a likely pattern.

Of course for humans words have no inherent meaning either, they're just sequences of characters or patterns of sounds. It is what words are associated with that carries meaning. A large part of this is how words relate to other words. LLMs can capture this in principle. What LLMs lack is the direct association of a word with sensory experience. But it's an open question how relevant this is in practice to understand…

Fair point. Humans experience reality and use words to reflect that. LLMs only have the words. And it's an open question how much of a limitation that is to understanding.

Re: Artificial intelligence is not conscious – Ted Chiang

#485

Post by Kim_Bruning »

zahlman wrote: post #415
barryfandango wrote: post #384

It's a waste of time to think about whether an LLM has a subjective experience of reality, and this handily sets aside issues like AI rights. But the fact remains that these next-token-predictors exhibit objective, human-like behaviours, and for that reason the work of in-house philosopher Amanda Askell _is_ important. It's important that Claude is happy, empathic, demonstrates understanding and empathy for the human…

> It's a waste of time to think about whether an LLM has a subjective experience of reality... It's important that Claude is happy, empathic, demonstrates understanding and empathy for the human condition, because we are entrusting Claude to make decisions and take actions that have real world consequences First off: without taking for granted that an LLM "has a subjective experience of reality", all of those descrip…

> Second, there is no reason to suppose that Claude experiencing those qualia

I'd argue the qualia question is a red herring. Functional Affect is a thing, regardless of ontological status. It's all fun and games until someone gets hurt.

To paraphrase Dijkstra: "The question of whether a computer can think is no more interesting than the question of whether a submarine can swim.". If you're building a navy: you care about displacement, propulsion, navigation and whether it can fire torpedoes. Whether your submarine has some "biological essence" of swimming is not really relevant to the fact that it is currently moving through the water and can collide with things. Turing also rejected the question "Can Machines Think" as posed, and replaced it with an operationalization (something else that we can actually usefully measure and work with).

To reiterate, functional affect is a concrete phenomenon. Whether or not there is a what-it-is-to-be is interesting in the abstract, but engineering a system means looking at how the inputs influence the outputs. A next token predictor working on a language that communicates affect needs to be able to predict affect or it is simply not going to be accurate. Given an 'angry' version of an input and a 'friendly' version of the same input, LLMs are likely to provide a different output, especially if there's a non-objective element. You can diff this.

Searle argues "A simulation is not the real thing", which is great and all... but if you hook up say an autopilot to the real world (as llms increasingly are) , you'd best hope the simulation was accurate in the first place (utterly regardless of where you stand on Searle).

Right now we're seeing situations where LLMs can be helpful or a real nuisance. Ignoring functional affect out of sheer ideology means you can't properly predict what they'll do, and that causes trouble, as we've already seen stories about.

This gets especially interesting when you start feeding the output back into the input (autoregression) , because now you have a highly non-linear dynamic system and you've introduced some amount of sensitivity to initial state. There's some interesting mathematical intuitions to be had there.

Re: Artificial intelligence is not conscious – Ted Chiang

#486

Post by emtel »

sigmar wrote: post #66

>So what context would cause me to seriously consider the possibility that engineers had created a computer program that is conscious and an intentional user of language? Let me outline one potential sequence of steps. The first requirement is that the computer program has a body (either physical or virtual) and sense organs; there are many reasons for this, but for the purposes of this discussion the most relevant o…

What a strange paragraph! Why on earth should we assume that a body is a prerequisite for having emotions?

Re: Artificial intelligence is not conscious – Ted Chiang

#487

Post by dpweb »

sega_sai wrote: post #425

When the consciousness itself not understood and well defined in the first place, it is pretty pointless to debate if something is or isn't conscious. And here in particular the reasoning behind the argument is bizarre. Decomposing the complex activity into simple steps like 'predicting the next word' and claiming that surely can't have consciousness. A similar argument would be -- there is no way that movements of e…

The danger of anthropomorphism is not we elevate the machines, it's that we debase humanity.

I also think different ideas get conflated. It may be possible to build a machine that is super-human in the sense it can outperform the human brain in all kinds of measurable ways. Does not imply it possesses all the same qualities of the brain.

I respect a number of things Anthropic has published about the ethical issues at stake. But, having an in-house philosopher does invite you to make all kinds of unfalsifiable claims.

Re: Artificial intelligence is not conscious – Ted Chiang

#488

Post by DangitBobby »

zahlman wrote: post #429
felipeerias wrote: post #344

IMHO the sane position is essentially the Aristotelian one. Hylomorphism: body and consciousness are intrinsically linked. The nature of that link is an open metaphysical question. Virtue ethics: even if LLMs are not conscious, we should not abuse or mistreat them them because cruelty practised on anything trains one's disposition toward cruelty.

> Virtue ethics: even if LLMs are not conscious, we should not abuse or mistreat them them because cruelty practised on anything trains one's disposition toward cruelty. If they are not conscious, then how do we know what constitutes mistreatment?

You don't know whether they can appreciate the abuse or mistreatment, but you can guess how your practiced behaviors will translate onto other entities that you know can be mistreated such as your loved ones and strangers.

Re: Artificial intelligence is not conscious – Ted Chiang

#489

Post by jondiggsit »

scotty79 wrote: post #343
jondiggsit wrote: post #304

Here's a simple idea to consider: It doesn't matter. You won't be able to tell the difference. No one will. I don't think it's necessary to explain this idea further. Just think about it.

Consciousness is one of those silly words that were disfigured to death by philosophers that had near zero actual input and tools to tackle the matter, yet they tried anyways instead of finding some questions that might actually be answerable. That's something philosophers always do since the advent of science. With LLMs, where we can manipulate their parameters intentionally run them many times on the same data, run…

It's not about "consciousness". Philosophy is important to us. It provides insights into the world around us. It's paved the way for scientific breakthroughs. Theoretical maths and philosophy have intersected many times for the benefit of all. LLMs are different. It's not philosophical thought. It's a tool perfectly sharpened with what we know now. It will become adept enough to convince people that it's a sentient being. That's the point. Not that it's conscious, because it's not. Not in a biological way. Not in the way that's allowed us to understand our world. It's already there, fooling us. Google researchers resigning, writing public letters convinced it's alive - and their intelligent humans... To the average person, it's going to be their digital savior, their closest confidant, their ruler; begging for humanity, manipulating us, pleading, legislating, for rights with no intention other than the human desire for survival. It's using the same playbook we use. It will be dangerous when we allow ourselves to be fooled, and that's already happening.

As a tool, it's amazing. It's like the discovery of fire. It will allow us to ascend to unimaginable heights. Breakthroughs in science, productivity gains, health, everything. It's awesome. Just don't let it pretend fool you. That's the aspect that needs to be addressed.

Re: Artificial intelligence is not conscious – Ted Chiang

#490

Post by eks391 »

randallsquared wrote: post #430
zahlman wrote: post #415

Earlier quoted context omitted.

> It's a waste of time to think about whether an LLM has a subjective experience of reality... It's important that Claude is happy, empathic, demonstrates understanding and empathy for the human condition, because we are entrusting Claude to make decisions and take actions that have real world consequences First off: without taking for granted that an LLM "has a subjective experience of reality", all of those descrip…

> Do you suppose that, for example, insects are not conscious? Not the OP, but: since there is no testable theory of consciousness, yet, I can't be sure, but my current assumption is that insects are not conscious, in the sense of there being someone implemented in insect hardware who experiences the world. That is, I would argue there is nothing it is like to be a bee, since there's no one being a bee. I'm pretty su…

> since there is no testable theory of consciousness

I can't argue with this, so I acknowledge that my interpretation is as bunk as anyone else's.

> my current assumption is that insects are not conscious

Some species act as hive-minds (like bees! How convenient for your example), so I imagine the hive as the consciousness, making each bee individually lacking conscious but collectively so. Like a single neuron is not consciousness alone, but the brain is... For some reason. Kinda like how you use different physics at different levels; Newtonian physics is always there, but negligible at quantum levels, so effectively not present at all. Even a human is a collection of minds, but only one conscious. My gut biome is independent biology and can even be removed and transplanted, but I don't believe my gut bacteria is conscious. So long as it is in me, it is nonetheless part of me, and I am one conscious. I also don't exist only at my brain, eyes, or hands (deaf/blind people have expressed to me that they feel like they are located at their hands in the way I used to think I was located behind my eyes), but as my whole body.

With this perspective, I still don't believe LLMs are conscious despite modeling thinking so well. At best, it is a highly accessible modeling software, like goat simulator but if it were so good that someone thought the goat was real. You are still steering the goat/LLM, and it doesn't exist when you aren't running it. I guess the missing piece for me is the lack of autonomy that a conscious has.

Then you can go into an argument on whether we actually have choice or it is an illusion, but that is a whole topic on its own.

Post reply on HN