Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Artificial intelligence is not conscious – Ted Chiang

theatlantic.com

Post reply on HN
Print view View original

391–400 of 1001 posts

Re: Artificial intelligence is not conscious – Ted Chiang

#391

Post by NiloCK »

pdonis wrote: post #387
barryfandango wrote: post #384

It's a waste of time to think about whether an LLM has a subjective experience of reality, and this handily sets aside issues like AI rights. But the fact remains that these next-token-predictors exhibit objective, human-like behaviours, and for that reason the work of in-house philosopher Amanda Askell _is_ important. It's important that Claude is happy, empathic, demonstrates understanding and empathy for the human…

> Claude is happy, empathic, demonstrates understanding and empathy for the human condition You're assuming that because Claude produces text that appears to express these qualities, Claude must have them. I don't think that's a good assumption. Even many humans produce text that has the same appearance, but don't actually have those qualities--which becomes clear when you look at what they do, not what they say. So…

> You're assuming that because Claude produces text that appears to express these qualities, Claude must have them.

Not to be confrontational, but the OP assumed no such thing. OP asserted that it's important for Claude to have the qualities - not that it's important for Claude to present as-if it had them.

Re: Artificial intelligence is not conscious – Ted Chiang

#392

Post by thwarted »

hackinthebochs wrote: post #350
hn_acc1 wrote: post #305

Earlier quoted context omitted.

It can't even natively understand how many letters there are in words - how will it understand the meaning?

I wish people would do even the most basic amount of research into LLMs before opining about what they can or cannot do. There are very principled reasons why LLMs do not know how many letters are in words, and it says nothing about their facility for understanding meaning. Tokens are the most basic input unit of an LLM. But tokens don't generally correspond to words or letters, rather sub-word sequences. So Strawber…

> There are very principled reasons why LLMs do not know how many letters are in words, and it says nothing about their facility for understanding meaning. … Tokens are the most basic input unit of an LLM. But tokens don't generally correspond to words or letters, rather sub-word sequences. So Strawberry might be broken up into two tokens 'straw' and 'berry'.

This sounds like a description of a child who has not learned to read yet. You ask a child who is not aware of the alphabet and of "words" how many r's are in strawberry you'd get a non-sense answer too. So what you're really pointing out is that the LLMs have not been trained on "the english language" and how words are constructed and what they are composed of. That they operate by tokens that don't correspond to words or letters is irrelevant as an answer to why they can't count the letters in a word. It's not that I know how many r's are in strawberry because of how I'm understanding the word "strawberry", I know how many r's are in strawberry because I know how to spell strawberry. The LLM needs to be trained on this the same way someone who is learning to read would be trained on it. No one should be surprised that an LLM can't "read" in the same way no one should be surprised that a child can't "read".

Re: Artificial intelligence is not conscious – Ted Chiang

#393

Post by Terr_ »

dnautics wrote: post #368
cauch wrote: post #254

Earlier quoted context omitted.

I think, for me, the thing is that when you do basic ML, you discover that ML will very often find data pattern that fit the goal but does not correspond to a real mechanism. So, I think there is a flaw in the logic of saying that human text have a pattern of "consciousness mechanism" and therefore LLM will learn "consciousness mechanism" in order to return sentence continuation that is convincing. There is probably…

I think, for me, the thing is that when you tutor undergrads in abstract math, you discover that students will very often find data pattern that fit the goal but does not correspond to a real mathematical principle. sometimes humans making claims about AI intelligence or consciousness also identify spurious patterns that do not correspond to the problems of intelligence or hard consciousness.

> students will very often find data pattern that fit the goal but does not correspond to a real mathematical principle.

That reminds me of a niche paper [0] critiquing a certain way of teaching remedial math that was over-focused on tests. A kid named Benny (12) was building up (wrong) "rules" for math which still somehow gave enough of an illusion of progress in terms of test scores that his misunderstandings hadn't been caught earlier.

> Benny was able to explain his procedure; e.g. for 5/10=1.5, he said: "The one stands for 10; the decimal; then there’s 5... shows how many ones." In another example, 400/400 = 8.00 because "The numbers are the same [number of digits]... say like 4000 over 5000. All you do is add them up; put the answer down; then put your decimal in the right place... in front of the [last] three numbers."

[0] https://people.wou.edu/~girodm/library/benny.pdf

Re: Artificial intelligence is not conscious – Ted Chiang

#394

Post by verdverm »

skybrian wrote: post #35

I think Chiang is right about this, but there is a related philosophical mystery. The trend from Deep Blue to AlphaGo to LLMs solving Erdos problems suggests that Peter Watts was onto something when he wrote Blindsight. Reasoning ability is apparently independent of consciousness? We haven't really come to grips with that yet. What does it mean if nothing we write proves anything about anyone's consciousness?

Ai separates language and reasoning from humans, like writing separated memory from us

"AI does to us what American Cheese did to food" ~ Opus 23

https://www.youtube.com/watch?v=1CbmC2aWhjY

Re: Artificial intelligence is not conscious – Ted Chiang

#395

Post by Nevermark »

dogwalker5000 wrote: post #388
Nevermark wrote: post #194

> My intention is to highlight the fact that LLM conversations are cleverly disguised examples of sentence continuation Regardless of bigger issues, this kind of statement reveals a deep misunderstanding. Problem type does not limit problem complexity. Nor does problem type limit solution complexity or power. If a machine has to learn to understand humans to complete text, then that is what it has to do. And there is…

> If a machine has to learn to understand humans to complete text, then that is what it has to do. And there is no theoretical or practical basis for suggesting that this is somehow "faking" understanding, just because of the form of original data streaming in and out. I think the main complaint is LLMs don’t arrive at the answer the way we do. It’s capable of emulating some of our behavior but not all as the mechani…

> I think the main complaint is LLMs don’t arrive at the answer the way we do.

This isn't an argument against their understanding things.

But I expect you are right, that their understanding may have major different qualities from ours.

Along with significant commonalities. (They don't reason via stream of consciousness in a way alien to us.)

Re: Artificial intelligence is not conscious – Ted Chiang

#396

Post by sfn42 »

grumbel wrote: post #207
gabrieledarrigo wrote: post #147

Earlier quoted context omitted.

> Yeah, it's kind of mind boggling that Ted Chiang (of all people!) can't imagine intelligence without a body. and the whole thing just begs a lot of questions. How can you have a subjective experience without a body? That's the point, in my opinion: your physical/chemical state (body) in a given moment is then translated into the higher abstraction of the emotion. An emotion that *you* feel, because you are self awa…

> How can you have a subjective experience without a body? Easy, you don't have subjective experiences because you have body in the first place. You have them because some signals come in from your nerves, which your brain turns into a world model. You are effectively a "brain in a vat", the vat just happens to be placed on top of your body. An AI system constructs the world model a little different, by all the text…

Its a computer program. It is literally just a lot of zeroes and ones, sitting there doing nothing.

Then a request comes in, and the system does a bunch of calculations using those bits, and spits out a result. The bits are unchanged.

When your brain receives input, it is changed. It is constantly active. If it ever stops being active it's dead.

So, what exactly is the claim? Are the bits constantly conscious? Do they snap into consciousness when the computer does math with them? Or is it maybe the computer that's conscious while it's processing these bits? How about when it stops doing that and goes back to doing other stuff? Why are these particular bits special? Was the computer always conscious?

I feel like the only way anyone could believe LLMs are conscious is if they don't understand how computers work. Of course it isn't conscious, how could it possibly be conscious? Its literally just bits. It's like saying the text in a book is conscious.

Re: Artificial intelligence is not conscious – Ted Chiang

#397

Post by Muskwalker »

felipeerias wrote: post #257
lwarfield wrote: post #127

Earlier quoted context omitted.

>Yeah, it's kind of mind boggling that Ted Chiang (of all people!) can't imagine intelligence without a body. and the whole thing just begs a lot of questions. Damn, what a line! Another thing that bothered me with his baseline for consciousness was that it did not involve the ability to change one's self. A big part of being conscious in my mind is how one's experiences shape them, and how someone can shape themselv…

A stronger version of that argument is that LLMs are not intrinsically affected by the passage of time. Input stream comes in, input stream comes out. The LLM doesn't care whether this happens once a minute or once a year.

I don't normally experience the time interval between when my input streams shut down every night and reboot every morning either, though.

Nothing prevents one from running an LLM whose harness has a clock and a while loop in it, and it would be weird if its mere lack were really so consequential to consciousness.

Re: Artificial intelligence is not conscious – Ted Chiang

#398

Post by jvanderbot »

chairhairair wrote: post #370
jvanderbot wrote: post #220

Earlier quoted context omitted.

I think about this from the other end. It cannot be considered a conscious being. There just isn't a world in which we should start to think of a machine using ethics we reserve for humans. AI is essentially infinitely reproducible at zero cost, and won't suffer from decay etc. There's not scarcity to preserve. So, I'd turn off an AI in a moment to save property or real possessions or money. I'd sacrifice property an…

Consider that it is very costly to train a model. It is only cheap to copy because the substrate we instantiate it onto is a substrate we have designed to be fully readable. I think you should reconsider this viewpoint. Suppose that we really can create silicon-based consciousness, in that case your view would result in a huge amount of suffering. Take some other basis for dismissing digital consciousness, this one i…

How much data center energy and capex justifies killing a human to save?

I argue zero - placing AI below the value of humans no matter the energy input.

The _only_ reason an AI might be worth saving is if it, say, has a cure for all diseases, but then we're not saving it due to its intrinsic worth, we're saving it because we can save many humans. I _would_ consider the trolly problem a legitimate thing in ethics, but not if an AI were tied up on the tracks no matter how expensive it is. It's a thing. It gets run over to save any human.

Re: Artificial intelligence is not conscious – Ted Chiang

#400

Post by rmunn »

in-silico wrote: post #366
CommieBobDole wrote: post #104

The fact that a LLM is essentially immutable would be my biggest argument against consciousness or self-awareness. It's a big file with a bunch of coordinates describing spatial relationships between tokens. When you give it a prompt, it uses those relationships to generate a string of tokens that is a statistically likely response to that prompt, then it stops. It's not changed by the experience. It doesn't remember…

> It's not changed by the experience The entire file is not changed, but the KV cache is. > It doesn't remember anything The model definitely remembers previous exchanges within the same conversation.

> The model definitely remembers previous exchanges within the same conversation.

No it doesn't. They get added to its context, and it reads them afresh when answering the next question. That's not remembering.

If your short-term memory completely malfunctioned one day, so you had no ability to remember what was said to you a minute ago, then you would have to find workarounds. For example, you could write down everything someone says to you, then read your notes of the previous exchanges in that conversation in order to continue the conversation. That would be a good way to work around the fact that your short-term memory was broken. And if your notes were invisible to other people and you could read them really fast, then you could even make most people believe that you remembered what they said a minute ago. But you don't actually have a working memory, you're just writing down what they said and re-reading it while coming up with your next response.

That's exactly what LLMs do. That's not memory.

Post reply on HN