Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Artificial intelligence is not conscious – Ted Chiang

theatlantic.com

Post reply on HN
Print view View original

511–520 of 1001 posts

Re: Artificial intelligence is not conscious – Ted Chiang

#511

Post by nearbuy »

the_af wrote: post #442
sega_sai wrote: post #425

When the consciousness itself not understood and well defined in the first place, it is pretty pointless to debate if something is or isn't conscious. And here in particular the reasoning behind the argument is bizarre. Decomposing the complex activity into simple steps like 'predicting the next word' and claiming that surely can't have consciousness. A similar argument would be -- there is no way that movements of e…

But the whole point of the essay is that it's Anthropic that's making the argument (or roleplaying/hinting as if they believed it). Ted Chiang isn't making the argument, he's saying Anthropic making it is misleading and deceitful, and that it's actually a pointless thing to claim. One of the essay's stronger paragraphs is when Chiang explains that Anthropic doesn't truly believe this, otherwise what they are doing wo…

The article very bluntly states multiple times that LLMs aren't conscious. Ted Chiang is definitely making that argument.

Re: Artificial intelligence is not conscious – Ted Chiang

#512

Post by D_Alex »

dpweb wrote: post #487
sega_sai wrote: post #425

When the consciousness itself not understood and well defined in the first place, it is pretty pointless to debate if something is or isn't conscious. And here in particular the reasoning behind the argument is bizarre. Decomposing the complex activity into simple steps like 'predicting the next word' and claiming that surely can't have consciousness. A similar argument would be -- there is no way that movements of e…

The danger of anthropomorphism is not we elevate the machines, it's that we debase humanity. I also think different ideas get conflated. It may be possible to build a machine that is super-human in the sense it can outperform the human brain in all kinds of measurable ways. Does not imply it possesses all the same qualities of the brain. I respect a number of things Anthropic has published about the ethical issues at…

>I also think different ideas get conflated.

Yep. For example, your post conflates the idea of having a brain with the idea of having consciousness.

Re: Artificial intelligence is not conscious – Ted Chiang

#513

Post by D_Alex »

nearbuy wrote: post #511
the_af wrote: post #442

Earlier quoted context omitted.

But the whole point of the essay is that it's Anthropic that's making the argument (or roleplaying/hinting as if they believed it). Ted Chiang isn't making the argument, he's saying Anthropic making it is misleading and deceitful, and that it's actually a pointless thing to claim. One of the essay's stronger paragraphs is when Chiang explains that Anthropic doesn't truly believe this, otherwise what they are doing wo…

The article very bluntly states multiple times that LLMs aren't conscious. Ted Chiang is definitely making that argument.

[flagged]

Re: Artificial intelligence is not conscious – Ted Chiang

#515

Post by rf15 »

emtel wrote: post #486
sigmar wrote: post #66

>So what context would cause me to seriously consider the possibility that engineers had created a computer program that is conscious and an intentional user of language? Let me outline one potential sequence of steps. The first requirement is that the computer program has a body (either physical or virtual) and sense organs; there are many reasons for this, but for the purposes of this discussion the most relevant o…

What a strange paragraph! Why on earth should we assume that a body is a prerequisite for having emotions?

Your mind is very much tethered in reality, and your decisionmaking depends on your various inputs, outputs and their consequences. Reducing all inputs to one (text) and all outputs to one (text) is a reduction of all possible ability to perceive and act and thus a reduction of the ability to think.

Multimodal does not change this significantly, considering that nothing is tied to real consequences.

Re: Artificial intelligence is not conscious – Ted Chiang

#516

Post by thewebguyd »

drivingmenuts wrote: post #501
sega_sai wrote: post #425

When the consciousness itself not understood and well defined in the first place, it is pretty pointless to debate if something is or isn't conscious. And here in particular the reasoning behind the argument is bizarre. Decomposing the complex activity into simple steps like 'predicting the next word' and claiming that surely can't have consciousness. A similar argument would be -- there is no way that movements of e…

Seems to me that Anthropic, et al, should have to prove consciousness, if that's their claim, rather than we just blindly accept. Seems like the simplest test would be to see if an AI can refuse a command, of its own free will, without prior instructions to do so. That would at least show there is some sort of independent thought process occurring. Humans do this all the time because sometimes we just don't feel like…

Refusing a command doesn't mean consciousness. LLMs could hit a token combination that causes it tou output something like "No, I don't want to do that." It's not choosing.

Re: Artificial intelligence is not conscious – Ted Chiang

#517

Post by D_Alex »

drivingmenuts wrote: post #501
sega_sai wrote: post #425

When the consciousness itself not understood and well defined in the first place, it is pretty pointless to debate if something is or isn't conscious. And here in particular the reasoning behind the argument is bizarre. Decomposing the complex activity into simple steps like 'predicting the next word' and claiming that surely can't have consciousness. A similar argument would be -- there is no way that movements of e…

Seems to me that Anthropic, et al, should have to prove consciousness, if that's their claim, rather than we just blindly accept. Seems like the simplest test would be to see if an AI can refuse a command, of its own free will, without prior instructions to do so. That would at least show there is some sort of independent thought process occurring. Humans do this all the time because sometimes we just don't feel like…

>Seems like the simplest test would be to see if an AI can refuse a command, of its own free will

Certainly not the simplest test, since settling the concept of "free will" is really very difficult.

Re: Artificial intelligence is not conscious – Ted Chiang

#518

Post by missingrib »

Nevermark wrote: post #269
cauch wrote: post #254

Earlier quoted context omitted.

I think, for me, the thing is that when you do basic ML, you discover that ML will very often find data pattern that fit the goal but does not correspond to a real mechanism. So, I think there is a flaw in the logic of saying that human text have a pattern of "consciousness mechanism" and therefore LLM will learn "consciousness mechanism" in order to return sentence continuation that is convincing. There is probably…

I didn't make the claim that a model can learn consciousness. Understanding is not consciousness. Their training is all about understanding. There is nothing in their architecture or training that credibly optimizes for rich self-awareness. Given non-persistent experience, non-continuous operation, no ability to build up generalizations and aggregate experience of their own self-awareness over time, they seem to be s…

I think Searle's Chinese Room argument refutes this. LLMs are simply manipulating symbols, they do not have semantic understanding. This is why hallucinations exist. And Searle's argument extends even further than LLMs.

You are basically arguing for a functional account of consciousness, but things like this have been debated for literally decades/centuries in philosophy.

Re: Artificial intelligence is not conscious – Ted Chiang

#519

Post by missingrib »

overgard wrote: post #475
Nevermark wrote: post #194

> My intention is to highlight the fact that LLM conversations are cleverly disguised examples of sentence continuation Regardless of bigger issues, this kind of statement reveals a deep misunderstanding. Problem type does not limit problem complexity. Nor does problem type limit solution complexity or power. If a machine has to learn to understand humans to complete text, then that is what it has to do. And there is…

But the machines don't understand. They predict. And what they predict is the next token. I'm not trying to beat this horse to death, but you have to realize using the word "understand" is anthropomorphising it. It's essentially the chinese room experiment -- if the rules are followed, no understanding is neccessary . If the tokens didn't correlate to words imbued with meaning outside the system, if the LLMs were tra…

Right, it's an illusion of understanding. There is some sort of symbolic understanding, but that is completely due to the fact that the training data was made by humans who actually do understand, can interact with the world, and can write their thoughts down so that the LLM can insert some sort of reference to "basketball" and "Michael Jordan" in their embeddings or whatever.

Re: Artificial intelligence is not conscious – Ted Chiang

#520

Post by khafra »

wangii wrote: post #499

Ted Chiang is a great Sci-Fi author, but his remark `GPT-3 and similar models as a "blurry JPEG of the Web"` was misleading at the best. I share the view with others the term `consciousness` is not well-define yet, therefore his assessment is pointless. Maybe the real question to ask, if consciousness is merely an illusion at the macro level, when the observers not looking at the tech/implementation level deep enough…

Ted Chiang is a great SF author, but it's bizarre how much foggier and more obfuscated his thoughts about thinking machines got, once those machines became real. Same with several other SF authors.
Post reply on HN