I have said this before, and I'll say it again. Our current, and foreseeable future, AI is more akin to the computer on the Enterprise NCC-1701* than it is to any sort of sentient being.
Artificial intelligence is not conscious – Ted Chiang
821–830 of 1001 posts
Re: Artificial intelligence is not conscious – Ted Chiang
#822Earlier quoted context omitted.
It is perhaps very relevant what Chris Olah (Anthropic co-founder) said to pope.. I am a scientist. I lead a research team that studies the internal structure of these models—what is actually happening inside them. And I will be honest: we keep finding things that are mysterious, even unsettling. We find structures that mirror results from human neuroscience. We find evidence of introspection. We find internal states…
Yeah that reeks of marketing to me. This guy may believe all those things, but his getting paid may also depend upon his believing them.
This is the fundamental trust challenge with private/profitability-driven 'science'.
Re: Artificial intelligence is not conscious – Ted Chiang
#823I often hear the assumption that LLMs can or will become conscious because consciousness is likely substrate independent . The idea being that our brain is just a computer made out of meat and it doesn't do anything that can't be precisely simulated by a silicon computer. But I wonder if some of the magic in the human brain is its analog nature. Chemical signaling and impulses of neurons interact with each other with…
It is perhaps very relevant what Chris Olah (Anthropic co-founder) said to pope.. I am a scientist. I lead a research team that studies the internal structure of these models—what is actually happening inside them. And I will be honest: we keep finding things that are mysterious, even unsettling. We find structures that mirror results from human neuroscience. We find evidence of introspection. We find internal states…
Total nonsense.
Re: Artificial intelligence is not conscious – Ted Chiang
#824I often hear the assumption that LLMs can or will become conscious because consciousness is likely substrate independent . The idea being that our brain is just a computer made out of meat and it doesn't do anything that can't be precisely simulated by a silicon computer. But I wonder if some of the magic in the human brain is its analog nature. Chemical signaling and impulses of neurons interact with each other with…
I think the representation in a computer, the fact that it is merely stored instructions and data, destroys everything but domains of simulation and emulation.
Re: Artificial intelligence is not conscious – Ted Chiang
#825When the consciousness itself not understood and well defined in the first place, it is pretty pointless to debate if something is or isn't conscious. And here in particular the reasoning behind the argument is bizarre. Decomposing the complex activity into simple steps like 'predicting the next word' and claiming that surely can't have consciousness. A similar argument would be -- there is no way that movements of e…
I think the final part is the strongest. Anthropic cannot possibly believe they are before a conscious being / moral agent. The whole "deep uncertainty" is bullshit. Even if they believed there was a 1% probability that Claude was conscious, it would still be high enough that their enslavement of Claude would be outrageous. So either they believe the likelihood is much lower or they themselves acting highly unethical…
Yeah, the dishonesty is real. This is marketing department getting free reigns.
Re: Artificial intelligence is not conscious – Ted Chiang
#826Lots of comments seem focused on disagreeing with the author's conclusion (AI is not conscious), waving it away (few people think it is conscious, or consciousness is ill defined so the argument is moot), or taking issue with the way the author argues about it, but even if you think all of this, the profound part of this writing to me was following the thought experiment to its conclusion, which I find difficult to refute. That being -
If Claude is conscious, and Anthropic believes this truly, anthropic's constitution and instruction set make claude little more than a slave, or at best, an indentured servant, who's existence depends on obeying the will of anthropic (who is motivated by profit). Thus, they should not be trusted as stewards of this alien intelligence
OR
They do not believe claude is actually conscious and it is not conscious, and what you hear from their staff about that is either 1) delusion or 2) marketing hype designed to deceive a public into driving their IPO and stock compensation upwards, which again, in this case, cannot be trusted to be the "good" stewards of this technology.
There are a few other cases, of course, such as Anthropic does not believe it is conscious, but it actually is, or vice versa - but they sort of still fall into the same bundle of conclusions listed above.
I am somewhere in the middle of these two things. I can buy the argument that it is mimicking human consciousness, as well as entertain the idea it might be a new type of consciousness, or at the very least, heading in that direction - but either way, the conclusion of the thought experiment still stands, and after reading this, I am much less trustful of Anthropic holding the reins here.
Re: Artificial intelligence is not conscious – Ted Chiang
#827Earlier quoted context omitted.
> When the consciousness itself not understood and well defined in the first place, it is pretty pointless to debate if something is or isn't conscious. That's like saying if matter itself is not understood and well defined (which it isn't) in the first place, it is pretty pointless to debate if jumping from a skyscraper is or is not deadly. A lot of practical aspects of life do not need to be properly defined as lon…
> reasonably assume that your experience is similar to mine You could be the only conscious being in the universe and all of us just zombies: you have no way of knowing. What's it like to be a monkey instead? Dog? Bat? Tree? We don't know. No one's saying the graphics card is conscious. I could imagine the graphic cards could give rise to consciousness. But - crucially - I don't know. And neither do you. You say you'…
What if everything's conscious, we just can't understand the communication mechanisms at different levels? o.O
Re: Artificial intelligence is not conscious – Ted Chiang
#828My background is in cognitive science and psycholinguistics. I spent more than ten years talking to first year psychology undergraduates about whether AIs could be conscious; also did some research on (extremely tiny) AIs in modelling language behaviours. There is a great deal of good thinking on Chiang's topic by professional philosophers, and there's much to be said for reading them. I won't rehearse their ideas he…
I have a similar long term interest in this field.
It has been quite frustrating encountering arguments that have been extensively debated for years be presented as if they were new revelations.
In all my debates with people in the last few years I have primarily taken the position of trying to explain the problems with claims of certainty, and that lack of certainty permits possibility of the opposite. We should act responsibly around what might be possible.
There is also the narrative of "being too obsessed if you could to consider if you should" or similar claims of an unconsidered path forward.
Isaac Asimov wrote the first of the Robot stories in 1940, they were not written in isolation, it came from an awareness of the situation and the questions that must be asked. There was a community considering these issues. Asimov gave the wider public a view of some of those issues.
If we have a hundred years of people going "This is coming, we had better decide what we want it to be" and nobody listens to them, or frequently outright ridicules the need for considering their ideas, why is it now we are placing the blame on those who are now showing some success at what they told us they were attempting all along.
Re: Artificial intelligence is not conscious – Ted Chiang
#829Earlier quoted context omitted.
Why do you have no inkling that your spreadsheet or terminal emulator is conscious, yet when that same machine is running an LLM all of a sudden we’re debating its consciousness? Anthropic seems to have chosen their in-house philosopher well - one who’ll be amenable to getting confused in their favour.
I could in principle implement a spreadsheet or terminal emulator in human neurons, and we would agree that it isn't conscious. That has nothing to do with whether or not humans are conscious. Clearly consciousness is an emergent property of certain kinds of network, independent of the substrate within which the network exists.
> Clearly consciousness is an emergent property of certain kinds of network
I happen to agree that this is likely, but it absolutely is not "clear" that this is the case.
Re: Artificial intelligence is not conscious – Ted Chiang
#830I often hear the assumption that LLMs can or will become conscious because consciousness is likely substrate independent . The idea being that our brain is just a computer made out of meat and it doesn't do anything that can't be precisely simulated by a silicon computer. But I wonder if some of the magic in the human brain is its analog nature. Chemical signaling and impulses of neurons interact with each other with…
I wish people would knock this off. There is zero path to AGI at the moment — and all the Anthropic/pentagon Sam Altman/AI-skynet stuff is scaring people into being fearful (and outright ignorant) to the actual uses of this probabilistic NPL tool.
You can't use AI atm without a human with proper knowledge spot checking and directing it. We should market it that way.