It could be conscious, but none of the current slop models have anything like that. Consciousness requires continuous learning and introspection. A fly isn’t conscious because it follows a hard-coded script.
Artificial intelligence is not conscious – Ted Chiang
901–910 of 1001 posts
Re: Artificial intelligence is not conscious – Ted Chiang
#902I often hear the assumption that LLMs can or will become conscious because consciousness is likely substrate independent . The idea being that our brain is just a computer made out of meat and it doesn't do anything that can't be precisely simulated by a silicon computer. But I wonder if some of the magic in the human brain is its analog nature. Chemical signaling and impulses of neurons interact with each other with…
I think the representation in a computer, the fact that it is merely stored instructions and data, destroys everything but domains of simulation and emulation.
"I think the representation in a brain, the fact that it is merely stored activation potentials in neurons firing based on ionic accumulations at their synapses, destroys everything but domains of simulation and emulation."
;)
Re: Artificial intelligence is not conscious – Ted Chiang
#903The fact that a LLM is essentially immutable would be my biggest argument against consciousness or self-awareness. It's a big file with a bunch of coordinates describing spatial relationships between tokens. When you give it a prompt, it uses those relationships to generate a string of tokens that is a statistically likely response to that prompt, then it stops. It's not changed by the experience. It doesn't remember…
There are people whose brains don’t form new memories anymore after an accident or surgery, and they eternally live in the time before it happened, and have no memory of what happened a minute ago. Still they are conscious.
Someone getting in an accident that chops their leg off doesn’t mean humans don’t have legs. Come on man.
Re: Artificial intelligence is not conscious – Ted Chiang
#904Earlier quoted context omitted.
Do you believe consciousness to be an emergent property of the laws of physics?
Regardless, the question still stands: "What does computers getting more intelligent has to do with it getting conscious?" Just because consciousness emerged for we humans and other animals through one mechanism doesn't mean consciousness has/will/can emerge from current LLM technology. For this extraordinary claim, I think the burden is firmly on those who are arguing that it has/will/can.
how is anyone supposed to decide if something is conscious if no one has the slightest idea what consciousness really is
Re: Artificial intelligence is not conscious – Ted Chiang
#905How does any article or discussion about this get past 150 words.
"There is no agreed upon functional definition of consciousness, and no way to measure or observe it outside of oneself. It is as impossible to detect consciousness in waves or energy or inert matter or stochastic systems as it is in other living human beings."
Anything beyond that is akin to a theological treatise about the possibility of ensouled animals.
Re: Artificial intelligence is not conscious – Ted Chiang
#906This entire essay basically boils down to one simple contention: LLMs can’t be conscious because only humans can be conscious and LLMs are not human.
What an unimaginative argument: essentially ruling out the possibility of the title by definition. And the author fails to touch on the really interesting question posed by the article’s title: what if human consciousness is more like the working of LLMs than we think?
Re: Artificial intelligence is not conscious – Ted Chiang
#907I often hear the assumption that LLMs can or will become conscious because consciousness is likely substrate independent . The idea being that our brain is just a computer made out of meat and it doesn't do anything that can't be precisely simulated by a silicon computer. But I wonder if some of the magic in the human brain is its analog nature. Chemical signaling and impulses of neurons interact with each other with…
Probably not, we're nowhere near the complexity of the human brain yet, there are also quantization limits to the human brain (i.e. molecules, quantum physics, etc) so to characterize them as having infinite detail is probably a bad modal. If I'm going to be honest most of the people who advocate this type of thing tend to be, shall we say, crypto-duelists who really believe in a soul but not like intellectually but…
How are you measuring the complexity of the human brain?
Re: Artificial intelligence is not conscious – Ted Chiang
#908Earlier quoted context omitted.
> The model definitely remembers previous exchanges within the same conversation. No it doesn't. They get added to its context, and it reads them afresh when answering the next question. That's not remembering. If your short-term memory completely malfunctioned one day, so you had no ability to remember what was said to you a minute ago, then you would have to find workarounds. For example, you could write down every…
Continuous learning allows past behavior and past inputs to influence future inputs and future behavior. In humans. Attention over KV cache allows past behavior and past inputs to influence future inputs and future behavior. In LLMs. Until the cache runs out, that is. But even then, you could totally use any of 9000 methods of cache compression, truncation, dropping or streaming and get away with it. The difference b…
Maybe, every night, you send the AI off to "sleep" where it uses those in cache "memories" to influence the long term weights [1].
Re: Artificial intelligence is not conscious – Ted Chiang
#909Earlier quoted context omitted.
> It's not changed by the experience The entire file is not changed, but the KV cache is. > It doesn't remember anything The model definitely remembers previous exchanges within the same conversation.
Not the model though. The model really only takes input text and produces output text. Memory within a conversation is achieved by the harness adding the conversation (or parts of it) to the input text. The LLM itself has no memory, it’s the augmented system of several orchestrated LLM calls that does.
Wait until you hear about the hippocampus!!! [1]
I don't think physical integration within one contained is relevant to system level behavior.
Re: Artificial intelligence is not conscious – Ted Chiang
#910I often hear the assumption that LLMs can or will become conscious because consciousness is likely substrate independent . The idea being that our brain is just a computer made out of meat and it doesn't do anything that can't be precisely simulated by a silicon computer. But I wonder if some of the magic in the human brain is its analog nature. Chemical signaling and impulses of neurons interact with each other with…
It is perhaps very relevant what Chris Olah (Anthropic co-founder) said to pope.. I am a scientist. I lead a research team that studies the internal structure of these models—what is actually happening inside them. And I will be honest: we keep finding things that are mysterious, even unsettling. We find structures that mirror results from human neuroscience. We find evidence of introspection. We find internal states…
I mean, this reminds me of the early imagenet days, when people were first trying to explain the unreasonable efficiency of neural networks at interpreting content in images. They found out that, after enough backprop, some layers of the network became specialized in contours extraction, shape extraction, etc, in ways that can be seen as analogous to earlier CV techniques (Canny/Hough transforms, ...).
Later, Google was having fun feeding whole bunch of youtube content to artificial neural networks, unsupervised, and figured that certain parts of the network would, too, specialize, only to have the activation functions be run backwards and render an abstract image of a cat¹.
None of that is terribly new or surprising for anyone having studied and dealt with neural networks. The only difference today is that the field has completely flip flopped from approaching the subject with scientific rigor and cautious excitement to being a clueless billionaire infinite money printing machine fed on deceiving anthropomorphism and FUD.
¹: https://blog.google/innovation-and-ai/products/using-large-s...