[deleted]
Artificial intelligence is not conscious – Ted Chiang
781–790 of 1001 posts
Re: Artificial intelligence is not conscious – Ted Chiang
#782When the consciousness itself not understood and well defined in the first place, it is pretty pointless to debate if something is or isn't conscious. And here in particular the reasoning behind the argument is bizarre. Decomposing the complex activity into simple steps like 'predicting the next word' and claiming that surely can't have consciousness. A similar argument would be -- there is no way that movements of e…
Any argument that doesn't begin with making clear one's position about belief in the supernatural must be dismissed. "AI is just math", so is physics and the human body which is a physical system of molecules and electrical signals. The argument that ai is "merely" math by itself is only a valid way to dismiss it being conscious only if one also clarifies belief in the supernatural. Otherwise so are humans. Humans ar…
Humans are AI. The Universe is our prompter. A world within worlds! Jeeesus WEPT!
Re: Artificial intelligence is not conscious – Ted Chiang
#783Earlier quoted context omitted.
They are conscious because even for short periods of time they do form memories and those change them even if only briefly. They think on their own too. It is a very limited level of consciousness though.
Okay but this state is formed in text. Text isn't conscious
not really, it’s ultimately formed in electromagnetic fields.
Re: Artificial intelligence is not conscious – Ted Chiang
#784[deleted]
Re: Artificial intelligence is not conscious – Ted Chiang
#785Earlier quoted context omitted.
everyone has emotion even if it's muted, and we have reactions and desires as a result of having bodies and remembered experiences using them... these are constantly integrated into our working memories AI doesn't really have any of that yet, but we're maybe not so far off
Is it really so easy to assert that an AI doesn't have emotion? Lots of the AI models are capable of getting pissed off at the users, especially the earlier ones. And sure, their emotion is merely a bias that the context window generates in their weights... but how is that different from humans? In humans, emotion is sourced from a bunch of chemical signalling, and those chemicals bias your word choice and action cho…
because AI hasn't lived any experiences and it doesn't incorporate experience into its corpus
it's more like someone writing about a character getting angry in a book... I don't think anyone would argue that a character actually experienced anger, right? there's no subjectivity in that experience... it's the output of someone else's experience on what they'd expect the character's reaction to be, not a genuine outcome of that character's experience
That's the same here, LLMs are outputting the result of the written experiences of others... not its own experience, which it does and can not have.
Put another way, it only "knows" what it just output by reading it... it doesn't actually build experience to know anything.
We might be closer than ever to building these things, but they're not here yet.
Re: Artificial intelligence is not conscious – Ted Chiang
#786Earlier quoted context omitted.
Reinforcement learning changes the model. So it can and does change and remember based on experience. Eventually reinforcement learning can happen in real time.
But is the model aware of the training? Unless you hook the model up to an MCP server, or something similar, and have it analyze the RL changes, it will not know if it has changed or not. Even if it is real-time RL, it is not aware of the previous state.
Are you aware of each of your dreams from last week or last night even?
Re: Artificial intelligence is not conscious – Ted Chiang
#787When the consciousness itself not understood and well defined in the first place, it is pretty pointless to debate if something is or isn't conscious. And here in particular the reasoning behind the argument is bizarre. Decomposing the complex activity into simple steps like 'predicting the next word' and claiming that surely can't have consciousness. A similar argument would be -- there is no way that movements of e…
> When the consciousness itself not understood and well defined in the first place, it is pretty pointless to debate if something is or isn't conscious. I think that by debating what consciousness is not is one of the best ways we can gain a deeper understanding of consciousness itself. The weird thing is, you and Chiang have different arguments but you're using the same logic: > Before anyone can credibly claim that…
> IOW if we don't know what consciousness is, how can we call LLMs conscious when we haven't seen even the barest intermediary steps towards our nascent concept of consciousness first?
We stipulate that other human beings are conscious from their behavior and how it relates to ours when it is accompanied by our personal introspective experience or "awareness" of being conscious during normal cognition. The process for ascribing consciousness to LLMs would be same: A stipulation on the basis of behavior that relates to our own behavior and how it appears to be linked to the introspective experience of being conscious.
Re: Artificial intelligence is not conscious – Ted Chiang
#788> My intention is to highlight the fact that LLM conversations are cleverly disguised examples of sentence continuation Regardless of bigger issues, this kind of statement reveals a deep misunderstanding. Problem type does not limit problem complexity. Nor does problem type limit solution complexity or power. If a machine has to learn to understand humans to complete text, then that is what it has to do. And there is…
But the machines don't understand. They predict. And what they predict is the next token. I'm not trying to beat this horse to death, but you have to realize using the word "understand" is anthropomorphising it. It's essentially the chinese room experiment -- if the rules are followed, no understanding is neccessary . If the tokens didn't correlate to words imbued with meaning outside the system, if the LLMs were tra…
The understanding is inside of the system, in LLMs and in the Chinese Room. I agree with Daniel Dennett that it's preposterous to say that Chinese is not understood in any meaningful sense in the Chinese Room scenario -- it's just that the understanding has been hidden away in the background of the scenario.
Language is tremendously complicated. "Time flies like an arrow, but fruit flies like a banana." "Hard hats must be worn on site; dogs must be carried on escalators", etc. Predicting the next token requires understanding, full stop.
> if the rules are followed, no understanding is neccessary.
The rules are the understanding.
(Note that understanding != consciousness)
Re: Artificial intelligence is not conscious – Ted Chiang
#789Remind me in 5 years
Re: Artificial intelligence is not conscious – Ted Chiang
#790When the consciousness itself not understood and well defined in the first place, it is pretty pointless to debate if something is or isn't conscious. And here in particular the reasoning behind the argument is bizarre. Decomposing the complex activity into simple steps like 'predicting the next word' and claiming that surely can't have consciousness. A similar argument would be -- there is no way that movements of e…
> When the consciousness itself not understood and well defined in the first place, it is pretty pointless to debate if something is or isn't conscious.
But we know what consciousness is, even if you don't know how to explain it in materialistic[0] terms. Intentionality is what makes consciousness consciousness. We know that LLMs - and computers in general - do not possess intentionality, not even sensory intentionality. There is no "aboutness" and no semantics to the content within a machine, never mind that the processes within LLMs are not reasoning or inferential processes. It is a sophisticated behavioral simulation that may be syntactically defined, nothing more. It is purely a matter of transeunt causes and not the immanent causality needed for something like consciousness.
(There's also the more general problem that we cannot say that physical computing machines are objectively computing. There is nothing about the physical processes - which completely define what a computing device is doing - that could be identified with computation in any objective sense. Rather, we human observers assign a computational interpretation to the machine's operations, just as we assign them to the ink blobs in a book or the liquid crystals you are reading on your screen now. Computation is something we do, and we have built machines to simulate it. We formalized computation - which is a process of desemantification and syntactic codification - and build machines guided by these formalisms. Formalization is what makes mechanization possible.
> Decomposing the complex activity into simple steps like 'predicting the next word' and claiming that surely can't have consciousness. A similar argument would be -- there is no way that movements of electrons by tiny distance would produce consciousness.
The movement of electrons as such cannot produce consciousness. The reductionist paints himself into a corner by strapping reality into a Procrustean mechanistic and materialistic metaphysics that redefines reality according to some dedicated schema, and then strains endlessly to find how some phenomenon can arise or emerge from the paltry remains. (Either that, or he embraces insanity completely and becomes an eliminativist.) It is magical thinking. I suspect this fallacious line of thinking is what causes people to believe piling more syntactic operations will somehow magically fuse into semantics and intentionality.
[0] "Materialism" here is the metaphysical theory. I am not claiming that consciousness - at least not all consciousness - cannot be a physical phenomenon. I am only claiming that the materialistic view of matter cannot account for consciousness.