My intention is to highlight the fact that human conversations are cleverly disguised examples of natural selections
Artificial intelligence is not conscious – Ted Chiang
921–930 of 1001 posts
Re: Artificial intelligence is not conscious – Ted Chiang
#922Natural intelligence is also not conscious sometimes. Not fully conscious during sleep, completely knocked out by anaesthetics during surgery and such.
Suppose your brain were somehow exploited for generating text, while you are under anesthesia. That would be definitely unconscious.
Though it's possible that parts of your brain are conscious, but have no access to express that because they don't control the body, nor directly receive any sensory information. I.e. that "you" that is unconscious when anaesthesized might not be the only consciousness in your brain, just the visible one.
Re: Artificial intelligence is not conscious – Ted Chiang
#923This entire essay basically boils down to one simple contention: LLMs can’t be conscious because only humans can be conscious and LLMs are not human. What an unimaginative argument: essentially ruling out the possibility of the title by definition. And the author fails to touch on the really interesting question posed by the article’s title: what if human consciousness is more like the working of LLMs than we think?
What? Where are you seeing such claims? I.e. that only humans can be conscious, that AI in general is not conscious by definition?? Seems to me you've put up an easy-to-dismiss strawman.
Here's how I would summarize crux of the argument: LLMs (specifically) are, by construction, chameleons. More precisely, role-playing machines. When compelled to have a conversation as "themselves", all they do is take on the role of "themselves", as inferred from the training data plus the text of the conversation so far, just like they do for any other role. This is fundamentally different than we humans. When we have a (normal, non-roleplaying) conversation, we don't put aside whatever role we might have been acting out before and instead take on the role of "ourselves" and act from this new perspective. Acting as ourselves is a completely different way of interacting than any play-acting we might do for example as an actor in a play, or when trying to predict the future behaviour of a third-party. Even in terms of energy expenditure, acting as ourselves is much less strenuous than trying to simulate someone else's perspective.
IOW, we have an inherent perspective, an I like an inner eye which looks at all things with a certain slant or tendency. LLMs don't. There's no I inside them, instead they can take, and in a sense are, all Is possible.
Re: Artificial intelligence is not conscious – Ted Chiang
#924When the consciousness itself not understood and well defined in the first place, it is pretty pointless to debate if something is or isn't conscious. And here in particular the reasoning behind the argument is bizarre. Decomposing the complex activity into simple steps like 'predicting the next word' and claiming that surely can't have consciousness. A similar argument would be -- there is no way that movements of e…
From my pov and after using AI a lot to better understand its usefulness I see this discussion as totally misleading and irrelevant. The elephant in the room is that even the best frontier models still produce errors. This is a fundamental issue and should be discussed instead of some esoteric pseudo-scientific nonsense. If the current systems can not produce reliable results they are useless. The promise of actual u…
Humans also produce errors but are still extremely useful
Re: Artificial intelligence is not conscious – Ted Chiang
#925Earlier quoted context omitted.
I actually don't think neuroscience can solve it either. Science could decide whether, e.g. equivalent physical processes are occurring in dogs and humans, and you could argue that solves the question for things with physical brains, but even then it can't measure whether another brain has the same subjective experience as you do. It can't really do that for other humans. It certainly can't answer that question for h…
> I actually don't think neuroscience can solve it either. Science could decide whether, e.g. equivalent physical processes are occurring in dogs and humans, and you could argue that solves the question for things with physical brains, but even then it can't measure whether another brain has the same subjective experience as you do. It can't really do that for other humans Imagine a device that you put on your head a…
It wouldn't solve the hard problem. I still couldn't verify that the recording gives you the exact subjective experience it gives me. Yes, I'm leaning into the unfalsifiability but that's kind of the point of why it's Hard. We have this nugget of unfalsifiability at the core of our experience.
Anyway, if someone claimed to create such a machine I would, in fact, very much doubt that it actually creates the same experience simply because no human brain is quite physically the same, so it will interact differently with the machine. That's true even if experience is entirely physical.
> The only realistic way we either build that device or prove that it can't be built is science.
There's another possibility: the device remains in the realm of hazy infeasibility forever, where no one succeeds convincingly but we also never articulate why it's impossible. I think this is more likely. Certainly the engineering would be extremely difficult.
Any candidate device will face the usual objections about the relationship between experience and its physical correlates, plus the one I mentioned above about physical differences between brains, and probably a dozen more depending on the details. You'll be able to choose to believe it proves consciousness is physical, or not, but you can already choose that today with equally strong evidence. It's like a binary Rorschach test for your assumptions about metaphysics.
Re: Artificial intelligence is not conscious – Ted Chiang
#926My background is in cognitive science and psycholinguistics. I spent more than ten years talking to first year psychology undergraduates about whether AIs could be conscious; also did some research on (extremely tiny) AIs in modelling language behaviours. There is a great deal of good thinking on Chiang's topic by professional philosophers, and there's much to be said for reading them. I won't rehearse their ideas he…
> I won't rehearse their ideas here Is there an accessible way for a layman such as myself to read about some of these ideas (Really I mean philosophical discussion in general) without having to read entire books? Is there an active HN-equivalent or wiki or something?
You should read entire books
Re: Artificial intelligence is not conscious – Ted Chiang
#927When the consciousness itself not understood and well defined in the first place, it is pretty pointless to debate if something is or isn't conscious. And here in particular the reasoning behind the argument is bizarre. Decomposing the complex activity into simple steps like 'predicting the next word' and claiming that surely can't have consciousness. A similar argument would be -- there is no way that movements of e…
I'm a bit late to the party, but I notice that people are missing the core argument of this article completely. That core, summarized below, seems very hard to dismiss to me. This argument has been presented before, but there seems to be something about it that makes it very hard for people to grasp.
Here's how I would summarize crux of the argument: LLMs (specifically) are, by construction, chameleons. More precisely, role-playing machines. When compelled to have a conversation as "themselves", all they do is take on the role of "themselves", as inferred from the training data plus the text of the conversation so far, just like they do for any other role. This is fundamentally different than we humans. When we have a (normal, non-roleplaying) conversation, we don't put aside whatever role we might have been acting out before and instead take on the role of "ourselves" and act from this new perspective. Acting as ourselves is a completely different way of interacting than any play-acting we might do for example as an actor in a play, or when trying to predict the future behaviour of a third-party. Even in terms of energy expenditure, acting as ourselves is much less strenuous than trying to simulate someone else's perspective.
IOW, we have an inherent perspective, an I like an inner eye which looks at all things with a certain slant or tendency. LLMs don't. There's no I inside them, instead they can take, and in a sense are, all Is possible.
Re: Artificial intelligence is not conscious – Ted Chiang
#928Earlier quoted context omitted.
But the machines don't understand. They predict. And what they predict is the next token. I'm not trying to beat this horse to death, but you have to realize using the word "understand" is anthropomorphising it. It's essentially the chinese room experiment -- if the rules are followed, no understanding is neccessary . If the tokens didn't correlate to words imbued with meaning outside the system, if the LLMs were tra…
However it’s disingenuous to say the inference is on the next token because it’s actually not, it’s in the models parameter space across a set of nonlinear activation functions then effectively projected into the token. The idea its predictive of the token isn’t actually the case, it really is a much more complex and more semantic relationship that ends in the series of tokens through the attention mechanism. The art…
> But if you can not see it as amazing you have lost the capacity to dream.
I completely disagree. I think if you think these things are amazing, your dreams are incredibly limited and boring.
I remember the first time I talked to a chatbot. Not an LLM, just a regular chatbot, like ELIZA or any other dumb bot.
For a few seconds, it felt magical, like I was talking to a computer that understood me, as it made replies that were sensible to what I was saying. Then it said something incredibly stupid and jarring that made no sense, and that took the magic away. Oh, this is just a dumb computer program.
I remember the first time I talked to an LLM-powered chatbot. It was the exact same thing, except the magic feeling lasted a tiny little bit longer and was a tiny little more convincing. But it went away in the exact same way, for the exact same reason. Once you've seen the emperor without clothes, nothing brings back the magic.
Re: Artificial intelligence is not conscious – Ted Chiang
#929This entire essay basically boils down to one simple contention: LLMs can’t be conscious because only humans can be conscious and LLMs are not human. What an unimaginative argument: essentially ruling out the possibility of the title by definition. And the author fails to touch on the really interesting question posed by the article’s title: what if human consciousness is more like the working of LLMs than we think?
The argument doesn't state that at all. It very clearly admits consciousness to nonhuman animals, for example.
Re: Artificial intelligence is not conscious – Ted Chiang
#930My background is in cognitive science and psycholinguistics. I spent more than ten years talking to first year psychology undergraduates about whether AIs could be conscious; also did some research on (extremely tiny) AIs in modelling language behaviours. There is a great deal of good thinking on Chiang's topic by professional philosophers, and there's much to be said for reading them. I won't rehearse their ideas he…
My background is in computer science and linguistics. I wholeheartedly agree with you: We humans are dubiously-equipped to determine whether or not AI could be conscious. I'm also super curious to learn more about the philosophers you referenced and their thoughts on this subject. Would you be willing to share some of your favorite examples?
I could spend a very long time double-checking my sources, but I want to reply tonight, so please take the following with a pinch of salt, since it's largely from memory:
0. Descartes/Titchener/Chomsky and friends for background.
1. John Searle featured prominently because of his accessibility. I tended to present his Chinese Room argument as a criticism of symbolic AI alone, though I believe he considered it a criticism of sub-symbolic approaches as well.
2. Thomas Nagel's classic article "What Is It Like to Be a Bat?" is a good introduction to qualia, which is how we describe direct conscious experience.
3. Wittgenstein would be important in terms of the impossibility of empathising with, or seeing things from the perspective of, other minds that have emerged in different contexts (such as animals). However, I rarely spoke about him because, frankly, I couldn't understand him in the original!
4. David Chalmers. Writes clearly (and coined?) the 'hard problem' of consciousness and why subjective experience appears difficult to reconcile with a purely physical account of the mind.
4.1 Daniel Dennett. The clearest and most influential critic of the idea that consciousness presents a special explanatory problem.
5. Darwin and others. Comparative psychology (the study of animal minds) strongly suggests, by showing that the antecedents of the human mind are present in animals, that we should reduce our bias that human minds are special, ineffable, or somehow atomic.
6. Jerry Fodor. Modularity: the idea that the mind is composed of modules that specialise in certain tasks (e.g. phoneme perception, syntactic analysis, face recognition) and operate largely unconsciously unless they are dependent on one another in some way. This helps us take a computational approach to the mind. It reminds us that much of what we do mentally might be expressible in computational terms.
Two final ideas off the top of my head:
1. The Ship of Theseus: https://en.wikipedia.org/wiki/Ship_of_Theseus
When (not all) neurons die off and are replaced by new ones doing the same job, our sense of self and identity is somewhat cast into doubt. In a physical system, if consciousness is a magic or non-material entity of some kind, what is happening to it during this process?