I love how the point being made is a tautology :
AI is not conscious because it is not conscious.
Artificial intelligence is not conscious – Ted Chiang
671–680 of 1001 posts
Re: Artificial intelligence is not conscious – Ted Chiang
#672Earlier quoted context omitted.
By "they" do you mean LLMs, or people, or you in particular? Because you're unoriginally parroting a banal thought stopping sequence of tokens that many people have said before without understanding what it means, so it's impossible to say exactly how much conciousness you actually have.
I can't actually figure out what you're reacting to - perhaps you could elaborate?
Try pasting it into an LLM and asking it to explain it to you.
If an LLM can explain what you can't understand, after you literally repeat unoriginal drive-by slogans you heard other people say like "stochastic parrot", without understanding how banal and reductionist and inaccurate that is, or addressing how LLMs actually work and what they can do, then what does that say about you as a human?
By your own reductionist argument, it says you don't actually understand anything, it's just neurons firing.
Are you claiming that parrots can write and debug code, explain mathematics, translate languages, summarize research papers, and engage in extended technical discussions?
Stanislaw Lem's Solaris, His Master's Voice, Golem XIV, The Cyberiad, and his other works describe how humans have a pathological tendency to mistake their own conceptual categories for universal truths. You have identified a mechanism and mistaken it for an explanation.
The term "stochastic parrot" is a slogan masquerading as an explanation, only a shallow surface description of the mechanism, that totally fails to explain the phenomenon, or account for all that LLMs and language itself can do.
Re: Artificial intelligence is not conscious – Ted Chiang
#673When the consciousness itself not understood and well defined in the first place, it is pretty pointless to debate if something is or isn't conscious. And here in particular the reasoning behind the argument is bizarre. Decomposing the complex activity into simple steps like 'predicting the next word' and claiming that surely can't have consciousness. A similar argument would be -- there is no way that movements of e…
Why do you have no inkling that your spreadsheet or terminal emulator is conscious, yet when that same machine is running an LLM all of a sudden we’re debating its consciousness? Anthropic seems to have chosen their in-house philosopher well - one who’ll be amenable to getting confused in their favour.
I could in principle implement a spreadsheet or terminal emulator in human neurons, and we would agree that it isn't conscious. That has nothing to do with whether or not humans are conscious.
Clearly consciousness is an emergent property of certain kinds of network, independent of the substrate within which the network exists.
Re: Artificial intelligence is not conscious – Ted Chiang
#674When the consciousness itself not understood and well defined in the first place, it is pretty pointless to debate if something is or isn't conscious. And here in particular the reasoning behind the argument is bizarre. Decomposing the complex activity into simple steps like 'predicting the next word' and claiming that surely can't have consciousness. A similar argument would be -- there is no way that movements of e…
From my pov and after using AI a lot to better understand its usefulness I see this discussion as totally misleading and irrelevant.
The elephant in the room is that even the best frontier models still produce errors.
This is a fundamental issue and should be discussed instead of some esoteric pseudo-scientific nonsense.
If the current systems can not produce reliable results they are useless. The promise of actual usefulness needs to be fulfilled in a very near future.
Re: Artificial intelligence is not conscious – Ted Chiang
#675Earlier quoted context omitted.
That's the "Chinese room" argument: https://en.wikipedia.org/wiki/Chinese_room It has been very well discussed before I don't understand how anyone sane can argue against it (and indeed you can see on Wikipedia that the replies to it are sparse and don't make any sense).
Well I don't understand how one can accept this argument. I mean if you believe in mind-body dualism it can make sense. But AFAIK Searle doesn't, instead he holds that there's something special about the brain biology that enables consciousness and that you won't find in a computer. I don't see why that would be the case if the computer can simulate the real world, and I find Searle's argument against simulation, tha…
> simulating rain doesn't make you wet, falls flat: it can make things wet in simulation, and if you connect it to sprinklers it will make you wet
It's not simulating rain if it's making you wet by using sprinklers?
Re: Artificial intelligence is not conscious – Ted Chiang
#676The article is a farce. Is this really the sort of slop we want to use as a proof that humans write better articles than AI?
> "If a company builds a machine that, when fed descriptions of assorted ethical dilemmas, emits sentences either of the form “Compromise your values” or “Don’t compromise your values,” it is not building a tool that assists people in their decision making; it is encouraging people to stop making decisions. "
A human is not diminished by access to tools or other humans.
As much as we want to pretend that decision-making is what makes us human, the economy and governments are built on delegation. Choice paralysis is a thing.
There is so many logical fallacies in the article I don't even know where to begin.
Re: Artificial intelligence is not conscious – Ted Chiang
#677The more I think about this, the more I become convinced that consciousness, as understood by humans, is meaningless once you can stop, store and replay the internal machinery that is supposed to produce it.
A human brain in a jar is still human, still conscious - and can still suffer. But if you somehow managed to digitize the whole thing, and run it in a computer it becomes something different entirely. You could record the most pleasurable thing in existence and have the digital brain relive it a million times, and it would be equally meaningless to torturing it a million times.
This is NOT inherently tied to meat vs machine - although it's difficult to imagine how you'd access the information stored in biological neurons, while for silicon chips it's trivial.
Whatever makes experiences, both good and bad, meaningful is tied to their permanence. Memory rooted in linear time, not something you can store, load or replay. Remove that, and whatever you're left with might be intelligent, but not conscious.
I don't think you could build something with LLMs today that would be considered conscious, even if you somehow manged to keep their context window inaccessible and linear in time. The separation of training vs inference probably makes that infeasible, even if you store "memories" in context, once the contents in it become too disjointed and too numerous, the resulting output of the LLM becomes gibberish. But it is certainly something that can change in the future.
Conversely, even the most intelligent and capable artificial intelligence system, far exceeding human capabilities, would not be conscious in a meaningful way, if you could store, load and replay everything it does.
Re: Artificial intelligence is not conscious – Ted Chiang
#678The fact that a LLM is essentially immutable would be my biggest argument against consciousness or self-awareness. It's a big file with a bunch of coordinates describing spatial relationships between tokens. When you give it a prompt, it uses those relationships to generate a string of tokens that is a statistically likely response to that prompt, then it stops. It's not changed by the experience. It doesn't remember…
There are people whose brains don’t form new memories anymore after an accident or surgery, and they eternally live in the time before it happened, and have no memory of what happened a minute ago. Still they are conscious.
I was like this for a bit and you still have memories from like 30 seconds to minutes ago, but after that you have a cliff where you don't remember.
I don't think LLMs structurally even get the 30 seconds part. It's literally 0 for them.
Re: Artificial intelligence is not conscious – Ted Chiang
#679Re: Artificial intelligence is not conscious – Ted Chiang
#680When the consciousness itself not understood and well defined in the first place, it is pretty pointless to debate if something is or isn't conscious. And here in particular the reasoning behind the argument is bizarre. Decomposing the complex activity into simple steps like 'predicting the next word' and claiming that surely can't have consciousness. A similar argument would be -- there is no way that movements of e…
Why do you have no inkling that your spreadsheet or terminal emulator is conscious, yet when that same machine is running an LLM all of a sudden we’re debating its consciousness? Anthropic seems to have chosen their in-house philosopher well - one who’ll be amenable to getting confused in their favour.
> Why do you have no inkling that your spreadsheet or terminal emulator is conscious, yet when that same machine is running an LLM all of a sudden we’re debating its consciousness?
Difference in size and complexity and nature of calculations being run?
I'd ask the other way - why do you (general you, people who do not have this inkling) have no problem debating consciousness of meat based brains, but it somehow becomes a category error when talking about silicon? Assuming you don't believe in divine magic, and that divine magic is core to consciousness, there's no reason to assume it's impossible a complex enough machine running complex enough software could be intelligent, or conscious - thinking is computation, and computation is made of math - it's independent of substrate that does the computing in the real world.
LLMs are definitely a different beast than regular software - both in their structure and in their generality. They may not be conscious or intelligent, maybe this specific design could never truly be (though I think it could) - but bucketing them with spreadsheets and terminal emulators is a real category error. If you stop fixating on the underlying substrate, then LLMs are already much more similar to biological minds than to any "regular" computer programs.
But that's still somewhat abstract. In immediate practical terms, it's also why I keep saying that anthropomorphising them gives a better high-order intuition: they are, by design, emulating human thinking in full generality, which makes their overall behavior, including their well-known problems like hallucinations or prompt injections (i.e. manipulation/gullibility), match what you'd expect of a people-like component of a system. It's a real, dangerous mistake, to treat LLMs like regular software components when designing systems.