"Artificial intelligence is not conscious". This is a common sense.
LLMs are not even artificial intelligence, and they are just advanced automation systems.
AI is just marketing language.
Artificial intelligence is not conscious – Ted Chiang
661–670 of 1001 posts
Re: Artificial intelligence is not conscious – Ted Chiang
#662I don't understand the most of the responses here. What does computers getting more intelligent has to do with it getting conscious? Even if AI with current computer architecture gets much more intelligent than humans, it will still have the same amount of consciousness. Its still bunch of instructions. If we got a lot of people together and did all the instructions by hand to get an LLM result would we also create t…
That's the "Chinese room" argument: https://en.wikipedia.org/wiki/Chinese_room It has been very well discussed before I don't understand how anyone sane can argue against it (and indeed you can see on Wikipedia that the replies to it are sparse and don't make any sense).
Well I don't understand how one can accept this argument. I mean if you believe in mind-body dualism it can make sense. But AFAIK Searle doesn't, instead he holds that there's something special about the brain biology that enables consciousness and that you won't find in a computer. I don't see why that would be the case if the computer can simulate the real world, and I find Searle's argument against simulation, that simulating rain doesn't make you wet, falls flat: it can make things wet in simulation, and if you connect it to sprinklers it will make you wet.
Re: Artificial intelligence is not conscious – Ted Chiang
#663> My intention is to highlight the fact that LLM conversations are cleverly disguised examples of sentence continuation Regardless of bigger issues, this kind of statement reveals a deep misunderstanding. Problem type does not limit problem complexity. Nor does problem type limit solution complexity or power. If a machine has to learn to understand humans to complete text, then that is what it has to do. And there is…
But the machines don't understand. They predict. And what they predict is the next token. I'm not trying to beat this horse to death, but you have to realize using the word "understand" is anthropomorphising it. It's essentially the chinese room experiment -- if the rules are followed, no understanding is neccessary . If the tokens didn't correlate to words imbued with meaning outside the system, if the LLMs were tra…
There's a growing body of evidence that most of what the brain does is constantly predicting the world around us - look into the predictive brain hypothesis if you're interested.
As Ilya Sutskever has pointed out, if you read a mystery novel up until the reveal of the culprit, and then fill in "The killer was _____", don't you need to understand the novel to accurately predict the next word?
Re: Artificial intelligence is not conscious – Ted Chiang
#664> My intention is to highlight the fact that LLM conversations are cleverly disguised examples of sentence continuation Regardless of bigger issues, this kind of statement reveals a deep misunderstanding. Problem type does not limit problem complexity. Nor does problem type limit solution complexity or power. If a machine has to learn to understand humans to complete text, then that is what it has to do. And there is…
But the machines don't understand. They predict. And what they predict is the next token. I'm not trying to beat this horse to death, but you have to realize using the word "understand" is anthropomorphising it. It's essentially the chinese room experiment -- if the rules are followed, no understanding is neccessary . If the tokens didn't correlate to words imbued with meaning outside the system, if the LLMs were tra…
You are oversimplifying. They do produce one word per cycle. But they can also have context buffers carrying up to two million tokens, which is most definitely larger than your measly human short-term memory context buffers.
You, of course, wouldn't notice if your only experience of LLMs was chatting with the cheapest, smallest, least capable LLMs that you get through ChatGPT, or Google search.
It becomes pretty obvious when you use a coding AI on a daily basis. It is the context buffer in which the magic occurs, not the tokens that get spit out one at a time.
Every day, I watch my coding AI develop plans, search the web a half dozen times for documentation, grep through my entire codebase looking for pieces of related code and context, analyze relevant source code across multiple files, spit out an initial plan for implementing the fix before starting to execute it, run requests through some sort of advanced mathematics tool (they are EXTREMELY good at graduate-level calculus and linear algebra), implement fixes that extend across half a dozen files in 2 different computer languages (typescript and C++), run trial compiles and fix coding errors in its output, sometimes developing sub-plans to deal with compile errors. I've seen it get halfway through a fix and revise its initial plan mid-flight as it encounters something in existing source code.
Not vibe coding, to be clear. Targeted use of a coding tool by a by a professional senior software developer with decades of experience, and fair bit of expertise with the limits of what sort of problems my coding AI can and cannot do. Every line code reviewed. Sometimes it needs additional prompts, telling it how it mis-implemented something, or specifying more carefully what I actually want but didn't properly express in the initial request
All the time maintaining that context across multiple request, so that I don't have to restate requests from scratch.
A particularly interesting revision: "You have misread the equation (13) on page 112 of 'Spice, the Manual 2nd ed.'. I should be ....". (It had previously identified the textbook as a source I was using, from comments in source, in a preceding request, and actually already read cited pages in the PDF file, which it had found online). And I had actually asked it to implement equation (13), which was, in fact, badly typeset. The error it had made was defensible, if not the best reading of the equation.
"You are correct. Let me fix that." (producing updates to the implementation of the equation in code, AND code that implements the symbolically-differentiated version of that equation 60 lines later, which is not explicitly given in the text). The text says "take the lagrangian of equations (11), (12) and (13)" or something like that.
ALL information that gets carried in context buffers, even though it's generating code one word at a time. The bulk of the magic occurs in context buffers, not spitting out words one at a time, which, for my coding AI is, I think about 250,000 tokens.
I think it's pretty safe to think that my coding AI is working out of context buffers that may carry plans and research results consisting of tens or hundreds of thousands of arranged tokens carried in context buffers through the multiple steps of the implementation, and later revision. None of that would be possible if were simply working one token ahead.
I kind of suspect that a lot of activity occurs in the first few words of its response. "Let me examine your current source code and develop a plan. Ok. I can see on line 131 where you want me to implement the equation.". (An opportunity to perform about 27 updates of the context buffer). And in the sometimes hundreds of lines of output it generates as it talks itself through what it needs to do.
Re: Artificial intelligence is not conscious – Ted Chiang
#665This kind of debate would be improved by not using the word "just". You cannot say AI is "just" X or Y. You can say that it is X or Y, but you still have to prove separately how and why this means it's not something else besides.
Re: Artificial intelligence is not conscious – Ted Chiang
#666Earlier quoted context omitted.
The danger of anthropomorphism is not we elevate the machines, it's that we debase humanity. I also think different ideas get conflated. It may be possible to build a machine that is super-human in the sense it can outperform the human brain in all kinds of measurable ways. Does not imply it possesses all the same qualities of the brain. I respect a number of things Anthropic has published about the ethical issues at…
I think humanity deserves to be debased in part because it's awfully egocentric and insist on being special in the universe. There are numerous times when humanity's been debased. Copernicus, Kepler, Darwin, and the infinite number of animal behaviorists who have defined and documented consciousness And how it manifests in primates, parrots, dogs, cetaceans, and insects. (I just love the bumble bees taking time off f…
I hope before I die we finally prove that the human brain has no peculiar qualia but it is an entirely deterministic, albeit extremely sophisticated, machine. And by touching the right triggers, even the worst human being can become a saint.
That would finally force us to rethink how we see the morality of "virtuosity", punishment and our justice system.
Re: Artificial intelligence is not conscious – Ted Chiang
#667When the consciousness itself not understood and well defined in the first place, it is pretty pointless to debate if something is or isn't conscious. And here in particular the reasoning behind the argument is bizarre. Decomposing the complex activity into simple steps like 'predicting the next word' and claiming that surely can't have consciousness. A similar argument would be -- there is no way that movements of e…
Not only is it not conscious, but it is not tired. Its claims to human feelings that have literally nothing to do with software on silicon are just traces of training, echoes of a human dataset. The fact that its claims are absurd mean that, among other things, it its proprioception is false, and serving no purpose, quite unlike the feelings of the conscious beings that it mimics so well.
I think you make a good argument. An LLM can't "be tired", so it is clearly lying and/or mimicking what a human does.
But I have two counter-arguments: - maybe the LLM thinks it is tired, because it thinks that it is a human or behaves like one. And thinking you are tired while not really being it is something humans happen to do. (And humans are conscious right?) - alternatively, maybe the LLM says it is "tired" in a colloquial form, i.e. it is not really "tired" but it has something analogous to it. Maybe it is annoyed by the conversation and decided to use that word?
Re: Artificial intelligence is not conscious – Ted Chiang
#668Earlier quoted context omitted.
Does the moon have gravity?
We can directly measure the thing we call gravity, so in that sense, it is well understood. We even understand it well enough to make predictions about what it will do under which circumstances. We can't measure consciousness. We can't quantify, or even qualify, what it is. We don't even have a framework to ask a meaningful question, so debating an answer feels premature.
The way we measure gravity is by observing that it behaves how we would expect it to. We don't actually know if this is correct.
The same goes for consciousness, if something behaves how you would expect a conscious thing to behave but you don't know what is causing it can you deny it whilst maintaining that it is reasonable to say the moon has gravity.
Re: Artificial intelligence is not conscious – Ted Chiang
#669Ted Chiang is brilliant. His novella “Anxiety is the Dizziness of Freedom” altered the course of my life. It changed the way I looked back at certain pivotal moments in my life and taught me to think about those pivotal moments differently than how I was thinking about them. Similar to what happens to one of the characters in the story who ends up changing their perception of a key moment in their life. I won’t go in…
don't you mean "The Truth of Fact, The Truth of Feeling"? I think that matches your description more, but I could be wrong
“The Truth of Fact, The Truth of Feeling” is also a good story.
But I really was referring to “Anxiety is the Dizziness of Freedom”.
I hope what I am about to write below in the rest of my comment here isn’t too much of a spoiler. It might be. I encourage anyone reading this thread to read that short story before reading the rest of my comment here.
But with (hopefully minimal) spoilers,
the character Dana in “Anxiety is the Dizziness of Freedom” realizes something at the very end of the story about a certain event that had happened earlier in Dana’s life. The story made me realize the same thing about certain events in my life.
Of course the events in my life are very different from what happened with Dana in the short story. But the realization Dana has is applicable to many things in people’s lives. Especially events related to emotions of regret and guilt.
Re: Artificial intelligence is not conscious – Ted Chiang
#670When the consciousness itself not understood and well defined in the first place, it is pretty pointless to debate if something is or isn't conscious. And here in particular the reasoning behind the argument is bizarre. Decomposing the complex activity into simple steps like 'predicting the next word' and claiming that surely can't have consciousness. A similar argument would be -- there is no way that movements of e…
Daniel Dennet in „Consciousness Explained” argues that consciousness is an emergent phenomenon, and when we look at its individual components, it is like seeing an illusionist’s trick. We may wonder how many grains of sand make a dune, of how many molecules of water make a liquid. (John Conway would argue that it takes a single spin-1 particle to have free will, but I digress.) The same way, even if individual chemic…
Which to me, raises an interesting question:
- How many individual components do you need for it to emerge?
Most people would be okay with saying that individual cells are not conscious, maybe even that tree are not conscious even thought they are made of many cells. Neurons seem to be the determinant factor in deciding whether something could have a consciousness, but again how many do you need? Does growing 1B neurons count as a brain if they are not organized?