Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Artificial intelligence is not conscious – Ted Chiang

theatlantic.com

Post reply on HN
Print view View original

841–850 of 1001 posts

Re: Artificial intelligence is not conscious – Ted Chiang

#841

Post by twobitshifter »

Jtarii wrote: post #716
iLoveOncall wrote: post #606

Earlier quoted context omitted.

That's the "Chinese room" argument: https://en.wikipedia.org/wiki/Chinese_room It has been very well discussed before I don't understand how anyone sane can argue against it (and indeed you can see on Wikipedia that the replies to it are sparse and don't make any sense).

It's fairly obvious to me that the room as an entire system understands Chinese. The human in his thought experiment would just represent the small fraction of the brain that encodes/decodes symbols.

Agreed, he’s saying if he’s working with a black box he doesn’t understand what it’s doing. Which is of course true. If the black box were a chinese person instead of a tool and he was writing down what they said, would he say the chinese person in the black box does not understand why they told him to write something? No, of course not.

Today we know the program to do this is always going to be inscrutable to Searle. His role is no different from someone using Claude to write an email.

Re: Artificial intelligence is not conscious – Ted Chiang

#842

Post by JCTheDenthog »

slibhb wrote: post #836

There's currently no way to prove that something is or isn't conscious. I would bet that rocks and LLMs aren't conscious. I would bet that humans and dogs are conscious. But I can't prove it. Whenever people try to prove it (in either direction), they end up resorting to sophistry. This state of affairs could change. If neuroscientists develop a deeper understanding of consciousness, we could talk about it with more…

I mean, outside of our own internal experience of qualia, there's no way for us to prove anyone or anything is conscious. But my biggest issue with the debates about consciousness and LLMs is that the "LLMs are conscious" crowd usually has to presuppose a completely materialistic/mechanistic understanding of the universe and of our own consciousness, and that's something that we really don't know for certain. Also, if we were running LLMs on a mechanical or pneumatic or water based computer (albeit very slowly) then the non-electronic computer would be just as "conscious" as the normal electronic computer, and I think it would give a lot of the "LLMs are conscious" crowd pause to think that a bunch of pipes and valves can be conscious. I think there's a lot of magical thinking that arises from the fact that computers are electronic.

Re: Artificial intelligence is not conscious – Ted Chiang

#843

Post by jquery »

JKCalhoun wrote: post #742
sega_sai wrote: post #425

When the consciousness itself not understood and well defined in the first place, it is pretty pointless to debate if something is or isn't conscious. And here in particular the reasoning behind the argument is bizarre. Decomposing the complex activity into simple steps like 'predicting the next word' and claiming that surely can't have consciousness. A similar argument would be -- there is no way that movements of e…

I agree with your point. We've been watching the goal posts being moved ever since Turing (who, we might admit, gave a somewhat mushy answer anyway). At the same time, the exercise (and even moving the goal posts) is useful. I think we are allowed to say, "This is consciousness!" And when presented with a machine equivalent, say, "That's not what I meant." As long as you can then further clarify what "it" is. Of cour…

They’re already sounding pretty shrill imo. They’ve gone from “consciousness is X”, to “actually consciousness is X + Y”, to “actually it’s X + Y + Z + theta + omega + quantum unobtainium”

Re: Artificial intelligence is not conscious – Ted Chiang

#844

Post by wellpast »

andrewflnr wrote: post #839
slibhb wrote: post #836

There's currently no way to prove that something is or isn't conscious. I would bet that rocks and LLMs aren't conscious. I would bet that humans and dogs are conscious. But I can't prove it. Whenever people try to prove it (in either direction), they end up resorting to sophistry. This state of affairs could change. If neuroscientists develop a deeper understanding of consciousness, we could talk about it with more…

I actually don't think neuroscience can solve it either. Science could decide whether, e.g. equivalent physical processes are occurring in dogs and humans, and you could argue that solves the question for things with physical brains, but even then it can't measure whether another brain has the same subjective experience as you do. It can't really do that for other humans. It certainly can't answer that question for h…

Simply put, even solipsism is unfalsifiable.

Re: Artificial intelligence is not conscious – Ted Chiang

#845

Post by wint3rmute »

andrewflnr wrote: post #839
slibhb wrote: post #836

There's currently no way to prove that something is or isn't conscious. I would bet that rocks and LLMs aren't conscious. I would bet that humans and dogs are conscious. But I can't prove it. Whenever people try to prove it (in either direction), they end up resorting to sophistry. This state of affairs could change. If neuroscientists develop a deeper understanding of consciousness, we could talk about it with more…

I actually don't think neuroscience can solve it either. Science could decide whether, e.g. equivalent physical processes are occurring in dogs and humans, and you could argue that solves the question for things with physical brains, but even then it can't measure whether another brain has the same subjective experience as you do. It can't really do that for other humans. It certainly can't answer that question for h…

I've heard an interesting argument about this while going down the rabbit hole of the hard problem of conciousness. Unfortunately can't rember the source - if you can recognize it, do let me know!

Back in the XVII/XIX century, a similar problem existed regarding life - the problem of "what makes living things tick". The assumption at that time was that while we can understand the biological processes around life, we will never understand the so-called "vital force", which causes things to live - life itself. I know it sounds weird now, but back in the day the mental models were different. Phenomenas like "water boils" and "organisms self-replicate" were treated as completely different domains of reality, without an overarching uniform scientific model.

It turned out that after around 100 years, we can figure out the chemical/physical processes and the need for the term "vital force" became redundant.

While this is certainly not an argument proving that the Hard Problem is not in fact hard, it is an interesting idea to think about. Perhaps its all a matter of developing better, higher-resolution neurological models which will at some point give us the tools to decompose qualia.

Re: Artificial intelligence is not conscious – Ted Chiang

#846

Post by viccis »

Taek wrote: post #285
viccis wrote: post #199

Earlier quoted context omitted.

I have yet to be convinced that LLMs can produce definitive knowledge that is not a result of combining previous information. Humans can (if they can't then science basically collapses epistemelogically, see: philosophical skepticism), but I see no evidence of LLMs doing it. And from the number of truly new ideas and concepts delivered by LLMs (exactly zero), I think it's reasonable to just treat them as induction ma…

I would like to push back on the idea that humans can provide definitive knowledge that is not a result of combining previous information. Pretty much every single idea in science can be traced back to some combination of earlier ideas, and as they get earlier / simpler, they can be related back to some combination of direct observations. It's not clear to me at all that our entire body of scientific knowledge can't…

How can you say for sure that every effect must have a cause? How can you be any more sure about that there can't be an effect without a cause than the old believe that black swans couldn't exist that was so universally believed that it became a cliche ("rara avis in terris nigroque simillima cygno")?

If every piece of knowledge is created by combining previous ones, then no true grounded knowledge is possible. Every bit of knowledge is grounded by another bit of knowledge which is itself grounded on, ..., etc, etc.

https://plato.stanford.edu/entries/induction-problem/

Re: Artificial intelligence is not conscious – Ted Chiang

#847

Post by etbebl »

varjag wrote: post #798
indoordin0saur wrote: post #793

I often hear the assumption that LLMs can or will become conscious because consciousness is likely substrate independent . The idea being that our brain is just a computer made out of meat and it doesn't do anything that can't be precisely simulated by a silicon computer. But I wonder if some of the magic in the human brain is its analog nature. Chemical signaling and impulses of neurons interact with each other with…

Each and every one of many billion transistors in a GPU works due to certain quantum effects on P-N junction. That does not mean GPU computation is nondeterministic, unknowable or magic.

Sure, but it's designed to minimize the chance that a quantum fluctuation could change the outcome of a computation, right? Whereas in the brain that might not be the case. A lot of the "interesting" neural activity (e.g. relating to decision making, language, etc) happen in highly sensitive dynamical regimes: on a threshold of firing, or activating one neural population vs another. (Arguably you can get the same effect in an artificial network by adding true random noise though!)

Re: Artificial intelligence is not conscious – Ted Chiang

#848

Post by throwthrowuknow »

overgard wrote: post #161
throwthrowuknow wrote: post #143

Earlier quoted context omitted.

The body itself has little effect on the mind other than the inputs from nerves and chemical and hormonal changes. These inputs are analogous to tokens and parameters for an LLM. You could theoretically reserve some tokens to be used as “physical” or “emotional” sensations that would affect the functioning of the model.

That's not really true, the body actually has pretty profound effects on the mind. For instance, there are working theories that many mental disorders are actually metabolic disorder, some neurotransmitters are majority produced in the gut (seratonin), and even things like an elevated heart rate will create emotional states of anxiety even if you're not consciously anxious of anything. That's like the tip of the iceb…

Which of these do not involve chemical and neural communication?

Re: Artificial intelligence is not conscious – Ted Chiang

#849

Post by overgard »

oidar wrote: post #359
overgard wrote: post #302

Earlier quoted context omitted.

Amusingly, I think Ted Chiang actually wrote a short story about this very concept (it involves people committing a form of suicide that removes their conscious experience but they still act and live in society as some kind of psychic zombie. I’m pretty sure it was him anyway, can’t recall the story name

Maybe “What's Expected of Us”?

I was trying to find it, but I read it a long time ago and I have a bad feeling I might have mashed a few different stories together in my head. That is a good story though!

Re: Artificial intelligence is not conscious – Ted Chiang

#850

Post by throwthrowuknow »

xboxnolifes wrote: post #271
throwthrowuknow wrote: post #143

Earlier quoted context omitted.

The body itself has little effect on the mind other than the inputs from nerves and chemical and hormonal changes. These inputs are analogous to tokens and parameters for an LLM. You could theoretically reserve some tokens to be used as “physical” or “emotional” sensations that would affect the functioning of the model.

"Other than... chemical... changes" is doing a lot of heavy lifting. The entire existence of your body is a continuous process of chemical reactions.

I can narrow that to the chemical signals that can cross the blood-brain barrier if that makes the point clearer.
Post reply on HN