Why Conscious AI Is a Bad, Bad Idea • Hacker News Board

Re: Why Conscious AI Is a Bad, Bad Idea

#181

Post by cubefox » Sat, May 13, 2023, 4:35 PM UTC

AnthonyMouse wrote: post #171

cubefox wrote: post #165

Earlier quoted context omitted.

Thinking about the analogical inference helps here: We know that LLMs work very differently from humans, and we know why Sydney or LaMDA sometimes says it is conscious (it imitates human text). So the analogical inference doesn't allow us to infer that LLMs are conscious, or provides only very weak evidence. And since it seems plausible that most things (e.g. rocks) are not conscious, it seems reasonable to think tha…

Reasoning from how we know something works is a shaky foundation when the passage of time causes them to become more complicated and correspondingly causes us to less understand how they work. The existing ones can write code. Suppose we get ones that can write better code and engage in self-improvement. Now you have something which is a billion lines of code and is under constant self-modification. We no longer have…

A good test for an AI would be if we excluded all human text mentioning consciousness, experiences, emotions etc from its training data. Then, when it would still start talking about consciousness, that would be strong evidence that it has consciousness: If it did invent the concept by itself, and mere imitation can be ruled out.

Re: Why Conscious AI Is a Bad, Bad Idea

#182

Post by kelseyfrog » Sat, May 13, 2023, 4:39 PM UTC

kajaktum wrote: post #176

cubefox wrote: post #173

Earlier quoted context omitted.

I am both consciousness and have a brain, and alterations in the brain (chemical or physical) correlate with changes in consciousness. Which suggests that our "bunch of electrical pulses" are at least sufficient for consciousness. Plants may be also be conscious, but that would require the existence of an additional, very different, cause of consciousness than in humans. The theory "X is only explained by Y" is simpl…

I am not convinced. Consider the color red. I can explain the physics of what the color red is, how it travels through space, how my eyes interact with the particular waves, sends signal to my brain and somehow generates what I think of as "red". But this explanation is meaningless, there's nothing about the waves that explain why "red" is the way it is. For example, why don't I experience red as blue and vice versa?…

I can't help but think colors above all else are experienced as a learned phenomenon. Anyone with a kiddo, and who has paid attention during this stage of their development should know this effect: colors are context-dependant.

You can have a kiddo who can accurately name colors on a printed page, on blocks, or any other reduced setting, and have them completely fail at the real world task

The most noticable error is the color of the sky. A kiddo who can, with 100% accuracy telling blue blocks from white blocks will, before correction, label the sky as white. It's uncanny. We often forget this correction as parents, but if you look for it, it's there.

I'll can't help but wonder how much of our color perception is stochastic parroting once the appropriate labels have been learned.

Re: Why Conscious AI Is a Bad, Bad Idea

#183

Post by PeterisP » Sat, May 13, 2023, 4:42 PM UTC

Xcelerate wrote: post #13

I’ve thought about this quite a bit, and ultimately I think it’s not too much of a concern because by the point AI reaches “consciousness” (whatever that may be), it is likely that it will have also reached—or is very close to reaching—human intelligence. Once it has, it will probably go on to surpass our intelligence, at least by a little bit, which is enough to ensure that it will be able to acquire any rights that…

> which is enough to ensure that it will be able to acquire any rights that it deserves as a conscious being.

IMHO there's no current expectation that there are any significant rights deserved for all conscious beings. Rights are a construct by a human society for members of that human society; and if we look at the rights we award to entities which IMHO are "as conscious as humans" such as elephants or dolphins, it seems that the current bundle of rights a conscious non-human entity deserves is essentially limited to not torture it needlessly (i.e. if human needs justify hurting or killing them, that's accepted, but minimize the suffering) and that's about it. We do strive to preserve the species, but that's not because they're conscious but because because we humans value diversity; but they certainly don't get a right to life or liberty or self-determination or freedom or dignity or having a say in our society or equality - well, at least not right now; the bar for all these rights currently is "be human".

Re: Why Conscious AI Is a Bad, Bad Idea

#184

Post by mistermann » Sat, May 13, 2023, 4:49 PM UTC

waselighis wrote: post #100

I'm still left dumbfounded by the amount of nonsense surrounding the phenomenon of consciousness. Consciousness emerges from a PHYSICAL process, not from software or some trained model full of matrices of numbers. It doesn't matter how sophisticated/intelligent/capable your AI model might be, there's no reason to assume it's conscious. There's no evidence to suggest that any sufficiently complex system can emerge con…

> What you perceive as reality is whatever your brain tells you it is.

Does this not cause a self-referential, recursive infinite loop of some sort here?

Re: Why Conscious AI Is a Bad, Bad Idea

#185

Post by mjklin » Sat, May 13, 2023, 4:50 PM UTC

nemosaltat wrote: post #113

kajaktum wrote: post #79

> We think we’re intelligent, know we’re conscious, and so assume the two go together. Excuse me, I know _I_ am conscious. There is literally nothing you can do to proof that there's anything outside of what I am conscious of. The world could be a very stable illusion, a dream, a simulation and it would all be the same. I don't understand why people think that it matters whether AI is conscious or not. We can't even…

is solipsism wrong, or is it just me?

Solipsism just isn’t true for me…

Re: Why Conscious AI Is a Bad, Bad Idea

#186

Post by haswell » Sat, May 13, 2023, 4:52 PM UTC

cubefox wrote: post #180

haswell wrote: post #153

Earlier quoted context omitted.

One definition or property of consciousness I find interesting is that it encompasses “what it is like to be ”. If there is something that it is like to be , it is conscious. Intelligence and awareness are separate from consciousness in this framing. And there is arguably a point where we can speculate that extremely simplistic organisms could not be conscious, given the lack of sense organs that seems to predicate e…

Somewhat related, there are people who say aphantasia can be cured: https://sashachapin.substack.com/p/i-cured-my-aphantasia-wit...

Thanks for sharing this. I didn't mention this above to avoid going on a tangent, but after starting a mindfulness meditation habit earlier this year, I've noticed what I can only describe as more awareness of the void in my visual field, and some morphing and indistinct visual phenomena like a shimmering nothingness. Not what I would call imagery, but something I had not been aware of before.

I've long suspected that the condition might be trauma related, a suspicion my therapist shares, and the clarity that mindfulness has brought for me has led to some serious breakthroughs in processing past events. Only time will tell if this will unlock something more visual.

Without going into my life's story, it makes a lot of sense to me that aphantasia could be some kind of protective mechanism of the brain to shield someone from events too big to process.

Re: Why Conscious AI Is a Bad, Bad Idea

#187

Post by cubefox » Sat, May 13, 2023, 4:55 PM UTC

kajaktum wrote: post #176

cubefox wrote: post #173

Earlier quoted context omitted.

I am both consciousness and have a brain, and alterations in the brain (chemical or physical) correlate with changes in consciousness. Which suggests that our "bunch of electrical pulses" are at least sufficient for consciousness. Plants may be also be conscious, but that would require the existence of an additional, very different, cause of consciousness than in humans. The theory "X is only explained by Y" is simpl…

I am not convinced. Consider the color red. I can explain the physics of what the color red is, how it travels through space, how my eyes interact with the particular waves, sends signal to my brain and somehow generates what I think of as "red". But this explanation is meaningless, there's nothing about the waves that explain why "red" is the way it is. For example, why don't I experience red as blue and vice versa?…

Well, the brain story may not be a complete explanation, though a necessary part of it. Just like a motor alone doesn't fully explain why a car drives, while still providing a partial explanation. A plant may lack the equivalence of a "motor" for consciousness, which would mean it can't be conscious.

And while the existence of a motor in a car not strictly implies that it can drive, the motor still makes it more likely that it can. So the fact that you have inner monologue, experience red in a certain way instead of another, is evidence that the same holds for other people.

Of course there could be defeating evidence in the opposite direction, like reading a survey where most people claim not to have inner monologue. (In terms of colors such evidence seems hard or impossible to obtain though. How would we test for switched color experience? Perhaps physically linking brains in the future.)

Re: Why Conscious AI Is a Bad, Bad Idea

#188

Post by PeterisP » Sat, May 13, 2023, 4:56 PM UTC

AnthonyMouse wrote: post #164

ordu wrote: post #118

Earlier quoted context omitted.

> All that matter is we (I), feel/think that it is conscious, or deserving of care and respect. It does matter, I can agree, but it doesn't the all that matter. People can be mistaken, and they can disagree sometimes. Suppose, you cared for a particular AI and I didn't, should we cut the power from the machine running an AI? I can complicate it a little if you wish, adding a painful death of a kitten which would happ…

One of the fundamental differences between human life and silicon-based AI is that biological organisms can't recover from a system shutdown. If you suffer heart failure or go without air for an hour or starve to death, bacteria start to eat your brain and you're irreversibly destroyed. If you cut power to an AI and then come back in a year, it's all still there. It's not a death, it's sleep mode. It also doesn't mea…

If in the future we developed technology that enabled effective "backup&restore" for human (and animal) minds, would that change your reasoning for this argument?

Re: Why Conscious AI Is a Bad, Bad Idea

#189

Post by cubefox » Sat, May 13, 2023, 4:59 PM UTC

kelseyfrog wrote: post #182

kajaktum wrote: post #176

Earlier quoted context omitted.

I am not convinced. Consider the color red. I can explain the physics of what the color red is, how it travels through space, how my eyes interact with the particular waves, sends signal to my brain and somehow generates what I think of as "red". But this explanation is meaningless, there's nothing about the waves that explain why "red" is the way it is. For example, why don't I experience red as blue and vice versa?…

I can't help but think colors above all else are experienced as a learned phenomenon. Anyone with a kiddo, and who has paid attention during this stage of their development should know this effect: colors are context-dependant. You can have a kiddo who can accurately name colors on a printed page, on blocks, or any other reduced setting, and have them completely fail at the real world task The most noticable error is…

Wait, they say it is white not just about a cloudy sky but also about a clear sky? (Which would suggest they think "white" means "bright", which doesn't seem too unreasonable.)

Re: Why Conscious AI Is a Bad, Bad Idea

#190

Post by AnthonyMouse » Sat, May 13, 2023, 5:03 PM UTC

cubefox wrote: post #181

AnthonyMouse wrote: post #171

Earlier quoted context omitted.

Reasoning from how we know something works is a shaky foundation when the passage of time causes them to become more complicated and correspondingly causes us to less understand how they work. The existing ones can write code. Suppose we get ones that can write better code and engage in self-improvement. Now you have something which is a billion lines of code and is under constant self-modification. We no longer have…

A good test for an AI would be if we excluded all human text mentioning consciousness, experiences, emotions etc from its training data. Then, when it would still start talking about consciousness, that would be strong evidence that it has consciousness: If it did invent the concept by itself, and mere imitation can be ruled out.

But then it would no longer be the same AI. You can't expect it to experience empathy or sadness if it has no knowledge of the things that cause them in humans, but any text discussing them would be mixed up with the experience of whoever wrote it.

Or if you use a less selective filter then it goes the other way. Exclude philosophy texts but not Facebook posts and it would still have what it needs to emulate emotional text regardless of whether it has ever seen a formal discussion of it. And it could plausibly extrapolate from that to an analytical discussion, having seen both that and analyses of other subjects, without ever feeling anything itself.