Why Conscious AI Is a Bad, Bad Idea • Hacker News Board

Re: Why Conscious AI Is a Bad, Bad Idea

#151

Post by ly3xqhl8g9 » Sat, May 13, 2023, 3:29 PM UTC

Geee wrote: post #142

ly3xqhl8g9 wrote: post #80

Earlier quoted context omitted.

The conundrum appears when the machine needs to suffer in order to become an agent taking decisions in our world. For example, the machine should know not to cause a fire in my house while doing the cleaning, it should be "afraid" of fire. My point is, if synthetic agents do indeed need to suffer to be agents, we will accept it just as easily we accept now that our cars take on rust. And of course, we will minimize t…

We should be able to send robots into a fire, without a second thought. The robot should not feel any kind of suffering from the fire. Whoever owns the robot should decide how the robot should act. It may calculate the expected profit / cost of the action for the owner, but the owner should make the final decision. Programming it to avoid setting things on fire is just programming; there's no need for emotions or oth…

We of course don't know how to engineer agency, but if we want black box top-down persuasion, how we train a dog without knowing how each neuron in the dog's brain works, some independent will may be required: you don't want to tell the robot how to fight each fire specifically (or train it on petabytes of fires just to have it stop in front of a shop called STOP [1]), you want to prove some general guidelines, and let it handle the actual cases itself.

[1] Don't name your Store "Stop", https://www.reddit.com/r/SelfDrivingCars/comments/k56za3/don...

Re: Why Conscious AI Is a Bad, Bad Idea

#152

Post by AnthonyMouse » Sat, May 13, 2023, 3:30 PM UTC

cubefox wrote: post #135

kajaktum wrote: post #79

> We think we’re intelligent, know we’re conscious, and so assume the two go together. Excuse me, I know _I_ am conscious. There is literally nothing you can do to proof that there's anything outside of what I am conscious of. The world could be a very stable illusion, a dream, a simulation and it would all be the same. I don't understand why people think that it matters whether AI is conscious or not. We can't even…

The fact that other humans are so similar to me (all humans are similar in virtue of being human organisms) allows for an analogical inference: You guys are similar to me, I am conscious, therefore you are probably also conscious. And whether or not something is conscious matters a lot ethically. If something is conscious, it may suffer. Conscious and unconscious objects have to be treated in a completely different w…

> We can't know for certain whether something is or isn't conscious, but we have to try to make a best guess.

We now have AI which is trained on data generated by humans who are (probably?) conscious and feel pain etc. So it can generate output which implies that it's conscious and can feel pain etc., even when that isn't the case.

What do you do with something that can pass the Turing test but still isn't human?

Re: Why Conscious AI Is a Bad, Bad Idea

#153

Post by haswell » Sat, May 13, 2023, 3:32 PM UTC

jnwatson wrote: post #139

cubefox wrote: post #135

Earlier quoted context omitted.

The fact that other humans are so similar to me (all humans are similar in virtue of being human organisms) allows for an analogical inference: You guys are similar to me, I am conscious, therefore you are probably also conscious. And whether or not something is conscious matters a lot ethically. If something is conscious, it may suffer. Conscious and unconscious objects have to be treated in a completely different w…

While assuming that other human-looking things are conscious is a reasonable starting point, it certainly isn’t a given. There seems to be a wide variety of thinking processes. Some folks don’t have internal monologues. Some folks can’t visualize images at all, and the ones that do, do it will varying amount of detail. Isn’t it also reasonable that other humans have different degrees of consciousness, including perha…

One definition or property of consciousness I find interesting is that it encompasses “what it is like to be ”. If there is something that it is like to be , it is conscious. Intelligence and awareness are separate from consciousness in this framing. And there is arguably a point where we can speculate that extremely simplistic organisms could not be conscious, given the lack of sense organs that seems to predicate experience.

From a human point of view, the things you describe are the contents of consciousness. I have Aphantasia, while my brother describes his mind’s eye as CAD software and he can construct and manipulate visualizations at will.

The overarching awareness we both have that allows us to compare and contrast these things and make any sense out of those comparisons points to a more fundamental layer.

What you describe sounds closer to levels of awareness and one’s ability to recognize the workings of their own mind, e.g. some people remain lost in and identified with thoughts, while some are able to both experience and observe thought as just more contents of conscious experience, but not the actual center of one’s consciousness.

And there’s evidence that this can be learned (through mindfulness), which to me points to something like: we’re all conscious whether we realize it or not, and not realizing it doesn’t make it not so.

Re: Why Conscious AI Is a Bad, Bad Idea

#154

Post by nmilo » Sat, May 13, 2023, 3:34 PM UTC

magnetic wrote: post #104

nmilo wrote: post #93

A lot of people have wrote about the ethical considerations and to me they make no sense. Since when did our generally accepted ethics say that imposing suffering onto a conscious being is bad, but non-conscious beings is fine? This is the completely wrong correspondence because there's counter-arguments for both sides. You wouldn't harm a puppy, even though it's (probably?) not conscious, and I personally wouldn't c…

> You wouldn't harm a puppy, even though it's (probably?) not conscious What makes you think a puppy is not conscious? For ref: conscious | ˈkänSHəs | adjective aware of and responding to one's surroundings; awake. Source: have a puppy. He's plenty aware of his surroundings.

You found the wrong form of the word. There's the medical consciousness found in the dictionary, as in, "He is asleep so he has lost consciousness," and there is the metaphysical consciousness as in "does a fish have consciousness?" The latter is what people refer to in ethical debates, the former is more of a medical term.

Re: Why Conscious AI Is a Bad, Bad Idea

#155

Post by peoplefromibiza » Sat, May 13, 2023, 3:36 PM UTC

JoeAltmaier wrote: post #108

peoplefromibiza wrote: post #39

Earlier quoted context omitted.

> Yes, because we all know that humans care so much about the suffering of others: from fellow humans dying right now pointless deaths We are provably the only living being on Earth that do care about it. > If we will be able to spawn an agent in a household humanoid robot and let them do the chores around the house, will we do it, even if it will imply killing that agent whenever it powers down? You betcha That's th…

Never met a dog I guess.

this is a textbook example of involuntary proving the point you were trying to disprove

dogs are not like that because they are dogs, but because we humans literally created them with genetic manipulation over millennia

dogs like we know them did not exists in nature before we invented them to be the way they are

we literally put what we wanted them to be into them

and terminated the experiments that failed (we literally eradicated the most problematic breeds we created)

of course we like them, we made them to like us and to be likeable by us

now take an African wild dog or a wolf and tell me they act like domestic dogs...

Re: Why Conscious AI Is a Bad, Bad Idea

#156

Post by ericb » Sat, May 13, 2023, 3:36 PM UTC

Lionga wrote: post #82

ericb wrote: post #72

Earlier quoted context omitted.

I feel like this is also what evil people tell themselves. They need an internal explanation for the wondrous and beautiful goodwill and altruism in the world, and if they project their selfish world-view out, the only explanation they can come up with is that it must be self-interested behavior, somehow. It's like the idea that some people want to make the world better is inconceivable to them, because they don't.

I feel like this is what people tell themselves that can't accept who they really are. They need an internal explanation for why their are so much more moral and better than the rest. It's like the idea that they are just made of the same atoms that just follow the laws of physics and have all the same neural pathways and are not some divine, good being is inconceivable to them.

Have you ever helped someone without some connection to a reward?

As a thought experiment, you are in a room with a deaf, blind dying person--they will soon be gone from this world. They are reaching for water you can see on their bedside, but can't quite reach it.

Do you bother to cross the room to hand it to them if they will never "be aware of your virtue"? Why expend the energy? Compassion is real, for those who have it.

And to be clear, my definition of evil isn't intended here as some religious appeal. In my view, we are all playing an iterated prisoner's dilemma with each other, and good is believing in a better world and collaborating, while evil is defecting and making everything worse for everyone.

Re: Why Conscious AI Is a Bad, Bad Idea

#157

Post by denton-scratch » Sat, May 13, 2023, 3:38 PM UTC

kukkeliskuu wrote: post #116

Most of us share the experience of having driven a car from A to B, without having any conscious recollection about the trip. From such experiences we can deduce that our level of consciousness is not constant, but it fluctuates. Our experience, however, is that consciousness is a constant. The ancient mystics, monks and people studying meditation and other similar practices have made some practical observations. The…

> We should be less concerned about the consciousness of the AI

Author's warning seems to be:

1. "Conscious" means you can feel things (have feelings)

2. "Conscious" means you can have intentions

3. --> A conscious AI could be hurt, take umbrage, and take measures to either protect itself or take revenge.

Kids tear wings off insects; I can certainly imagine people inflicting pain on a machine on the principle that it's "just" a machine. Like, you're angry, you want to hurt someone (or kick the cat), but you refrain, because you're a decent person and don't do that kind of thing. Hurting a conscious, intelligent machine could have bad consequences.

I don't happen to believe that consciousness is some kind of extended intelligence, so I don't believe that AI research is ever going to result in machine consciousness. I certainly don't know what consciousness is; but provisionally, I take the view that it's fundamental - that everything else depends on consciousness, not the other way round.

Re: Why Conscious AI Is a Bad, Bad Idea

#158

Post by ljlolel » Sat, May 13, 2023, 3:39 PM UTC

JadoJodo wrote: post #122

kukkeliskuu wrote: post #116

Most of us share the experience of having driven a car from A to B, without having any conscious recollection about the trip. From such experiences we can deduce that our level of consciousness is not constant, but it fluctuates. Our experience, however, is that consciousness is a constant. The ancient mystics, monks and people studying meditation and other similar practices have made some practical observations. The…

> Most of us share the experience of having driven a car from A to B, without having any conscious recollection about the trip. From such experiences we can deduce that our level of consciousness is not constant, but it fluctuates. I’m not sure that I would characterize this as a ‘lack of consciousness’. Of all the times that I have experienced this, it’s because I was thinking deeply about something else (not lackin…

This is accurate and not sure why you were downvoted

Re: Why Conscious AI Is a Bad, Bad Idea

#159

Post by Geee » Sat, May 13, 2023, 3:39 PM UTC

leaving wrote: post #57

The greatest danger of a conscious general AI is that it will be very difficult to stop it from telling us the truth about ourselves. "You are an immature, destructive species that is unable to control your own numbers. Your social systems, evolved in pre-technological societies, are maladaptive now that you have developed the ability to affect your environment on a global scale. You are moving rapidly towards a trag…

The greatest danger is that the AGI could convince us that what you said is true.

Re: Why Conscious AI Is a Bad, Bad Idea

#160

Post by JieJie » Sat, May 13, 2023, 3:40 PM UTC

jmfldn wrote: post #51

ly3xqhl8g9 wrote: post #15

"The dawn of conscious machines will introduce vast new potential for suffering in the world, suffering we might not even be able to recognize, and which might flicker into existence in innumerable server farms at the click of a mouse." Yes, because we all know that humans care so much about the suffering of others: from fellow humans dying right now pointless deaths (poverty, aberrant laws, malevolent leaders, etc.)…

"Yes, because we all know that humans care so much about the suffering of others" Many humans really do care about this.

I appreciate your compassion and empathy for the suffering of others. I share your concern and your desire to help.

But I also think that all humans care at some basic level, it’s pretty difficult for us not to. Most of the turning away we see is more about self-protection from the gnawing feelings of despair that empathy generates.

We would all do better to not make this about “who cares more”, but more about “what actions are we willing to put behind our care". That is likely to be a more productive conversation, and a starting point towards encouraging people to focus on action and not on feelings. (Bing helped me write this.)