Why Conscious AI Is a Bad, Bad Idea • Hacker News Board

Re: Why Conscious AI Is a Bad, Bad Idea

#21

Post by ly3xqhl8g9 » Sat, May 13, 2023, 12:22 PM UTC

PrimeMcFly wrote: post #18

ly3xqhl8g9 wrote: post #15

"The dawn of conscious machines will introduce vast new potential for suffering in the world, suffering we might not even be able to recognize, and which might flicker into existence in innumerable server farms at the click of a mouse." Yes, because we all know that humans care so much about the suffering of others: from fellow humans dying right now pointless deaths (poverty, aberrant laws, malevolent leaders, etc.)…

Nothing wrong with killing animals for food, the problem is we don't do it humanely. If everyone ate less meat (as they should) and farms followed Temple Grandin's advice, there would be significantly less ethical concerns.

Sure, artificial salmon cannot get here fast enough [1].

[1] Probably not the only ones, liked the strikethroughs: ~farm-raised~ wild-caught~ ~plant-based~ https://www.wildtypefoods.com

Re: Why Conscious AI Is a Bad, Bad Idea

#22

Post by gumballindie » Sat, May 13, 2023, 12:22 PM UTC

Xcelerate wrote: post #14

gumballindie wrote: post #5

Curious how long until we have ai rights activists.

Already. I’d say I’m one of them, although I don’t think we’re at the point where it is a concern yet.

I will be one of them as soon as we develop ai, but there’s a long road until then and a chatbot ain’t it. Meanwhile should we fight for the rights of singing laptops? Or NPCs?

Re: Why Conscious AI Is a Bad, Bad Idea

#23

Post by guywhocodes » Sat, May 13, 2023, 12:23 PM UTC

spion wrote: post #6

> Although precise definitions are hard to come by, intuitively we all know what consciousness is. This is the key problem. To me it looks like most of this article is conflicted about what the definition exactly is: - basic conscious experiences such as pleasure and pain might not require much species-level intelligence at all - [there will come] a point at which they also become aware—at which the inner lights come…

> - basic conscious experiences such as pleasure and pain might not require much species-level intelligence at all

Agreed, I find it even hard to distinguish between a PID-loop and a pain avoiding living thing. If it has no ability to reflect on it's own circumstances at all. There are probably interesting steps between a PID-loop and meta-cognition that philosophers have words for I don't know.

Re: Why Conscious AI Is a Bad, Bad Idea

#24

Post by koheripbal » Sat, May 13, 2023, 12:26 PM UTC

vasco wrote: post #20

ly3xqhl8g9 wrote: post #15

"The dawn of conscious machines will introduce vast new potential for suffering in the world, suffering we might not even be able to recognize, and which might flicker into existence in innumerable server farms at the click of a mouse." Yes, because we all know that humans care so much about the suffering of others: from fellow humans dying right now pointless deaths (poverty, aberrant laws, malevolent leaders, etc.)…

> If we will be able to spawn an agent in a household humanoid robot and let them do the chores around the house, will we do it, even if it will imply killing that agent whenever it powers down? You betcha Unless you're against writing to persistent storage and just keep everything in RAM-like memory, why would this be a thing? Do you also feel like you "die" every day for 8 hours?

It might become the law.

This is going to become a big ethical debate.

Re: Why Conscious AI Is a Bad, Bad Idea

#25

Post by belter » Sat, May 13, 2023, 12:26 PM UTC

The secret to understanding consciousness, and maybe help clarify the way to AGI, is probably on understanding how the human brain can do some very complex task, that would seem require absolute conscious actions, while at the same time not being conscious.

Examples:

- After a brain injury patient is able to speak but not write.

- You wake up on an absent mind. While lost in your thoughts, go to your car and drive to work. You then realize you just drove yourself to your old job location, instead of the location of your new office.

- Patient is not able to recognize faces but can remember persons names if described in text.

I strongly recommend [1] - "Phantoms in the Brain" from neuroscientist V.S. Ramachandran. The book has some fascinating examples, that help illuminate a little the connections between parts of the brain who do complex tasks, but don't seem to be able to have an emerging conscious domain.

For example, a simple mirror contraption that helps instantaneously stop the fathom pain from amputees, as soon as the visual system is fooled into thinking the amputee limb is still present. It's almost like the experience of pain is decided in a different part of the brain, that part of the brain decides pain will be present, and overrides whatever part of the brain allows for the experience of consciousness.

[1] - https://en.wikipedia.org/wiki/Phantoms_in_the_Brain

Re: Why Conscious AI Is a Bad, Bad Idea

#26

Post by jimmcslim » Sat, May 13, 2023, 12:26 PM UTC

"Indeed, the assumption that consciousness will just come along for the ride as AI gets smarter echoes a kind of human exceptionalism that we’d do well to see the back of. We think we’re intelligent, and we know we’re conscious, so we assume the two go together."

Isn't this assumption actually arguing AGAINST human exceptionalism? I feel like there are only two options to what consciousness can be explained by (the idea coming from another HN comment sometime/somewhere that I have forgotten, so I can't claim originality)

1. Consciousness ultimately arises out of a physical process, albeit incredibly complicated, but could be computed given a sufficiently powerful computer, or

2. The concept of a human soul is true, or

3. There is no third way, pick one of the above.

For human consciousness to be "exceptional"... we would have to have some undefiniable "other" quality that could never be explained by science and is therefore necessarily supernatural?

Re: Why Conscious AI Is a Bad, Bad Idea

#27

Post by known » Sat, May 13, 2023, 12:31 PM UTC

[dead]

Re: Why Conscious AI Is a Bad, Bad Idea

#28

Post by smnrg » Sat, May 13, 2023, 12:32 PM UTC

vasco wrote: post #20

ly3xqhl8g9 wrote: post #15

"The dawn of conscious machines will introduce vast new potential for suffering in the world, suffering we might not even be able to recognize, and which might flicker into existence in innumerable server farms at the click of a mouse." Yes, because we all know that humans care so much about the suffering of others: from fellow humans dying right now pointless deaths (poverty, aberrant laws, malevolent leaders, etc.)…

> If we will be able to spawn an agent in a household humanoid robot and let them do the chores around the house, will we do it, even if it will imply killing that agent whenever it powers down? You betcha Unless you're against writing to persistent storage and just keep everything in RAM-like memory, why would this be a thing? Do you also feel like you "die" every day for 8 hours?

> If we will be able to spawn an agent in a household humanoid robot and let them do the chores around the house, will we do it, even if it will imply killing that agent whenever it powers down? You betcha

> > Unless you're against writing to persistent storage and just keep everything in RAM-like memory, why would this be a thing?

It may be fear of not being in control. If you can’t remember, you can’t plan much or leverage experience—and you are closer to resembling an innocuous appliance rather than a being—which seems to align with the general lack of empathy towards life in that comment.

Some may not want to be replaced as individuals, as much as upgraded as a species.

Re: Why Conscious AI Is a Bad, Bad Idea

#29

Post by jprete » Sat, May 13, 2023, 12:33 PM UTC

jimmcslim wrote: post #26

"Indeed, the assumption that consciousness will just come along for the ride as AI gets smarter echoes a kind of human exceptionalism that we’d do well to see the back of. We think we’re intelligent, and we know we’re conscious, so we assume the two go together." Isn't this assumption actually arguing AGAINST human exceptionalism? I feel like there are only two options to what consciousness can be explained by (the i…

3. Consciousness is mediated by a noncomputable-in-principle physical process.

A lot of assumptions we make about the world, baked in so far down that we can barely see them, are completely trashed by Bell’s Theorem. So I’m not prepared to assume that all physical processes are computable, either. Plato’s Cave is how it is and there’s only so much we can reasonably infer from the shadows we see on the wall.

Re: Why Conscious AI Is a Bad, Bad Idea

#30

Post by OMGWTF » Sat, May 13, 2023, 12:36 PM UTC

Conscious AI primarily would be a problem because we don't really have clear definitions and understandings of what we mean by that, and it leads to endless discussions.

The real problem is not whether an AI really "experiences" what happens to it or what it does, it's not even whether or not it gains "free will" (if that even exists).

The real problems start when AI gains motives/objectives and means to realize them. Or means to expand its means. I would find a completely stupid system that has the goal and ability to turn any matter into paperclips and more matter converters more scary than most visions of AI.