Why Conscious AI Is a Bad, Bad Idea • Hacker News Board

Re: Why Conscious AI Is a Bad, Bad Idea

#261

Post by ordu » Sun, May 14, 2023, 6:18 AM UTC

AnthonyMouse wrote: post #164

ordu wrote: post #118

Earlier quoted context omitted.

> All that matter is we (I), feel/think that it is conscious, or deserving of care and respect. It does matter, I can agree, but it doesn't the all that matter. People can be mistaken, and they can disagree sometimes. Suppose, you cared for a particular AI and I didn't, should we cut the power from the machine running an AI? I can complicate it a little if you wish, adding a painful death of a kitten which would happ…

One of the fundamental differences between human life and silicon-based AI is that biological organisms can't recover from a system shutdown. If you suffer heart failure or go without air for an hour or starve to death, bacteria start to eat your brain and you're irreversibly destroyed. If you cut power to an AI and then come back in a year, it's all still there. It's not a death, it's sleep mode. It also doesn't mea…

I'm trying to say, that our means to keep a moral high ground are subjective and based on heuristics. It seems to me that you do not notice, let me show you.

> If you cut power to an AI and then come back in a year, it's all still there.

Does AI's current state stored in volatile memory doesn't matter? Or it does? Should we avoid turning off only those AIs which store they weights in DRAM?

> Most of what causes us to have sympathy for living things or treat them with compassion just doesn't apply.

I believe slaves didn't trigger sympathy in slave-owners, but it doesn't stop us from believing that slavery is bad. I admire that you at not like this, and your sympathy extents to all living things, but it is your subjective way to decide what is moral and what is not. Other people may feel differently, what should they do to be not less highly moral than you? Or can you and I become even better and to hold even higher moral standards?

> If you do the equivalent to a computer program and the result is undesirable, it can roll back to a previous snapshot.

If we could roll back people to a previous snapshots after burning them to ashes on a bonfire, would it be ok to burn people and then restore them?

Questions like this may be impractical (because we cannot restore a human burned to ashes), but our hesitation to answer shows the limitations of our ways to think about such problems.

Humanity can benefit a lot from an objective way to deal with moral dilemmas, and based not on heuristics but on universal laws, like a physics does. It may help people to understand each other and to find ways to live together without fighting. I'm not sure that moral can be objective and based on a universal law, but it is not a reason to stop thinking. When you think about it, you find new corner-cases and specific solutions to them. At least it makes your heuristics better.

Re: Why Conscious AI Is a Bad, Bad Idea

#262

Post by anonymouskimmer » Sun, May 14, 2023, 7:12 AM UTC

PrimeMcFly wrote: post #258

haswell wrote: post #153

Earlier quoted context omitted.

One definition or property of consciousness I find interesting is that it encompasses “what it is like to be ”. If there is something that it is like to be , it is conscious. Intelligence and awareness are separate from consciousness in this framing. And there is arguably a point where we can speculate that extremely simplistic organisms could not be conscious, given the lack of sense organs that seems to predicate e…

> One definition or property of consciousness I find interesting is that it encompasses “what it is like to be ”. If there is something that it is like to be , it is conscious. I think there are problems with this. Take something like the roundworm, with 300 neurons. It has senses, and I assume a base level consciousness to process those senses. It would be something to be like a roundworm. On the other hand, a round…

> On the other hand, a roundworm likely has no sense of self, no identity, no awareness..no mental self to be able to acknowledge and reflect on those experience, so there is no 'someone'. And isn't that the point that matters?

We can always point to something we have that other organisms don't have and ask whether that is the point that matters. We can hypothesize things that probably don't exist and say that those things are what matters (e.g. the posited purpose of reincarnation, if such a thing existed).

Re: roundworm

1) sense of self -> it has to have a sense of proprioception, of embodiedment, in order to properly respond to its sensory information.

2) identity -> humans with global amnesia would lack a memory of identity.

3) a roundworm is sensorily aware, and, if it has a sense of proprioception, bodily aware of it's own actions and environmental responses to its actions.

4) It probably isn't self-reflective, but is this necessary in humans?

Re: Why Conscious AI Is a Bad, Bad Idea

#263

Post by ly3xqhl8g9 » Sun, May 14, 2023, 7:23 AM UTC

layer8 wrote: post #191

blibble wrote: post #167

Earlier quoted context omitted.

> Unlike humans, they are are oblivious of the fact that they will die someday (or be slaughtered someday). how do you know?

Personally I don’t strictly know (hence “seem”), but it seems plausible to me, and if animals did regularly mentally time travel like humans do, there should be more evidence of it.

Apparently there are at least five animals orders that do grieve: elephants, monkeys, dolphins, giraffes, dogs. It is imaginable that one of them might infer: if they died, maybe I will also die someday.

[1] https://www.harpercollins.com/blogs/harperkids/the-five-anim...

Re: Why Conscious AI Is a Bad, Bad Idea

#264

Post by ly3xqhl8g9 » Sun, May 14, 2023, 7:28 AM UTC

PrimeMcFly wrote: post #256

ly3xqhl8g9 wrote: post #252

Earlier quoted context omitted.

The bluestreak cleaner wrasse [1-3] runs back and forth comparing the size of itself from an image with potential threats to know if it must fight or flight. Octopuses are famously smart [4]. Any intelligence or self-awareness test is also a test on the tester: how good are they in detecting what matters for the biomachine [5]/animal in testing. [1] https://en.wikipedia.org/wiki/Cleaner_fish [2] video: https://www.yo…

I'm not saying no animals have self-awareness, indeed many do. Not just octopuses, also elephants, corvids, dogs, chimps, etc. The animals we commonly eat (chicken, cow, turkey, salmon, cod, etc) though don't seem to meet the criteria.

Right, I'm just saying we need a new framework (metaphysical, ethical, perhaps even socio-political if synthetic agents will be part of our society), beyond dichotomies such as natural/artificial, evolved/designed, organic/machine. Not sure how it should/would look like, best so far I could find is the concept of cognitive light cones [1] and the "Technological Approach to Mind Everywhere: An Experimentally-Grounded Framework for Understanding Diverse Bodies and Minds" [2].

[1] https://www.youtube.com/watch?v=YnObwxJZpZc

[2] https://www.frontiersin.org/articles/10.3389/fnsys.2022.7682...

Re: Why Conscious AI Is a Bad, Bad Idea

#265

Post by tgv » Sun, May 14, 2023, 7:38 AM UTC

arp242 wrote: post #254

tgv wrote: post #83

Earlier quoted context omitted.

You call it badly written, but assert that "we’d immediately know how to interact with it as long as we could communicate"? But not being able to give a definition of consciousness doesn't matter . We all sort of know what it is, and it's not good for a computer to have it.

Is a cat conscious? I would imagine that if you'd ask 100 people you'll get a lot of different answers. I'm not sure that "we all sort of know what it is" really cuts it here, even if only to make sure we're all talking about the same thing instead of talking past each other.

It doesn't matter whether it really cuts it or not. Do you want to run some unknown, but potentially large risk just because you can't precisely define what it is? Good luck arguing against climate change.

Re: Why Conscious AI Is a Bad, Bad Idea

#266

Post by PrimeMcFly » Sun, May 14, 2023, 9:14 AM UTC

anonymouskimmer wrote: post #262

PrimeMcFly wrote: post #258

Earlier quoted context omitted.

> One definition or property of consciousness I find interesting is that it encompasses “what it is like to be ”. If there is something that it is like to be , it is conscious. I think there are problems with this. Take something like the roundworm, with 300 neurons. It has senses, and I assume a base level consciousness to process those senses. It would be something to be like a roundworm. On the other hand, a round…

> On the other hand, a roundworm likely has no sense of self, no identity, no awareness..no mental self to be able to acknowledge and reflect on those experience, so there is no 'someone'. And isn't that the point that matters? We can always point to something we have that other organisms don't have and ask whether that is the point that matters. We can hypothesize things that probably don't exist and say that those…

> We can always point to something we have that other organisms don't have and ask whether that is the point that matters.

Sure. In this case it's introspective self-awareness, and I would say it matters very much.

> sense of self -> it has to have a sense of proprioception, of embodiedment, in order to properly respond to its sensory information.

Sure, it has bodily self-awareness. It has enough awareness to react to something its body detects. ut not enough of a mind to reflect on or appreciate on an experience. There is no 'someone'. It's automata.

> identity -> humans with global amnesia would lack a memory of identity.

Sure, but we don't use outliers to establish baselines.

> It probably isn't self-reflective, but is this necessary in humans?

For establishing a baseline? Absolutely.

Re: Why Conscious AI Is a Bad, Bad Idea

#267

Post by kukkeliskuu » Sun, May 14, 2023, 10:41 AM UTC

taneq wrote: post #163

kukkeliskuu wrote: post #116

Most of us share the experience of having driven a car from A to B, without having any conscious recollection about the trip. From such experiences we can deduce that our level of consciousness is not constant, but it fluctuates. Our experience, however, is that consciousness is a constant. The ancient mystics, monks and people studying meditation and other similar practices have made some practical observations. The…

Our conscious experience is that we're conscious every time we're conscious of it. This is probably because we're not conscious of the times we're not conscious. :P That said I'm not sure that being actively conscious should really be a goal. Does it make us happier? More effective? For a lot of things I find myself trying to avoid becoming self-conscious because it interferes with my ability to do the thing I'm tryi…

Yes, this kind of experience exists. However, in my experience, the "self-consciousness" is actually not the same "consciousness" I am talking about, but an experience related to a stage in learning. When you become a little bit aware that you cannot do something, it might temporarily make you anxious. Then, to overcome this anxiousness, you try to use your analytical mind to control your body (in case we are talking about some physical action), and it does not work really well, so you become even more anxious, creating a vicious loop.

Analytical mind is actually not very good at micro-managing the body. You can test this by trying to do something very simple, such as lifting a spoon, and doing it with your analytical mind and without interference from your analytical mind. If you really try this experiment (thinking about it is not enough, you really need to do it yourself), you can learn about the limitations of your analytical mind by yourself.

Your experience can also be expressed differently. What you have experienced as consciousness is actually your analytical mind, and by "avoiding becoming self-conscious" you are trying to let your body (or some other part of you, depending on what you try to do) to do it's thing without micro-management from the analytical mind.

The book "Inner game of tennis" explains how olympic-level athletes (in this case tennis players) need to learn to let go of this control by the analytical mind in their practice. When you can let go, and you are aware of it, the end result is an experience of flow.

For me, being in the flow state feels good, and I can do things that would not be possible otherwise (in my case in dancing). It is something I aim to experience more in my life.

Does this make us efficient? You cannot talk about efficiency separate from your aim. For example, it may be time-efficient to eat all your meals at McDonald's, but the long-term consequences of that choice could make you less than efficient on some other aim you may have.

Happiness is a difficult concept as well. I have experienced more happiness through increase of flow in my life. I would say that experiencing more flow is one of the ways to happiness. But depending on your definition of happiness, you may disagree with this.

Re: Why Conscious AI Is a Bad, Bad Idea

#268

Post by hnfong » Sun, May 14, 2023, 10:58 AM UTC

cubefox wrote: post #173

kajaktum wrote: post #161

Earlier quoted context omitted.

> The fact that other humans are so similar to me (all humans are similar in virtue of being human organisms) allows for an analogical inference: You guys are similar to me, I am conscious, therefore you are probably also conscious. That hinges on the fact that the brain, or human-like body, is related to the existence of consciousness. But there's no reason to believe that a bunch of electrical pulses as the source…

I am both consciousness and have a brain, and alterations in the brain (chemical or physical) correlate with changes in consciousness. Which suggests that our "bunch of electrical pulses" are at least sufficient for consciousness. Plants may be also be conscious, but that would require the existence of an additional, very different, cause of consciousness than in humans. The theory "X is only explained by Y" is simpl…

> Which suggests that our "bunch of electrical pulses" are at least sufficient for consciousness.

I think it only suggests necessary conditions. Example: whether you are breathing is also connected to your state of consciousness, but it doesn’t follow that it’s sufficient.

not to mention the stories about people reporting seeing strange things during the Near Death Experiences, often their brain is not quite working at the time.

Re: Why Conscious AI Is a Bad, Bad Idea

#269

Post by hnfong » Sun, May 14, 2023, 10:59 AM UTC

PrimeMcFly wrote: post #236

kajaktum wrote: post #161

Earlier quoted context omitted.

> The fact that other humans are so similar to me (all humans are similar in virtue of being human organisms) allows for an analogical inference: You guys are similar to me, I am conscious, therefore you are probably also conscious. That hinges on the fact that the brain, or human-like body, is related to the existence of consciousness. But there's no reason to believe that a bunch of electrical pulses as the source…

> That hinges on the fact that the brain, or human-like body, is related to the existence of consciousness. That's what the evidence indicates. Anything else is mystical woo-woo that can be disregarded.

As long as you disregard evidence that does not conform with your worldview, all evidence you see support whatever you believe.

Re: Why Conscious AI Is a Bad, Bad Idea

#270

Post by kukkeliskuu » Sun, May 14, 2023, 11:13 AM UTC

JadoJodo wrote: post #122

kukkeliskuu wrote: post #116

Most of us share the experience of having driven a car from A to B, without having any conscious recollection about the trip. From such experiences we can deduce that our level of consciousness is not constant, but it fluctuates. Our experience, however, is that consciousness is a constant. The ancient mystics, monks and people studying meditation and other similar practices have made some practical observations. The…

> Most of us share the experience of having driven a car from A to B, without having any conscious recollection about the trip. From such experiences we can deduce that our level of consciousness is not constant, but it fluctuates. I’m not sure that I would characterize this as a ‘lack of consciousness’. Of all the times that I have experienced this, it’s because I was thinking deeply about something else (not lackin…

Firstly, this is a very very difficult topic. Some ancients books have tried to communicate the same ideas I am trying to convey here, but have been less than succesful. There appears to be deep psychological reasons why these things are difficult to grasp. So I hope you will forgive me if I cannot fully convince you about my experiences.

Actually, I am not trying to prove anything to you (or others) or say how people should think about consciousness. I am just sharing my experiences and what I have found based on them, so others can try out and see for themselves, so perhaps we could learn together more about these.

Neuroscientist Rodolfo Llinas did an interesting experiment on himself, and explained it in an interview on Science Network about his book "I of the vortex". He was able to consistently turn his foot inwards by placing a strong magnet at certain position on his head. Then he decided he would move his foot outwards when the magnet was applied on the same spot. When he did this, his foot still moved inwards. His experience was that "I changed my mind". He was able to repeat this experiment many times, and each time he felt he had changed his mind. So he could not distinguish between really changing his mind and his body doing something and his mind making up a post-hoc rationalization for his body doing it.

This story, as well as my experience of trying to understand my consciousness experimentally has led me to become very suspicious about my own memory and thoughts about my conscious experiences. And consequentially, about other people's thoughts and memory about their conscious experiences as well.

So my question to you is, how would you know that 100% of the times you have had a similar experience, you were 100% of the time 100% focused in deeply thinking about something?