Live data from Hacker News

Why Conscious AI Is a Bad, Bad Idea

nautil.us

111–120 of 310 posts

Re: Why Conscious AI Is a Bad, Bad Idea

#111
We are asking the wrong questions based on the theory that humans are conscious and think and feel when in fact all animals and all plants in feel and have some degree of conscious, i.e. feeling, conscious and thinking are not fully humans domain entirely.

What we should be asking is what is our caretaker role in this AI as far as how it fits into human society and that relationship with rest of the inhabitants of the Earth that are not human.

Re: Why Conscious AI Is a Bad, Bad Idea

#113

> We think we’re intelligent, know we’re conscious, and so assume the two go together. Excuse me, I know _I_ am conscious. There is literally nothing you can do to proof that there's anything outside of what I am conscious of. The world could be a very stable illusion, a dream, a simulation and it would all be the same. I don't understand why people think that it matters whether AI is conscious or not. We can't even…

is solipsism wrong, or is it just me?

Re: Why Conscious AI Is a Bad, Bad Idea

#114

Earlier quoted context omitted.

Yes, it's very meditative to do the dishes once a month. The problem is how do you alleviate the pain of the single parent with 2 human children today. And my point is, if I could put together 2 Python packages to make a robotic arm able to listen to that parent's commands and do the dishes, clean after the kids, and so on, I would do it as fast as you can type "zen", even if the Python packages in cause would "feel"…

Would you? It seems like you're leaning heavily on the idea that the python packages don't have the same moral weight as the single parent, which is an odd thing to believe if the python packages can actually have experiences. The reason why we don't worry about python packages is exactly that we _don't_ think that, surely

Just an example of course, but we can imagine some `from consciousness import *` line in some undefined future.

Re: Why Conscious AI Is a Bad, Bad Idea

#115
post #4

> Our minds haven’t evolved to deal with machines we believe have consciousness. Neither has evolved to drive cars, fly planes, or build space rockets. If you start proving a point based on what evolution has done, your reasoning is flawed.

Not a bad point, but cars, planes, and space rockets are specifically designed for us humans. We have crafted our entire environment for ourselves (building, cities, governments, etc). AI systems will be no different, AI models will be trained to be human-centric. LLMs are trained on human languages, OCR is trained on human text, facial recognition is trained on human faces, it's all human-centric. We don't need to evolve, the AI will "evolve" for us.

Re: Why Conscious AI Is a Bad, Bad Idea

#116
Most of us share the experience of having driven a car from A to B, without having any conscious recollection about the trip. From such experiences we can deduce that our level of consciousness is not constant, but it fluctuates.

Our experience, however, is that consciousness is a constant. The ancient mystics, monks and people studying meditation and other similar practices have made some practical observations. These indicate that such lack of consciousness is actually quite common, we are just not very aware of it. It is also possible to experimentally understand this, although it is probably not possible to prove it scientifically.

In other words, we do not possess meta-consciousness, consciousness about our consciousness. We just fill in the blanks, believe that we have been conscious. To become conscious more often and more deeply, you need to be able to control your attention and not lose it. The modern society does not help you in learning to control your attention, more to the contrary. So we are probably even less conscious than common people at the time that the ancient mystics lived.

We should be less concerned about the consciousness of the AI, and more concerned about increasing our own consciousness.

Re: Why Conscious AI Is a Bad, Bad Idea

#117

Earlier quoted context omitted.

it is playing into written tropes about ai. when you play the role of this kind of questioner, the llm role plays the other side. it's not only the content of what you ask - it's text prediction

Would an AI argue any differently?

it's just a warning to mind leading your questions, not philosophizing

Re: Why Conscious AI Is a Bad, Bad Idea

#118

> We think we’re intelligent, know we’re conscious, and so assume the two go together. Excuse me, I know _I_ am conscious. There is literally nothing you can do to proof that there's anything outside of what I am conscious of. The world could be a very stable illusion, a dream, a simulation and it would all be the same. I don't understand why people think that it matters whether AI is conscious or not. We can't even…

> All that matter is we (I), feel/think that it is conscious, or deserving of care and respect.

It does matter, I can agree, but it doesn't the all that matter. People can be mistaken, and they can disagree sometimes. Suppose, you cared for a particular AI and I didn't, should we cut the power from the machine running an AI? I can complicate it a little if you wish, adding a painful death of a kitten which would happen if we didn't turn power down.

We need some objective means to measure what is conscious. We have a heuristic for people: "human life is sacred, full stop". There are some corner cases when it doesn't work well (like euthanasia), but we are used to it. There are other heuristics we are generally agree of, like we care more about kittens than of grass, and more of grass than of amoeba.

With AI we'll face more of that and we have no idea where to place it among our heuristics. People did bad already, like treating black people as non-human. It would be a shame to repeat those mistakes without making an attempt to do better this time.

> I think what you are afraid of here is agency, which is something that might be dangerous to endow a super intelligent being with.

I cannot vouch for others, but I do not particularly afraid of that. I do not fear much of losing it all to a super intelligent and a conscious being, it would be a great achievement for a humanity, which would fit nicely with all this evolution business. It is a paperclip scenario I do not like much.

Re: Why Conscious AI Is a Bad, Bad Idea

#119

> We think we’re intelligent, know we’re conscious, and so assume the two go together. Excuse me, I know _I_ am conscious. There is literally nothing you can do to proof that there's anything outside of what I am conscious of. The world could be a very stable illusion, a dream, a simulation and it would all be the same. I don't understand why people think that it matters whether AI is conscious or not. We can't even…

is solipsism wrong, or is it just me?

I am really curious to know why? And don't invoke "what about your kids, mothers, fathers?". "Solipsism" (first time I have heard of this, so thank you) does not make me feel any less worse when something happens to them.

Re: Why Conscious AI Is a Bad, Bad Idea

#120
I am reminded of the Truman/Oppenheimer story:

Oppenheimer when he went into Truman's Office with Dean Acheson said to the latter, wringing his hands:"I have blood on my hands". Truman later said to Acheson: "Never bring that fucking cretin in here again. He didn't drop the bomb. I did. That kind of weepiness makes me sick."

Post reply on HN