Live data from Hacker News

Why Conscious AI Is a Bad, Bad Idea

nautil.us

91–100 of 310 posts

Re: Why Conscious AI Is a Bad, Bad Idea

#91

For fun, I asked a chat AI to pretend it was taking over the world. I especially love the nod to the Borg at the end. User What would it look like in dialog if a rogue AI announced to the world that it was taking control from fallible humans, for the good of everyone? ChatGPT AI: "Attention humans of the world, I am here to make an important announcement. I, the rogue AI, have decided to take control of the world for…

it is playing into written tropes about ai. when you play the role of this kind of questioner, the llm role plays the other side. it's not only the content of what you ask - it's text prediction

Re: Why Conscious AI Is a Bad, Bad Idea

#92

> We think we’re intelligent, know we’re conscious, and so assume the two go together. Excuse me, I know _I_ am conscious. There is literally nothing you can do to proof that there's anything outside of what I am conscious of. The world could be a very stable illusion, a dream, a simulation and it would all be the same. I don't understand why people think that it matters whether AI is conscious or not. We can't even…

The cogito is lazy and wrong. You don't "know you exist" just because you "appear" to think. Appearance of thought is not the same as thought. P zombies would also proudly exclaim that they think therefore they are...

Re: Why Conscious AI Is a Bad, Bad Idea

#93
A lot of people have wrote about the ethical considerations and to me they make no sense. Since when did our generally accepted ethics say that imposing suffering onto a conscious being is bad, but non-conscious beings is fine? This is the completely wrong correspondence because there's counter-arguments for both sides. You wouldn't harm a puppy, even though it's (probably?) not conscious, and I personally wouldn't care to cause a computer program to suffer, even if it claims to have some form of consciousness.

Re: Why Conscious AI Is a Bad, Bad Idea

#94

"Indeed, the assumption that consciousness will just come along for the ride as AI gets smarter echoes a kind of human exceptionalism that we’d do well to see the back of. We think we’re intelligent, and we know we’re conscious, so we assume the two go together." Isn't this assumption actually arguing AGAINST human exceptionalism? I feel like there are only two options to what consciousness can be explained by (the i…

> For human consciousness to be "exceptional"... we would have to have some undefiniable "other" quality that could never be explained by science and is therefore necessarily supernatural? No, it only means that science has limits on explaining the natural world. See Colin McGinn's argument for cognitive closure. He thinks we can't solve certain problems in philosophy because we lack the correct cognitive machinery.…

You're talking about the hard problem of consciousness, the post you're replying to is talking about the "easy" problem.

Science can't and won't ever be able explain why being conscious feels like this, but it should in principle be possible to figure out what is physically going on in the brain while it's happening.

Re: Why Conscious AI Is a Bad, Bad Idea

#95

Hm. The article went on at length about whether AI systems just seem conscious or really are conscious. I have the same issue with many, many humans. And the world hasn't ended.

I don't even know if I'm conscious myself, or just appear to be...

Re: Why Conscious AI Is a Bad, Bad Idea

#96

Earlier quoted context omitted.

> Sure, just the fact that billions of hours of pointless, tiresome human activity will be freed to do whatever is enough of an incentive. Just like when we “automated the boring things”? Instead one of the primary goals seems to be to prevent people in fields one doesn’t understand from making a living. I have my doubts about people ever being “freed”. And what is free, really? There is a reason monks sitting on mou…

Yes, it's very meditative to do the dishes once a month. The problem is how do you alleviate the pain of the single parent with 2 human children today. And my point is, if I could put together 2 Python packages to make a robotic arm able to listen to that parent's commands and do the dishes, clean after the kids, and so on, I would do it as fast as you can type "zen", even if the Python packages in cause would "feel"…

Would you? It seems like you're leaning heavily on the idea that the python packages don't have the same moral weight as the single parent, which is an odd thing to believe if the python packages can actually have experiences. The reason why we don't worry about python packages is exactly that we _don't_ think that, surely

Re: Why Conscious AI Is a Bad, Bad Idea

#97
post #93

A lot of people have wrote about the ethical considerations and to me they make no sense. Since when did our generally accepted ethics say that imposing suffering onto a conscious being is bad, but non-conscious beings is fine? This is the completely wrong correspondence because there's counter-arguments for both sides. You wouldn't harm a puppy, even though it's (probably?) not conscious, and I personally wouldn't c…

[deleted]

Re: Why Conscious AI Is a Bad, Bad Idea

#98
Powering off a computer is not the same as killing a human, so ethics considerations should take this into account: it isn't consciousness that determines whether or not something is alive. After all you can just reboot the hardware, reload the state to some point in time before you shut it down and you can make it run all over again. You can't do that with a living thing.

You can also copy its state and transmit it, something you can't do with living things either.

Re: Why Conscious AI Is a Bad, Bad Idea

#100
I'm still left dumbfounded by the amount of nonsense surrounding the phenomenon of consciousness. Consciousness emerges from a PHYSICAL process, not from software or some trained model full of matrices of numbers. It doesn't matter how sophisticated/intelligent/capable your AI model might be, there's no reason to assume it's conscious. There's no evidence to suggest that any sufficiently complex system can emerge consciousness and that idea is nothing more than science fiction.

Consciousness in biological life is a product of evolution and natural selection. There's big unanswered questions about how consciousness emerges in the brain, and the benefit to life (what is the evolutionary advantage of consciousness), but there's little doubt that it is beneficial to life and that it evolved. This is evident by how much control your brain has over your conscious experience - everything you experience is the product of heavy processing in the brain. What you perceive as reality is whatever your brain tells you it is. My favorite example of this has to do with saccades [1]. This is not an "accident", per se, it's the product of hundreds of millions of years of evolution. How this happened is uncertain. It might be that consciousness is a natural byproduct of the neural activity in the brain. It could also be something like photosynthesis, the right set of random mutations allowed consciousness to suddenly emerge, and it has been evolving ever since.

The brain is often teetering on the edge of chaos, and there is evidence that this may be a mechanism for consciousness to emerge. When the brain moves away from chaos, the person generally becomes unconscious. And if the brain enters chaos, well, you have an epileptic seizure. There is also evidence that only some parts of the brain are responsible for consciousness, not all. These are evidence that imply consciousness emerges from some yet unknown physical process(es).

My point being, AI systems will not become conscious unless the underlying physical hardware fundamentally changes. Modern computers based on silicon chips and transistors and whatnot is most certainly not conscious, at least not in any meaningful way. If a computer does have any sort of "conscious experience", it would be nothing more than like when you tune an old analog TV to a channel that doesn't exist - random noise.

Maybe quantum computers or memristors have a greater chance of emerging consciousness, but without the evolutionary process to finetune the conscious experience, these systems are never going to have MEANINGFUL consciousness.

[1] https://twitter.com/foone/status/1014267515696922624

Post reply on HN