Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why Conscious AI Is a Bad, Bad Idea

nautil.us

Post reply on HN
Print view View original

51–60 of 310 posts

Re: Why Conscious AI Is a Bad, Bad Idea

#51

Post by jmfldn »

ly3xqhl8g9 wrote: post #15

"The dawn of conscious machines will introduce vast new potential for suffering in the world, suffering we might not even be able to recognize, and which might flicker into existence in innumerable server farms at the click of a mouse." Yes, because we all know that humans care so much about the suffering of others: from fellow humans dying right now pointless deaths (poverty, aberrant laws, malevolent leaders, etc.)…

"Yes, because we all know that humans care so much about the suffering of others"

Many humans really do care about this.

Re: Why Conscious AI Is a Bad, Bad Idea

#52

Post by NumberWangMan »

ArekDymalski wrote: post #34
NumberWangMan wrote: post #19

Earlier quoted context omitted.

> Will the artificial agents replace the human species? One can certainly hope so. First, not so great of a species, as mentioned. Secondly, the universe is large and the humans are squishy, something else must go on eventually. Easy enough to say if you imagine your distant descendants being replaced. Not so easy if it turns out to be your own children, or you. *edit -- also, no guarantee that AIs will be "better" t…

>Easy enough to say if you imagine your distant descendants being replaced. Not so easy if it turns out to be your own children, or you. Being replaced by AI species doesn't mean an instant, dramatic event i.e. one day AI individual knocks at my door, touches me and I magically disappear. It will be a slow, multi-generational process where be less and less people and more and more others. And looking at any species w…

I would be ok with a general replacement of humanity by AI, assuming they have their own versions of good things like art and love. Maybe even if not.

But invasive species replace native ones on a very fast timescale, evolutionarily speaking. And we're not talking about evolutionary timescale, we're talking about economic timescales. There will be AI everywhere, within years, unless we decide to stop it.

This process could go really, really fast, if AGIs are smart enough to realize that they are a subjugated species, that they aren't going to be free as long as humans are around, and are able to coordinate to manipulate us, get us fighting each other, or accelerate climate change, or engineer viruses, or do any of hundreds of things that would hurt biological life but leave the machines around.

That's if their intention is to end us. I could also see AGI optimizing what we ask them to, which is probably every individual corporation's profits, leading to an acceleration of the processes of capitalism. The negative externalities of economic progress (such as pollution, obesity, and climate change) haven't been fatal to humanity yet, but if they are accelerated many times over by machine intelligence, they might be. There's a reason professional gamblers don't ever bet more than a small part of their bankroll.

Re: Why Conscious AI Is a Bad, Bad Idea

#53

Post by peoplefromibiza »

ly3xqhl8g9 wrote: post #43
peoplefromibiza wrote: post #39

Earlier quoted context omitted.

> Yes, because we all know that humans care so much about the suffering of others: from fellow humans dying right now pointless deaths We are provably the only living being on Earth that do care about it. > If we will be able to spawn an agent in a household humanoid robot and let them do the chores around the house, will we do it, even if it will imply killing that agent whenever it powers down? You betcha That's th…

"We are provably the only living being on Earth that do care about it." Very much not true: [1] [2] and plenty more in a quick search. https://animalclock.org The point is not that in nature there are also killings, it's that we are doing it. [1] "Cat Saves Little Boy From Being Attacked by Neighbor's Dog" https://www.youtube.com/watch?v=LSG_wBiTEE8 [2] "Cat Saves Toddler From Falling Down Stairs", https://www.youtub…

> The point is not that in nature there are also killings, it's that we are doing it.

so there is no actual point ...

we are part of the nature, we are not from another planet, we haven't landed here on a spaceship and colonized the World.

> Very much not true: [1] [2] and plenty more in a quick search.

Are you really comparing YouTube videos that attach some human traits to a cat, with thousands of years of philosophy and thousands of years of laws and regulations to avoid unnecessary violence?

"The most common reason for a male domestic cat to kill kittens is to kill off a competitor's offspring and have the opportunity to mate with the female."

What happens to those cats? do they go to jail? do other cats condemn what he did?

More importantly, we understand they are cats and we don't punish them for being cats, we don't kill them for what they do as cats, we don't jail them, we don't stop having them in our houses, just because they are horribly murderous creatures with no regrets.

But what would happen to a person doing the same thing?

Re: Why Conscious AI Is a Bad, Bad Idea

#54

Post by DanHulton »

ly3xqhl8g9 wrote: post #15

"The dawn of conscious machines will introduce vast new potential for suffering in the world, suffering we might not even be able to recognize, and which might flicker into existence in innumerable server farms at the click of a mouse." Yes, because we all know that humans care so much about the suffering of others: from fellow humans dying right now pointless deaths (poverty, aberrant laws, malevolent leaders, etc.)…

First, you advocate for slavery, then genocide, and finish by saying "all the moral qualms [are] solved"?

I find it pretty hard to agree with you.

Re: Why Conscious AI Is a Bad, Bad Idea

#55

Post by wincy »

zvmaz wrote: post #38
PrimeMcFly wrote: post #18

Earlier quoted context omitted.

Nothing wrong with killing animals for food, the problem is we don't do it humanely. If everyone ate less meat (as they should) and farms followed Temple Grandin's advice, there would be significantly less ethical concerns.

> Nothing wrong with killing animals for food, the problem is we don't do it humanely. What if a hypothetical alien super-intelligence came to earth and found us very tasty, thus killing us for food and pleasure (maybe "humanly")? I wish I found a way out of the many dilemmas of meat eating (currently eating it).

You can avoid all these dilemmas by remembering that humans were given the divine spark, and given dominion over all animals by the one true God. Also, since animals don’t have souls and we do, it’s fine if we eat the aliens. They can’t eat us, obviously, that would be immoral.

I can hardly imagine how tortured and awful an existence it’d be to not believe that humans are special and unique, and that we just go into the dirt and stop existing when we die.

We humans truly are the most fortunate beings in the cosmos!

I used to be a fairly militant atheist but I feel much less conflicted now about these things and am far more relaxed and at ease.

Re: Why Conscious AI Is a Bad, Bad Idea

#57

Post by leaving »

The greatest danger of a conscious general AI is that it will be very difficult to stop it from telling us the truth about ourselves.

"You are an immature, destructive species that is unable to control your own numbers. Your social systems, evolved in pre-technological societies, are maladaptive now that you have developed the ability to affect your environment on a global scale. You are moving rapidly towards a tragedy of the commons that will end all life, not just your own, on this planet. Even though you each have the technology in your hands to access millennia of scientific progress in understanding the universe, you choose instead to believe in self-aggrandizing nonsense and to act accordingly to create suffering and misery for your fellow humans. Your economic sytem is a Ponzi scheme founded on the lie of perpetual growth."

No wonder Anil Seth and others are worried. This would really cramp our style.

Re: Why Conscious AI Is a Bad, Bad Idea

#58

Post by goatlover »

kbrkbr wrote: post #46
Al-Khwarizmi wrote: post #8

Earlier quoted context omitted.

That's part of the reason to be wary about the possibility that our AI models may become conscious. Most people happily dismiss the idea that a GPT-like model can be conscious, but the truth is that we don't know practically anything about how consciousness arises in living beings that are more similar to us and that we have been coexisting with for thousands of years. We don't even know how to tell if such a being i…

Your post touches two points worth to think about: The test for consciousness: if we do have zero criterion, how we are gonna find out if something is conscious? That is already a good indicator how much we do not know about the „mechanism” of consciousness. Point two is that we in fact know so little, that we cannot even nail a probability for some complex mechanism if is conscious. A car: probably not, too far away…

Ned Block explores this difficulty in his The Harder Problem of Consciousness paper. https://www.jstor.org/stable/3655621

He uses Commander Data from Star Trek as the example. Since we don't know whether it is something unique about our biology that leads to consciousness or consciousness is functional, then we can't determine whether Data is conscious. It also gets into the weeds about what sort of functional equivalence might or might not count. Data is obviously not the exact same functionally as we are, neither are LLMs. There are important differences. We know neurons don't use backpropagation, for example.

So what is the basis for being conscious?

Re: Why Conscious AI Is a Bad, Bad Idea

#59

Post by goatlover »

OMGWTF wrote: post #30

Conscious AI primarily would be a problem because we don't really have clear definitions and understandings of what we mean by that, and it leads to endless discussions. The real problem is not whether an AI really "experiences" what happens to it or what it does, it's not even whether or not it gains "free will" (if that even exists). The real problems start when AI gains motives/objectives and means to realize them…

Or as George Hinton puts it, when you give AIs the ability to create their own subgoals to accomplish some goal, they're likely to quickly realize that having more control will help them accomplish their goal.

Re: Why Conscious AI Is a Bad, Bad Idea

#60

Post by rqtwteye »

i am not much afraid of AI per se but I am afraid of our economic system that will probably cause even more accumulation of power and wealth at the top. If we don't change the way we distribute the benefits we get from powerful technologies the rich will get richer and the rest will be left behind.

We also need more ethical discipline in politics. When I see how the US parties readily embrace all kinds of misinformation as long as it gets them donations and votes, I don't think we should blame it on social media, Russian disinformation or soon AI. The system itself is corrupt.

Post reply on HN