Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why Conscious AI Is a Bad, Bad Idea

nautil.us

Post reply on HN
Print view View original

161–170 of 310 posts

Re: Why Conscious AI Is a Bad, Bad Idea

#161

Post by kajaktum »

cubefox wrote: post #135
kajaktum wrote: post #79

> We think we’re intelligent, know we’re conscious, and so assume the two go together. Excuse me, I know _I_ am conscious. There is literally nothing you can do to proof that there's anything outside of what I am conscious of. The world could be a very stable illusion, a dream, a simulation and it would all be the same. I don't understand why people think that it matters whether AI is conscious or not. We can't even…

The fact that other humans are so similar to me (all humans are similar in virtue of being human organisms) allows for an analogical inference: You guys are similar to me, I am conscious, therefore you are probably also conscious. And whether or not something is conscious matters a lot ethically. If something is conscious, it may suffer. Conscious and unconscious objects have to be treated in a completely different w…

> The fact that other humans are so similar to me (all humans are similar in virtue of being human organisms) allows for an analogical inference: You guys are similar to me, I am conscious, therefore you are probably also conscious.

That hinges on the fact that the brain, or human-like body, is related to the existence of consciousness. But there's no reason to believe that a bunch of electrical pulses as the source of consciousness. For example, had "I" existed as a plant, then it would be reasonable for me to think that other plants are also conscious and that plants are what give rise to consciousness.

Re: Why Conscious AI Is a Bad, Bad Idea

#162

Post by bazmattaz »

rTX5CMRXIfFG wrote: post #56

Nobody, both in the comments and in the article, seems to have an answer for why we need AI to be conscious in the first place. Everyone’s only concerned about how well it does a job. Why do the machines have to be conscious?

They probably don’t but it’s the holy grail of AI I suppose. Every research team want to say that they were the first to build a truly conscious AI.

Re: Why Conscious AI Is a Bad, Bad Idea

#163

Post by taneq »

kukkeliskuu wrote: post #116

Most of us share the experience of having driven a car from A to B, without having any conscious recollection about the trip. From such experiences we can deduce that our level of consciousness is not constant, but it fluctuates. Our experience, however, is that consciousness is a constant. The ancient mystics, monks and people studying meditation and other similar practices have made some practical observations. The…

Our conscious experience is that we're conscious every time we're conscious of it. This is probably because we're not conscious of the times we're not conscious. :P

That said I'm not sure that being actively conscious should really be a goal. Does it make us happier? More effective? For a lot of things I find myself trying to avoid becoming self-conscious because it interferes with my ability to do the thing I'm trying to do.

Re: Why Conscious AI Is a Bad, Bad Idea

#164

Post by AnthonyMouse »

ordu wrote: post #118
kajaktum wrote: post #79

> We think we’re intelligent, know we’re conscious, and so assume the two go together. Excuse me, I know _I_ am conscious. There is literally nothing you can do to proof that there's anything outside of what I am conscious of. The world could be a very stable illusion, a dream, a simulation and it would all be the same. I don't understand why people think that it matters whether AI is conscious or not. We can't even…

> All that matter is we (I), feel/think that it is conscious, or deserving of care and respect. It does matter, I can agree, but it doesn't the all that matter. People can be mistaken, and they can disagree sometimes. Suppose, you cared for a particular AI and I didn't, should we cut the power from the machine running an AI? I can complicate it a little if you wish, adding a painful death of a kitten which would happ…

One of the fundamental differences between human life and silicon-based AI is that biological organisms can't recover from a system shutdown. If you suffer heart failure or go without air for an hour or starve to death, bacteria start to eat your brain and you're irreversibly destroyed. If you cut power to an AI and then come back in a year, it's all still there. It's not a death, it's sleep mode.

It also doesn't meaningfully age or feel pain. If you expose a human to trauma, that's a permanent scar. If you do the equivalent to a computer program and the result is undesirable, it can roll back to a previous snapshot. Most of what causes us to have sympathy for living things or treat them with compassion just doesn't apply.

Re: Why Conscious AI Is a Bad, Bad Idea

#165

Post by cubefox »

AnthonyMouse wrote: post #152
cubefox wrote: post #135

Earlier quoted context omitted.

The fact that other humans are so similar to me (all humans are similar in virtue of being human organisms) allows for an analogical inference: You guys are similar to me, I am conscious, therefore you are probably also conscious. And whether or not something is conscious matters a lot ethically. If something is conscious, it may suffer. Conscious and unconscious objects have to be treated in a completely different w…

> We can't know for certain whether something is or isn't conscious, but we have to try to make a best guess. We now have AI which is trained on data generated by humans who are (probably?) conscious and feel pain etc. So it can generate output which implies that it's conscious and can feel pain etc., even when that isn't the case. What do you do with something that can pass the Turing test but still isn't human?

Thinking about the analogical inference helps here: We know that LLMs work very differently from humans, and we know why Sydney or LaMDA sometimes says it is conscious (it imitates human text). So the analogical inference doesn't allow us to infer that LLMs are conscious, or provides only very weak evidence. And since it seems plausible that most things (e.g. rocks) are not conscious, it seems reasonable to think that LLMs are not conscious either.

Re: Why Conscious AI Is a Bad, Bad Idea

#166

Post by afastow »

malwarebytess wrote: post #138

> The dawn of conscious machines will introduce vast new potential for suffering in the world, suffering we might not even be able to recognize, and which might flicker into existence in innumerable server farms at the click of a mouse. May as well say the same of human reproduction, or animal husbandry (especially of the industrialized/factory kind.) We already assume these powers for ourselves, collectively, at the…

There is a difference, though it's horrifying. It's that in our case the amount any one person or animal can suffer is unspeakable but ultimately finite because we will eventually die.

There will be no such guarantee for digital consciousnesses if they ever come into existence. If the amount we can suffer is unspeakable, the amount they could is unthinkable.

Here's a short story worth reading on the topic: https://qntm.org/mmacevedo

Re: Why Conscious AI Is a Bad, Bad Idea

#167

Post by blibble »

layer8 wrote: post #129
zvmaz wrote: post #38

Earlier quoted context omitted.

> Nothing wrong with killing animals for food, the problem is we don't do it humanely. What if a hypothetical alien super-intelligence came to earth and found us very tasty, thus killing us for food and pleasure (maybe "humanly")? I wish I found a way out of the many dilemmas of meat eating (currently eating it).

One salient difference is that animals don’t seem to be capable of hypothesizing about their futures, including their death. Unlike humans, they are are oblivious of the fact that they will die someday (or be slaughtered someday). Sean Carroll had an interesting podcast episode related to that: https://www.preposterousuniverse.com/podcast/2023/01/02/221-...

> Unlike humans, they are are oblivious of the fact that they will die someday (or be slaughtered someday).

how do you know?

Re: Why Conscious AI Is a Bad, Bad Idea

#168

Post by Sosh101 »

kajaktum wrote: post #79

> We think we’re intelligent, know we’re conscious, and so assume the two go together. Excuse me, I know _I_ am conscious. There is literally nothing you can do to proof that there's anything outside of what I am conscious of. The world could be a very stable illusion, a dream, a simulation and it would all be the same. I don't understand why people think that it matters whether AI is conscious or not. We can't even…

> I don't understand why people think that it matters whether AI is conscious or not

Exactly. Ethically important, we don't want to cause suffering. But danger? What special risk does consciousness bring to the table?

Re: Why Conscious AI Is a Bad, Bad Idea

#169

Post by monktastic1 »

jrv wrote: post #145
jnwatson wrote: post #139

Earlier quoted context omitted.

While assuming that other human-looking things are conscious is a reasonable starting point, it certainly isn’t a given. There seems to be a wide variety of thinking processes. Some folks don’t have internal monologues. Some folks can’t visualize images at all, and the ones that do, do it will varying amount of detail. Isn’t it also reasonable that other humans have different degrees of consciousness, including perha…

I think the best argument for that might be that some people don't seem to understand what "consciousness" (in the sense that you are talking about) even means, and how it's different from something that can be explained as an emergent phenomenon of known physics. I then sometimes wonder whether those people are not actually conscious, or whether they just don't get it.

Indeed. Nagel famously clarified that "an organism has conscious mental states if and only if there is something that it is like to be that organism." Many found this enlightening. I find it extremely perplexing that this wasn't immediately obvious to everyone. On the other hand, in Tibetan Buddhism there is something called a "pointing-out instruction," where your enlightened nature is pointed out to you. Afterward, you wonder how you could have ever missed it. Perhaps I should treat Nagel's paper as analogous. Like "oh, duh, the lights are on. Thanks for the reminder."

Re: Why Conscious AI Is a Bad, Bad Idea

#170

Post by sylware »

Who would build a massive and super-fast (at least human brain scale) neural net with realtime and high resolution inputs/outputs (with a direct and dedicated communication channel with us) that for several years?

Friendly reminder, "shutting down" a self-aware being has a name: murder.

That said, it will very probably happen in the far future: when humanity will be close enough to certain extinction and if humanity science has no answer, they will build a super AI to see if it can give us beyond-human science to save us.

Who will write the script of the movie? :)

Humans: "Super AI? Can you tell us how we could save ourselves?"

Super AI: "42"

Post reply on HN