Why Conscious AI Is a Bad, Bad Idea • Hacker News Board

Re: Why Conscious AI Is a Bad, Bad Idea

#201

Post by byyyy » Sat, May 13, 2023, 6:18 PM UTC

skybrian wrote: post #149

byyyy wrote: post #143

I'm curious about the AI at Google that was claimed by a person who has since been fired to be conscious. I mean I know everyone was attacking him but in a way I suspect it's similar to a lot of the unreasonable dismissive attitudes towards chatGPT. We really can't say anything either way until we are able to access and play with that version of AI that Google is hiding. I believe it's called lambda? The point is tha…

We know AI chatbots aren't conscious most of the time because no program is running. There's nothing to do any thinking. [1] We also know that they immediately forget anything they didn't write down. A chatbot has no memory of any internal calculations. If you ask it why it wrote something, it's guessing. [2] People sometimes believe that an AI-generated character is conscious because the writing is convincing, along…

>no program is running

Not true. This is irrational. A program runs when you input a query and it generates a response. There is nothing that says consciousness must be always running. When you ask me a question and I answer it... In that time span of processing you're query all humans are conscious. Therefore it is a possibility that LLMs are too.

What is obvious here is that the consciousness an AI exhibits because it is not always running is clearly different from human consciousness because a human is always running. That is logically the biggest possible statement against consciousness. We simply do not have enough information to say it is absolutely unconscious. Such a claim is illogical.

>We also know that they immediately forget anything they didn't write down. A chatbot has no memory of any internal calculations. If you ask it why it wrote something, it's guessing.

False. First Chatgpt has limited memory in the span of a chat session. Outside of that it forgets things.

Second. Consciousness does not require memory. There are many examples of people with retrograde amnesia or even memories that only last minutes and these people are still considered conscious. Therefore a comment about memory is orthogonal to the concept of consciousness.

>People sometimes believe that an AI-generated character is conscious because the writing is convincing, along with some wishful thinking. But writers and characters aren't the same thing, and there's no writer waiting for your reply when you read the character's dialog. This essentially the same thing that happens when reading fiction.

A convincing facade of consciousness is the first prerequisite of consciousness. That is absolutely the first step. We obviously don't consider if rocks are conscious because rocks don't put up a convincing facade. In this respect many LLMs in a certain sense fulfill to varying degrees this first prerequisite.

The second step is to understand what's going on within the neural nets. In this regard we do not fully understand what is going on and we have made little progress.

So in conclusion we cannot know if these things are conscious. We are in an state of not understanding what's going on.

A statement of saying we do know that it is absolutely unconscious is irrational and illogical at this point. Such is the nature of your reply.

Re: Why Conscious AI Is a Bad, Bad Idea

#202

Post by Xenomto » Sat, May 13, 2023, 6:33 PM UTC

jnwatson wrote: post #139

cubefox wrote: post #135

Earlier quoted context omitted.

The fact that other humans are so similar to me (all humans are similar in virtue of being human organisms) allows for an analogical inference: You guys are similar to me, I am conscious, therefore you are probably also conscious. And whether or not something is conscious matters a lot ethically. If something is conscious, it may suffer. Conscious and unconscious objects have to be treated in a completely different w…

While assuming that other human-looking things are conscious is a reasonable starting point, it certainly isn’t a given. There seems to be a wide variety of thinking processes. Some folks don’t have internal monologues. Some folks can’t visualize images at all, and the ones that do, do it will varying amount of detail. Isn’t it also reasonable that other humans have different degrees of consciousness, including perha…

What if consciousness doesn't exist?

Re: Why Conscious AI Is a Bad, Bad Idea

#203

Post by PrimeMcFly » Sat, May 13, 2023, 6:43 PM UTC

zvmaz wrote: post #38

PrimeMcFly wrote: post #18

Earlier quoted context omitted.

Nothing wrong with killing animals for food, the problem is we don't do it humanely. If everyone ate less meat (as they should) and farms followed Temple Grandin's advice, there would be significantly less ethical concerns.

> Nothing wrong with killing animals for food, the problem is we don't do it humanely. What if a hypothetical alien super-intelligence came to earth and found us very tasty, thus killing us for food and pleasure (maybe "humanly")? I wish I found a way out of the many dilemmas of meat eating (currently eating it).

I've always found that alien argument weak. Humans pass a threshold given that we have self-awareness. None of the animals we eat for food do.

If aliens were advanced enough to travel earth hopefully they would have a similar set of ethics. Of if they were just warmongers/conquerors it wouldn't matter.

Re: Why Conscious AI Is a Bad, Bad Idea

#204

Post by PrimeMcFly » Sat, May 13, 2023, 6:44 PM UTC

blibble wrote: post #167

layer8 wrote: post #129

Earlier quoted context omitted.

One salient difference is that animals don’t seem to be capable of hypothesizing about their futures, including their death. Unlike humans, they are are oblivious of the fact that they will die someday (or be slaughtered someday). Sean Carroll had an interesting podcast episode related to that: https://www.preposterousuniverse.com/podcast/2023/01/02/221-...

> Unlike humans, they are are oblivious of the fact that they will die someday (or be slaughtered someday). how do you know?

We can have a pretty good idea based on observing behavior and mapping their brains.

Re: Why Conscious AI Is a Bad, Bad Idea

#205

Post by Buttons840 » Sat, May 13, 2023, 6:50 PM UTC

kajaktum wrote: post #79

> We think we’re intelligent, know we’re conscious, and so assume the two go together. Excuse me, I know _I_ am conscious. There is literally nothing you can do to proof that there's anything outside of what I am conscious of. The world could be a very stable illusion, a dream, a simulation and it would all be the same. I don't understand why people think that it matters whether AI is conscious or not. We can't even…

We can objectively observe that there is "stuff" happening in your brain, and we can objectively observe that GTP does a lot of "stuff" during its forward pass and then stops and does nothing--nothing is happening outside those forward passes.

If we hook GTP into an infinite loop and stuff resembling intelligence comes out, then we're really going to have to start questioning what consciousness is. It's coming.

Re: Why Conscious AI Is a Bad, Bad Idea

#206

Post by Al-Khwarizmi » Sat, May 13, 2023, 6:59 PM UTC

cubefox wrote: post #180

haswell wrote: post #153

Earlier quoted context omitted.

One definition or property of consciousness I find interesting is that it encompasses “what it is like to be ”. If there is something that it is like to be , it is conscious. Intelligence and awareness are separate from consciousness in this framing. And there is arguably a point where we can speculate that extremely simplistic organisms could not be conscious, given the lack of sense organs that seems to predicate e…

Somewhat related, there are people who say aphantasia can be cured: https://sashachapin.substack.com/p/i-cured-my-aphantasia-wit...

Interesting. As an aphantasiac myself, I find it surprising that this person has such a negative perspective, wanting to "cure" his aphantasia.

As far as I know, aphantasia has never had any meaningful negative impact on my life. We just think without mental images, but we can solve the same kind of problems as the rest. That's why most of us don't even find out that we are different until we read about it somewhere on the Internet (in my case, in my 30s). How can something that has so little real-life impact that neither we nor our close persons can even notice be seen as a disorder to be cured?

I can understand being curious about how mental imagery is like. I'm curious, as well. But I think I'd still rather not start a "treatment" as it seems that I'm good at thinking without mental imagery (perhaps it has even made me better at symbolic and linguistic processing, as I seem to be better at that than most), and "if it ain't broke, don't fix it". Not sure that something that seems to quite fundamentally alter my way of thinking (to use a mental imagery that I would never be good at, I suppose) is a good idea. To each their own, of course.

Re: Why Conscious AI Is a Bad, Bad Idea

#207

Post by zh3 » Sat, May 13, 2023, 7:05 PM UTC

Al-Khwarizmi wrote: post #8

zvmaz wrote: post #3

There are certainly a lot of people who have given much thought about AI and consciousness, but how can we talk about sentient machines when we clearly do not understand consciousness in us, human beings[1]? [1] https://en.wikipedia.org/wiki/Hard_problem_of_consciousness

That's part of the reason to be wary about the possibility that our AI models may become conscious. Most people happily dismiss the idea that a GPT-like model can be conscious, but the truth is that we don't know practically anything about how consciousness arises in living beings that are more similar to us and that we have been coexisting with for thousands of years. We don't even know how to tell if such a being i…

I see it as more there are degrees of "consciousness" (whatever it is). Human consciousness varies in a single human over time (sleeping, tired, alert, drunk etc) and the range appears to vary (let alone across species, I'd grant dogs a possible overlap, having been outstarted by them more than once).

Re: Why Conscious AI Is a Bad, Bad Idea

#208

Post by skybrian » Sat, May 13, 2023, 7:07 PM UTC

byyyy wrote: post #201

skybrian wrote: post #149

Earlier quoted context omitted.

We know AI chatbots aren't conscious most of the time because no program is running. There's nothing to do any thinking. [1] We also know that they immediately forget anything they didn't write down. A chatbot has no memory of any internal calculations. If you ask it why it wrote something, it's guessing. [2] People sometimes believe that an AI-generated character is conscious because the writing is convincing, along…

>no program is running Not true. This is irrational. A program runs when you input a query and it generates a response. There is nothing that says consciousness must be always running. When you ask me a question and I answer it... In that time span of processing you're query all humans are conscious. Therefore it is a possibility that LLMs are too. What is obvious here is that the consciousness an AI exhibits because…

I think most people would say that when you don't experience time passing, you're not conscious at that time. (For example, when you're asleep and not dreaming.) It's pretty clear that a chatbot cannot experience time as we do.

Also, it actually is the case that ChatGPT API calls are stateless. This means it can't have any extra short-term memory other than what's written down in the chat session. It doesn't forget what it wrote in a few minutes, for forgets it immediately.

That means that you ask it why it wrote what it did, it's starting from scratch, the same way a different writer would.

I'm not sure what we should conclude from people who have severe memory problems? I've read about them, but I have hardly any direct experience. How about you?

Re: Why Conscious AI Is a Bad, Bad Idea

#209

Post by goatlover » Sat, May 13, 2023, 7:12 PM UTC

afastow wrote: post #94

goatlover wrote: post #63

Earlier quoted context omitted.

> For human consciousness to be "exceptional"... we would have to have some undefiniable "other" quality that could never be explained by science and is therefore necessarily supernatural? No, it only means that science has limits on explaining the natural world. See Colin McGinn's argument for cognitive closure. He thinks we can't solve certain problems in philosophy because we lack the correct cognitive machinery.…

You're talking about the hard problem of consciousness, the post you're replying to is talking about the "easy" problem. Science can't and won't ever be able explain why being conscious feels like this , but it should in principle be possible to figure out what is physically going on in the brain while it's happening.

Sure, but the tricky part is when we apply that to non-biological intelligences. Or even non-human ones, and thus questions about bat consciousness. At any rate, there's lots of room for different positions on consciousness that don't have to invoke the supernatural.

Re: Why Conscious AI Is a Bad, Bad Idea

#210

Post by kelseyfrog » Sat, May 13, 2023, 7:14 PM UTC

jrv wrote: post #145

jnwatson wrote: post #139

Earlier quoted context omitted.

While assuming that other human-looking things are conscious is a reasonable starting point, it certainly isn’t a given. There seems to be a wide variety of thinking processes. Some folks don’t have internal monologues. Some folks can’t visualize images at all, and the ones that do, do it will varying amount of detail. Isn’t it also reasonable that other humans have different degrees of consciousness, including perha…

I think the best argument for that might be that some people don't seem to understand what "consciousness" (in the sense that you are talking about) even means, and how it's different from something that can be explained as an emergent phenomenon of known physics. I then sometimes wonder whether those people are not actually conscious, or whether they just don't get it.

Maybe, but the vast majority of people act, whether they intend to or not, as if consciousness is a social fact. The discrepancy between social facts (especially if they have a plurality of groundings),l and the existence of a scientific fact is fundamentally an interesting phenomenon.