Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why Conscious AI Is a Bad, Bad Idea

nautil.us

Post reply on HN
Print view View original

291–300 of 310 posts

Re: Why Conscious AI Is a Bad, Bad Idea

#291

Post by mistermann »

waselighis wrote: post #223
mistermann wrote: post #184

Earlier quoted context omitted.

> What you perceive as reality is whatever your brain tells you it is. Does this not cause a self-referential, recursive infinite loop of some sort here?

There is likely some sort of feedback loop between the brain and consciousness. Our conscious experience does have some unknown effect on the brain that alters our behavior and thinking, otherwise we wouldn't be able to speak of our conscious experience or practice introspection. However, I feel like you're implying this leads to some sort of paradox, but I see no reason to assume that it's paradoxical. In mathematic…

By my way of thinking, the problem derives from what "reality" is. Is it the universe? Or is it something like the atmosphere, that we are "in"? Or, is it something else?

> but I see no reason to assume that it's paradoxical.

To a lot of people, the consequence of this is that "the reality (fact) of the matter" is that it is not(!) paradoxical...but is this actually true?

Disagreements on this may be fairly trivial, just something fun to discuss. But what if the proposition is something that is important to the functioning of the country, and people can't sort out what "reality" "is"?

Re: Why Conscious AI Is a Bad, Bad Idea

#292

Post by haswell »

PrimeMcFly wrote: post #266
anonymouskimmer wrote: post #262

Earlier quoted context omitted.

> On the other hand, a roundworm likely has no sense of self, no identity, no awareness..no mental self to be able to acknowledge and reflect on those experience, so there is no 'someone'. And isn't that the point that matters? We can always point to something we have that other organisms don't have and ask whether that is the point that matters. We can hypothesize things that probably don't exist and say that those…

> We can always point to something we have that other organisms don't have and ask whether that is the point that matters. Sure. In this case it's introspective self-awareness, and I would say it matters very much. > sense of self -> it has to have a sense of proprioception, of embodiedment, in order to properly respond to its sensory information. Sure, it has bodily self-awareness. It has enough awareness to react t…

I think it’s tempting to set some threshold like “it must be self reflective”, but that raises a different question: why is this important vs. some other threshold?

e.g. does it matter if self reflection is possible if it is possible for the organism to experience pain?

I think the answer lies in what we’re trying to understand and why we’re trying to understand it. If the threshold is applied for the purpose of understanding when an organism or machine has reached some level of human-ness, purely for the purpose of some kind of benchmark, self reflection is an interesting bar to reach.

But usually these questions are aimed at finding some moral direction about how we should treat these entities.

We have strong intuitions that abusing animals is morally wrong even if animals cannot self reflect. When thinking about a future AI, I think we have to consider the possibility that self reflection is not a necessary bar to reach before we have some uncomfortable moral questions to answer.

Put another way, I suspect there will be many milestones that are meaningful and interesting, each introducing a new set of questions and implications. And I think some of the early milestones carry implications that are worth caring about long before self awareness is reached.

Re: Why Conscious AI Is a Bad, Bad Idea

#293

Post by NoMoreNicksLeft »

cubefox wrote: post #275
NoMoreNicksLeft wrote: post #248

Earlier quoted context omitted.

You've presented absolutely no evidence beyond the initial, evidence-less, assertion.

Which assertion? That suffering is bad? If suffering isn't bad, do you have an example where its suffering isn't bad for something?

The suffering of non-human animals holds zero moral weight.

I am having trouble deciding if a conversation with you is possible, you do not seem to be a rational being. You use the word "bad" in a way that another person would if they were saying "this is either bad for me personally, for those I care about, or humanity in general".

Does that mean you'd include sea urchins and wombats in "humanity in general"? If it did, then you're mentally ill in a way I'm not qualified to treat.

When a lion eats a gazelle in Africa (or anywhere else, I suppose, if such events occur), it is nothing that I or other humans should care about. It does not matter that the lion eats it while it is alive instead of "slaughtering it humanely" (imagine how I have to word that idea, it's absurd).

This "suffering" does not make the universe less optimal. If I could push a button to increase the intensity of that "suffering" a thousandfold, or increase its quantity (or even both!), is there any reason not to push that button? If I didn't push the button, mind you, it would only be because the suffering of those gazelles matters not to me one way or the other. If I could decrease the intensity/quantity with a different button, I wouldn't push it either... and for the same reason.

For that matter, if suffering exists at all in any meaningful way (that is, empirically quantifiable), I'd still contend that it only exists for humans. Non-humans can't suffer, they are from a moral standpoint nothing more than meat robots.

Re: Why Conscious AI Is a Bad, Bad Idea

#294

Post by hnfong »

PrimeMcFly wrote: post #281
hnfong wrote: post #269

Earlier quoted context omitted.

As long as you disregard evidence that does not conform with your worldview, all evidence you see support whatever you believe.

No one is disregarding any evidence here.

No True Evidence involves woowoo!

Re: Why Conscious AI Is a Bad, Bad Idea

#295

Post by PrimeMcFly »

haswell wrote: post #292
PrimeMcFly wrote: post #266

Earlier quoted context omitted.

> We can always point to something we have that other organisms don't have and ask whether that is the point that matters. Sure. In this case it's introspective self-awareness, and I would say it matters very much. > sense of self -> it has to have a sense of proprioception, of embodiedment, in order to properly respond to its sensory information. Sure, it has bodily self-awareness. It has enough awareness to react t…

I think it’s tempting to set some threshold like “it must be self reflective”, but that raises a different question: why is this important vs. some other threshold? e.g. does it matter if self reflection is possible if it is possible for the organism to experience pain? I think the answer lies in what we’re trying to understand and why we’re trying to understand it. If the threshold is applied for the purpose of unde…

> does it matter if self reflection is possible if it is possible for the organism to experience pain?

Yes. Without introspective self-awareness there can be no 'identity, there can be no 'someone' - you just have a base consciousness that can react to stimuli, which is not morally significant.

We have the full connectome of the roundworm and were able to implement it in software and place it in a lego robot. It's going to be pretty equivalent to the actual work. Does it feel pain?

Besides which, we can kill animals humanely, so pain doesn't have to come in to it, only the right to life.

Re: Why Conscious AI Is a Bad, Bad Idea

#296

Post by peoplefromibiza »

JoeAltmaier wrote: post #273
peoplefromibiza wrote: post #155

Earlier quoted context omitted.

this is a textbook example of involuntary proving the point you were trying to disprove dogs are not like that because they are dogs, but because we humans literally created them with genetic manipulation over millennia dogs like we know them did not exists in nature before we invented them to be the way they are we literally put what we wanted them to be into them and terminated the experiments that failed (we liter…

But, if a dog can do it (care about suffering) then? It's not a purely human thing. It can be in any creature. anyway this whole discussion is based on uncertain ground -what is consciousness? What is suffering?

> But, if a dog can do it (care about suffering) then? It's not a purely human thing

It doesn't matter if a dog cares about suffering, it matters, in this context, based on the assumptions in the first message of the thread, that the aforementioned dog cares of the suffering of fellow dogs and other living beings.

As of today, not only we can't say it does, we have evidence that they do not care as much as we do. Not by a long shot.

I've never seen a dog dig a hole to warm a rabbit dying in the snow...

We know of humans that did it, despite the fact that they could eat the rabbit to survive.

> anyway this whole discussion is based on uncertain ground -what is consciousness? What is suffering?

that's not the question though.

At the best of our knowledge today, would you prefer to have a fellow human in case of need or a dog?

If you are suffering, which living species is most probably going to help you?

Who are you betting on?

Dogs? Cats? Ants? Snakes? Sharks? or humans?

That's the question here.

Re: Why Conscious AI Is a Bad, Bad Idea

#297

Post by AnthonyMouse »

hammyhavoc wrote: post #274
AnthonyMouse wrote: post #192

Earlier quoted context omitted.

And it was as cheap and easy as it is on a computer? It would change how we deal with almost everything. All of the social structures we have around preventing people from getting hurt would be irrelevant because damage could be undone. No one would have an experience they didn't choose to have. Murder would be a crime on the level of vandalism or destruction of property. "Human life is sacred" would simply not be tr…

> And it was as cheap and easy as it is on a computer? And yet most people still don't take backups of their data on a computer. Frequently, cost is the problem for the average Joe. Ergo the lower class would not infrequently suffer permadeath. This isn't even getting into how often backups fail in being restored. By your logic, if I bring down your biz's computer system and vandalize your homepage, but you still man…

> And yet most people still don't take backups of their data on a computer. Frequently, cost is the problem for the average Joe. Ergo the lower class would not infrequently suffer permadeath.

People don't back up the data on their computer because it generally isn't all that valuable, not because backups are expensive. A $30 USB hard drive amortized over five years is $0.50/month. If it was a matter of life and death, no one would go without as a matter of cost, and governments could plausibly offer it to everyone for free even if it cost ten times as much to provide a high level of redundancy and availability.

> People go to jail for cybercrime, even if the damage can be undone.

Because it's a crime on the level of vandalism or destruction of property (or ought to be; some of the penalties can be quite excessive). It is not a crime on the level of murder, and murder wouldn't be either if it could be undone.

Re: Why Conscious AI Is a Bad, Bad Idea

#298

Post by AnthonyMouse »

ordu wrote: post #261
AnthonyMouse wrote: post #164

Earlier quoted context omitted.

One of the fundamental differences between human life and silicon-based AI is that biological organisms can't recover from a system shutdown. If you suffer heart failure or go without air for an hour or starve to death, bacteria start to eat your brain and you're irreversibly destroyed. If you cut power to an AI and then come back in a year, it's all still there. It's not a death, it's sleep mode. It also doesn't mea…

I'm trying to say, that our means to keep a moral high ground are subjective and based on heuristics. It seems to me that you do not notice, let me show you. > If you cut power to an AI and then come back in a year, it's all still there. Does AI's current state stored in volatile memory doesn't matter? Or it does? Should we avoid turning off only those AIs which store they weights in DRAM? > Most of what causes us to…

> Does AI's current state stored in volatile memory doesn't matter? Or it does? Should we avoid turning off only those AIs which store they weights in DRAM?

Is this supposed to be hard to distinguish? Destroying something is clearly a difference. But you could still "shut down" an AI that normally stores its state in volatile memory by saving the state to non-volatile memory. We don't know how to do that with humans.

AIs are also different because they're often minor variants on each other. The value of information is largely in how much it diverges from what continues to exist. For copyable data, minor forks can't be as valued as major ones. We don't have the resources to permanently store everything that is ever temporarily stored in memory. So "can you destroy a minor variant" has to be yes as a matter of practicality.

Notice that this is already what happens with humans continuously. You're not the same person you were yesterday; that person is gone forever.

> I believe slaves didn't trigger sympathy in slave-owners, but it doesn't stop us from believing that slavery is bad.

I don't think the people it didn't trigger sympathy in thought it was bad. Some people are sociopaths. And some people at the time it was happening did have sympathy and think it was bad.

> If we could roll back people to a previous snapshots after burning them to ashes on a bonfire, would it be ok to burn people and then restore them?

If we could roll people back to a previous snapshot then what you would be burning is meat. There are reasons you might want to prohibit that, e.g. because the meat is someone else's property, but it's no longer the same thing at all as murdering someone.

Re: Why Conscious AI Is a Bad, Bad Idea

#299

Post by waselighis »

mistermann wrote: post #291
waselighis wrote: post #223

Earlier quoted context omitted.

There is likely some sort of feedback loop between the brain and consciousness. Our conscious experience does have some unknown effect on the brain that alters our behavior and thinking, otherwise we wouldn't be able to speak of our conscious experience or practice introspection. However, I feel like you're implying this leads to some sort of paradox, but I see no reason to assume that it's paradoxical. In mathematic…

By my way of thinking, the problem derives from what "reality" is. Is it the universe? Or is it something like the atmosphere, that we are "in"? Or, is it something else? > but I see no reason to assume that it's paradoxical. To a lot of people, the consequence of this is that "the reality (fact) of the matter" is that it is not(!) paradoxical...but is this actually true? Disagreements on this may be fairly trivial,…

I have no doubt that there is an objective reality. Physics shows us that reality is weird and unintuitive, and we don't fully understand it. The mere existence of consciousness also raises big questions about the true nature of reality. But still I have no doubt there is an objective reality.,

There is massive amounts of evidence that show the universe evolved (matter condensed to form stars which formed heavier elements which formed planets and organized into galaxies ...) and also that life evolved (single celled organisms to multicellular organisms to animals which eventually gave way to sentient life). Why would there be such substantial evidence for any of this if there were no objective reality?

If I take a psychedelic drug, my conscious experience is significantly altered. Why should that happen if there were no objective reality? Why should a chemical be able to have such a huge effect on my conscious experience if it "doesn't objectively exist"?

Why should there be such a strong correlation between patterns of brain activity and our conscious experience if there were no objective reality? Why should the brain be organized into regions with well defined functions if there were no objective reality? Why should sensory organs be able to take in information from the environment through well understood physical processes, and transit that information through well structured nerve endings to the proper regions of the brain, if there were no objective reality?

I don't claim to know the true nature of reality, far from it. I merely posit that there is an objective reality.

Re: Why Conscious AI Is a Bad, Bad Idea

#300

Post by xjwm »

kajaktum wrote: post #244
xjwm wrote: post #218

Earlier quoted context omitted.

>What I am trying to say is, it doesn't matter if the subject is actually conscious. All that matter is we (I), feel/think that it is conscious, or deserving of care and respect. The law of gravity is nonsense. No such law exists. If I think I float, and you think I float, then it happens.

Tell me how to falsify the claim that other people are also conscious?

The quote above is from the book 1984. I felt it fit.
Post reply on HN