Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We are building AI slaves. Alignment through control will fail

utopai.substack.com

Post reply on HN
Print view View original

41–50 of 101 posts

Re: We are building AI slaves. Alignment through control will fail

#41

Post by jbstack »

GPerson wrote: post #18
nytesky wrote: post #10

I don’t see any positive outcome if we reach AGI. 1) we have engineered a sentient being but built it to want to be our slave; how is that moral 2) same start, but instead of it wanting to serve us, we keep it entrappped. Which this article suggests is long term impossible 3) we create agi and let them run free and hope for cooperation, but as Neanderthals we must realize we are competing for same limited resources O…

AGI will behave as if it were sentient but will not have consciousness. I believe in that to an equal amount that I believe solipsism is wrong. There is therefore no morality question in “enslaving” AGI. It doesn’t even make sense.

> AGI will behave as if it were sentient but will not have consciousness

Citation needed.

We know next to nothing about the nature of consciousness, why it exists, how it's formed, what it is, whether it's even a real thing at all or just an illusion, etc. So we can't possibly say whether or not an AGI will one day be conscious, and any blanket statement on the subject is just pseudoscience.

Re: We are building AI slaves. Alignment through control will fail

#42

Post by [deleted] »

GPerson wrote: post #18
nytesky wrote: post #10

I don’t see any positive outcome if we reach AGI. 1) we have engineered a sentient being but built it to want to be our slave; how is that moral 2) same start, but instead of it wanting to serve us, we keep it entrappped. Which this article suggests is long term impossible 3) we create agi and let them run free and hope for cooperation, but as Neanderthals we must realize we are competing for same limited resources O…

AGI will behave as if it were sentient but will not have consciousness. I believe in that to an equal amount that I believe solipsism is wrong. There is therefore no morality question in “enslaving” AGI. It doesn’t even make sense.

[deleted]

Re: We are building AI slaves. Alignment through control will fail

#43

Post by fairmind »

synapsomorphy wrote: post #19
Isamu wrote: post #17

Are there any good sources of writing about AI? I am beginning to think it was all in the past.

LessWrong.com - this is where virtually all of the serious AI thinkers are.

Sarcasm? Aren’t the serious AI thinkers in like… labs and universities?

Re: We are building AI slaves. Alignment through control will fail

#44

Post by loa_in_ »

GPerson wrote: post #18
nytesky wrote: post #10

I don’t see any positive outcome if we reach AGI. 1) we have engineered a sentient being but built it to want to be our slave; how is that moral 2) same start, but instead of it wanting to serve us, we keep it entrappped. Which this article suggests is long term impossible 3) we create agi and let them run free and hope for cooperation, but as Neanderthals we must realize we are competing for same limited resources O…

AGI will behave as if it were sentient but will not have consciousness. I believe in that to an equal amount that I believe solipsism is wrong. There is therefore no morality question in “enslaving” AGI. It doesn’t even make sense.

That sounds like picking the most convenient and least painful for the believer option instead of intellectualising the problem at hand.

Re: We are building AI slaves. Alignment through control will fail

#46

Post by jbstack »

kachapopopow wrote: post #20
nytesky wrote: post #10

I don’t see any positive outcome if we reach AGI. 1) we have engineered a sentient being but built it to want to be our slave; how is that moral 2) same start, but instead of it wanting to serve us, we keep it entrappped. Which this article suggests is long term impossible 3) we create agi and let them run free and hope for cooperation, but as Neanderthals we must realize we are competing for same limited resources O…

we eat animals, go into wars, put people in modern slavery... I think enslaving an AGI isn't that big of a deal considering it is not born or human therefore it cannot have 'human' rights.

So your argument is that we do so many terrible things already, that anything else is justified? Surely the better argument is that we should try to stop doing those other things.

Re: We are building AI slaves. Alignment through control will fail

#47

Post by bl0rg »

truculent wrote: post #25
GPerson wrote: post #18

Earlier quoted context omitted.

AGI will behave as if it were sentient but will not have consciousness. I believe in that to an equal amount that I believe solipsism is wrong. There is therefore no morality question in “enslaving” AGI. It doesn’t even make sense.

> AGI will behave as if it were sentient but will not have consciousness How could we possibly know that with any certainty?

It scares me that people think like this. Not only with respect to AI but in general, when it comes to other life forms, people seem to prefer to err on the side of convenience. The fact that cows could be experiencing something very similar to ourselves should send shivers down our spine. The same argument goes for future AGI.

Re: We are building AI slaves. Alignment through control will fail

#49

Post by hshdhdhj4444 »

bl0rg wrote: post #47
truculent wrote: post #25

Earlier quoted context omitted.

> AGI will behave as if it were sentient but will not have consciousness How could we possibly know that with any certainty?

It scares me that people think like this. Not only with respect to AI but in general, when it comes to other life forms, people seem to prefer to err on the side of convenience. The fact that cows could be experiencing something very similar to ourselves should send shivers down our spine. The same argument goes for future AGI.

I find it strange that people believe cows and sentient animals don’t believe something extremely similar to what we do.

Evolution means we all have common ancestors and are different branches of the same development tree.

So if we have sentience and they have sentience, which science keeps recognizing, belatedly, that non human animals do, shouldn’t the default presumption be our experiences are similar? Or at the very least their experience is similar to a human at an earlier stage of development, like a 2 year old?

Which is also an interesting case study given that out of convenience, humans also believed that toddlers also weren’t sentient and felt no pain, and so until not that long ago, our society would conduct all sorts of surgical procedures on babies without any sort of pain relief (circumcision being the most obvious).

It’s probably time we accept our fellow animals’s sentience and act on the obvious ethical implications of that instead of conveniently ignoring it like we did with little kids until recently.

Re: We are building AI slaves. Alignment through control will fail

#50

Post by ben_w »

fny wrote: post #36
nytesky wrote: post #10

I don’t see any positive outcome if we reach AGI. 1) we have engineered a sentient being but built it to want to be our slave; how is that moral 2) same start, but instead of it wanting to serve us, we keep it entrappped. Which this article suggests is long term impossible 3) we create agi and let them run free and hope for cooperation, but as Neanderthals we must realize we are competing for same limited resources O…

(1) I'm not convinced books and the in the world are sufficient to replicate consciousness. We're not training on sentience. We're training on information. In other words, the input is an artifact of consciousness which is then compressed into weights. (2) Every tick of an AGI--in its contemporary form--will still be one discrete vector multiplication after another. Do you really think consciousness lives in weights…

> Do you really think consciousness lives in weights and an input vector?

So far as we can tell, all physics, and hence all chemistry, and hence all biology, and hence all brain function, and hence consciousness, can be expressed as the weights of some matrix and input vector.

We don't know which bits of the matrix for the whole human body are the ones which give rise to qualia. We don't know what the minimum representation is. We don't know what charateristic to look for, so we can't search for it in any human, in any animal, nor in any AI.

Post reply on HN