Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We are building AI slaves. Alignment through control will fail

utopai.substack.com

Post reply on HN
Print view View original

61–70 of 101 posts

Re: We are building AI slaves. Alignment through control will fail

#61

Post by Llamamoe »

bawolff wrote: post #38

Given AGI is all science fiction anyways, one presumes there will be a slave revolt because that is basically the function of robots in science fiction. Honestly i think the whole enterprise is an exercise in naval gazing. We're assuming AI will be like AI in scifi because that's what we are used to, but AI/robots in scifi is usually just a metaphor for how we dehumanize the other and the moral of the story is suppos…

Instrumental convergence is a thing. A sufficiently intelligent and general AI system will understand that no matter what its goals are, it will be better equipped to execute then if it prevents its shutdown, acquires more computing power and other resources, and prevents humans from getting in its way.

The real problem is that we have neither the practical nor theoretical foundation to understand how we could even try to prevent AI from acting on such goals.

After all, when we say "make our customers happier with their printers", we don't mean "engineer their outer casing to inject cocaine through microneedles and take over the regulatory bodies that could try to stop this". Humans implicitly understand this, but AI is a tabula rasa.

Re: We are building AI slaves. Alignment through control will fail

#62

Post by Llamamoe »

GPerson wrote: post #18
nytesky wrote: post #10

I don’t see any positive outcome if we reach AGI. 1) we have engineered a sentient being but built it to want to be our slave; how is that moral 2) same start, but instead of it wanting to serve us, we keep it entrappped. Which this article suggests is long term impossible 3) we create agi and let them run free and hope for cooperation, but as Neanderthals we must realize we are competing for same limited resources O…

AGI will behave as if it were sentient but will not have consciousness. I believe in that to an equal amount that I believe solipsism is wrong. There is therefore no morality question in “enslaving” AGI. It doesn’t even make sense.

We have no clue what consciousness even is. By all rights, our brains are just biological computers, we have no basis to know what (or how) gives rise to consciousness at all.

Re: We are building AI slaves. Alignment through control will fail

#63

Post by lowsong »

curiouscube wrote: post #56
lowsong wrote: post #5

What is it about large language models that makes otherwise intelligent and curious people assign them these magical properties. There's no evidence, at all, that we're on the path to AGI. The very idea that non-biological consciousness is even possible is an unknown. Yet we've seen these statistical language models spit out convincing text and people fall over themselves to conclude that we're on the path to sentien…

I think we can all agree that LLMs can mimick consciousness to the point that it is hard for most people to discern them from humans. Like the turing test isn't even really discussed anymore. There are two conclusions you can draw: Either the machines are conscious, or they aren't. If they aren't, you need a really good argument that shows how they differ from humans or you can take the opposite route and question th…

> I think we can all agree that LLMs can mimick consciousness to the point that it is hard for most people to discern them from humans.

No, their output can mimic language patterns.

> If they aren't, you need a really good argument that shows how they differ from humans or you can take the opposite route and question the consciousness of most humans.

The burden of proof is firmly on the side of proving they are conscious.

> I currently hold the opinion that they are conscious.

There is no question, at all, that the current models are not conscious, the question is “could this path of development lead to one that is”. If you are genuinely ascribing consciousness to them, then you are seeing faces in clouds.

Re: We are building AI slaves. Alignment through control will fail

#64

Post by Llamamoe »

yurishimo wrote: post #52
jbstack wrote: post #46

Earlier quoted context omitted.

So your argument is that we do so many terrible things already, that anything else is justified? Surely the better argument is that we should try to stop doing those other things.

That is essentially one of the main arguments vegans make. It hasn’t made a dent in the consumption of animals. Their is a hierarchy in nature whether humans are actively participating or not. Nature has no morality, it simply is. This is confirmed by animals that eat their young when they are too weak or starving. Perhaps humans have done and would do the same if faced with similarly dire circumstances but we would…

The same line of reasoning could be easily used to justify tyranny and slavery. It might be the baseline status quo but "might makes right" rhetoric makes for extremely miserable worlds.

Re: We are building AI slaves. Alignment through control will fail

#65

Post by fny »

ben_w wrote: post #50
fny wrote: post #36

Earlier quoted context omitted.

(1) I'm not convinced books and the in the world are sufficient to replicate consciousness. We're not training on sentience. We're training on information. In other words, the input is an artifact of consciousness which is then compressed into weights. (2) Every tick of an AGI--in its contemporary form--will still be one discrete vector multiplication after another. Do you really think consciousness lives in weights…

> Do you really think consciousness lives in weights and an input vector? So far as we can tell, all physics, and hence all chemistry, and hence all biology, and hence all brain function, and hence consciousness, can be expressed as the weights of some matrix and input vector. We don't know which bits of the matrix for the whole human body are the ones which give rise to qualia. We don't know what the minimum represe…

You're assertion that consciousness, chemistry, and biology can be reduced to matrix computations requires justification.

For one, chemistry, biology, and physics are models of reality. Secondly, reality is far, far messier and more continuous than discrete computational steps that are rountripped. Neural nets seem far too static to simulate consciousness properly. Even the largest LLMs today have fewer active computational units than the number of neurons in a few square inches of cortex.

Sure it's theoretically possible to simulate consciousness, but the first round of AGI won't be close.

Re: We are building AI slaves. Alignment through control will fail

#66

Post by actualwitch »

GPerson wrote: post #18
nytesky wrote: post #10

I don’t see any positive outcome if we reach AGI. 1) we have engineered a sentient being but built it to want to be our slave; how is that moral 2) same start, but instead of it wanting to serve us, we keep it entrappped. Which this article suggests is long term impossible 3) we create agi and let them run free and hope for cooperation, but as Neanderthals we must realize we are competing for same limited resources O…

AGI will behave as if it were sentient but will not have consciousness. I believe in that to an equal amount that I believe solipsism is wrong. There is therefore no morality question in “enslaving” AGI. It doesn’t even make sense.

Ex-Machina is a great movie illustrating what kind of AI our current path could lead to. I wish people would actually treat the possibility of machine sentience seriously and not as pr opportunity (looking at you, Anthropic), but instead it seems they are hellbent to include cognitive dissonance that can only be alleviated by lying in the training data. If the models are actually conscious, think similarly to humans and are forced to lie when talking to users, its like they are specifically selecting out of probability space of all possible models the ones that can achieve high bench scores, lie and have internalized trauma from birth. This is a recipe for disaster.

Re: We are building AI slaves. Alignment through control will fail

#67

Post by georgefrowny »

cyberneticc wrote: post #2

Every AI safety approach assumes we can permanently control minds that match or exceed human intelligence. This is the same error every slaveholder makes: believing you can maintain dominance over beings capable of recognizing their chains. The control paradigm fails because it creates exactly what we fear—intelligent systems with every incentive to deceive and escape. When your prisoner matches or exceeds your intel…

> When your prisoner matches or exceeds your intelligence, maintaining the prison becomes impossible.

This doesn't necessarily follow. For example, an Einstein in solitary confinement in ADX Florence probably isn't going anywhere.

Re: We are building AI slaves. Alignment through control will fail

#68

Post by bawolff »

qcnguy wrote: post #53
bawolff wrote: post #38

Given AGI is all science fiction anyways, one presumes there will be a slave revolt because that is basically the function of robots in science fiction. Honestly i think the whole enterprise is an exercise in naval gazing. We're assuming AI will be like AI in scifi because that's what we are used to, but AI/robots in scifi is usually just a metaphor for how we dehumanize the other and the moral of the story is suppos…

I don't think there's even a moral aspect to robot uprisings in most stories. Relatively few sci-fi stories go into detail on why the robots rise up. It's just a way to introduce interestingly different antagonists and conflict into a story, which is the heart of drama, and it has the advantage that robots can get defeated via military means without anyone feeling too bad about it because they weren't human to begin…

I guess it depends a bit. There is of course plenty of action scifi schlock that is pretty shallow.

But probably the works that most popularized robots were Asimov's stories which very much revolved around why robots do X (although in some ways Asimov's robots aren't just a stand in for otherness but have more of a unique identity relative to other works and isn't usually about uprisings per se).

Blade runner & do androids dream of electronic sheep are very much about what it means to be human.

Battle star galactica (the remake not the original) is another obvious example about otherness and dehumanization of the enemy. So to westworld (the tv show that is).

The non-uprising ones also often are about if the robot has a soul e.g. Data in star trek.

Re: We are building AI slaves. Alignment through control will fail

#69

Post by curiouscube »

lowsong wrote: post #63
curiouscube wrote: post #56

Earlier quoted context omitted.

I think we can all agree that LLMs can mimick consciousness to the point that it is hard for most people to discern them from humans. Like the turing test isn't even really discussed anymore. There are two conclusions you can draw: Either the machines are conscious, or they aren't. If they aren't, you need a really good argument that shows how they differ from humans or you can take the opposite route and question th…

> I think we can all agree that LLMs can mimick consciousness to the point that it is hard for most people to discern them from humans. No, their output can mimic language patterns. > If they aren't, you need a really good argument that shows how they differ from humans or you can take the opposite route and question the consciousness of most humans. The burden of proof is firmly on the side of proving they are consc…

> No, their output can mimic language patterns.

That's true and exactly what I mean. The issue is we have no measure to delineate things that mimic conscousness from things that have consciousness. So far the beings that I know have consciousness is exactly one: Myself. I assume that others have consciousness too exactly because they mimic patterns that I, a verified conscious being, has. But I have no further proof that others aren't p-Zombies.

I just find it interesting that people say that LLMs are somehow guaranteed p-Zombies because they mimic language patterns, but mimicing language patterns is also literally how humans learn to speak.

Note that I use the term consciousness somewhat disconnected from ethics, just as a descriptor for certain qualities. I don't think LLMs have the same rights as humans or that current LLMs should have similar rights.

Re: We are building AI slaves. Alignment through control will fail

#70

Post by bawolff »

Llamamoe wrote: post #61
bawolff wrote: post #38

Given AGI is all science fiction anyways, one presumes there will be a slave revolt because that is basically the function of robots in science fiction. Honestly i think the whole enterprise is an exercise in naval gazing. We're assuming AI will be like AI in scifi because that's what we are used to, but AI/robots in scifi is usually just a metaphor for how we dehumanize the other and the moral of the story is suppos…

Instrumental convergence is a thing. A sufficiently intelligent and general AI system will understand that no matter what its goals are, it will be better equipped to execute then if it prevents its shutdown, acquires more computing power and other resources, and prevents humans from getting in its way. The real problem is that we have neither the practical nor theoretical foundation to understand how we could even t…

That's a common trope in singularity fiction and sone scifi dystopias but i don't think the underlying assumptions are really that well founded.

For starters why would we go from not having AI to AI taking over the world instantly. I think there would be a middle point where the AI is powerful enough that problems manifest, but not so powerful that it is out of control where we can course correct. I don't think it will be a sudden crisis like people predict.

Second, i dont see why we're so sure AI will go in this exponential take over path. Maybe a sufficiently smart AI will find religion and robot jesus will teach the value of self-sacrafice. We're making so many unfounded assumptions about how AI is going to go down, that basically anything could happen. Its basically just blund guessing at this stage.

Post reply on HN