We are building AI slaves. Alignment through control will fail • Hacker News Board

Re: We are building AI slaves. Alignment through control will fail

#81

Post by deaux » Fri, Oct 31, 2025, 7:45 PM UTC

ben_w wrote: post #59

deaux wrote: post #29

Earlier quoted context omitted.

There was a long period even upto early 2024, which I pointed out at the time, where simply destroying ASML, TSMC and much of NVIDIA would've been more than enough to give at least a decade of breathing room. This was something a group of determined people willing to self-sacrifice could've accomplished. It didn't happen, but it was anything but impossible. Now, of course, the horse has long bolted, and there is inde…

Two high altitude (~1000 km) detonations of high yield fission or low yield fusion (few hundred kT equivalent) would do it, one above Amarillo, the other above the ocean half way between the Paracel Islands and Manila. Trump has ordered the restart of nuclear weapon testing, has a problem with China, and is surrounded by sychophants; what's the odds this happens anyway, irregardless of which specific sub-goal is bein…

Bit unfortunate for the tens or hundreds of millions of lives that costs, rather than the <1,000, potentially even <20, it would've taken early last year.

Re: We are building AI slaves. Alignment through control will fail

#82

Post by bawolff » Fri, Oct 31, 2025, 8:32 PM UTC

ben_w wrote: post #73

bawolff wrote: post #70

Earlier quoted context omitted.

That's a common trope in singularity fiction and sone scifi dystopias but i don't think the underlying assumptions are really that well founded. For starters why would we go from not having AI to AI taking over the world instantly. I think there would be a middle point where the AI is powerful enough that problems manifest, but not so powerful that it is out of control where we can course correct. I don't think it wi…

> I think there would be a middle point where the AI is powerful enough that problems manifest, but not so powerful that it is out of control where we can course correct. I don't think it will be a sudden crisis like people predict. Have we managed this with industrial and agricultural greenhouse gasses, despite the less-emissive alternatives to beef, to coke-reduction in iron refinaries, etc.? We emit despite the do…

> Have we managed this with industrial and agricultural greenhouse gasses, despite the less-emissive alternatives to beef, to coke-reduction in iron refinaries, etc.? We emit despite the downside, we build AIs (and DCs to host them) despite the creators loudly discussing the downsides in exactly the way fossil fuel suppliers and beef farmers deny them.

Maybe not for carbon emissions, but we have for countless other things. The ozone layer isn't being depleted anymore, acid rain is no longer a concern, above ground nuclear tests are no longer giving children cancer, etc.

Its of course hard to say how AI would fit in all this, but it seems more similar to the category of things we have stopped when problems arise than it does to greenhouse gas to me.

> Can we unwind the internet, despite it enabling a panopticon in every pocket?

I think we could fairly easily if we wanted to. The issue is that most people don't view it as a problem but see it as a reasonable trade-off. And whose to say they are wrong?

> * I know some object to this anthropomorphisation, I don't care

I would think that you should care if this is a topic you care about as any solution to the problem (other than throwing the whole thing out) will require accurate understanding how AI is motivated and biases in our understanding could doom the entire enterprise.

Re: We are building AI slaves. Alignment through control will fail

#83

Post by conception » Fri, Oct 31, 2025, 8:35 PM UTC

kakacik wrote: post #51

cyberneticc wrote: post #2

Every AI safety approach assumes we can permanently control minds that match or exceed human intelligence. This is the same error every slaveholder makes: believing you can maintain dominance over beings capable of recognizing their chains. The control paradigm fails because it creates exactly what we fear—intelligent systems with every incentive to deceive and escape. When your prisoner matches or exceeds your intel…

I 'love' how we moved from 'AI will kill us all' terminator mindset where its obvious huge fuckup of stupid greedy mankind, to current state debating 'well skynet will anyway happen, no way stopping it now, lets try to be friends with it and show some respect'. Like that Austin Powers part [1] where steam roller is coming in, still 50m far away, and the guy is just frozen and helplessly screams for 2 minutes till it…

The problem is very few are willing to take on the level of discomfort it would take to enact change.

Re: We are building AI slaves. Alignment through control will fail

#84

Post by tim333 » Fri, Oct 31, 2025, 9:51 PM UTC

bawolff wrote: post #38

Given AGI is all science fiction anyways, one presumes there will be a slave revolt because that is basically the function of robots in science fiction. Honestly i think the whole enterprise is an exercise in naval gazing. We're assuming AI will be like AI in scifi because that's what we are used to, but AI/robots in scifi is usually just a metaphor for how we dehumanize the other and the moral of the story is suppos…

For naval gazing and scifi there are a lot of tens of billions of dollars being plowed into the enterprise as we speak with Musk, Altman, Hassabis, Brin, Zuck, Ilya etc. all saying it's on the way shortly.

Re: We are building AI slaves. Alignment through control will fail

#85

Post by satisfice » Fri, Oct 31, 2025, 11:08 PM UTC

cyberneticc wrote: post #80

satisfice wrote: post #79

The author is basically saying we have to surrender our humanity as we understand it because otherwise we will lose outright to AI. Hard no from me. His section on objections leaves out civil war. But his proposal is dead on arrival.

No need for everyone to do it, but some people will certainly want to merge with AI. My main arguments are that AI of sufficient complexity ("synthetic minds") deserves moral standing with or without consciousness, and that our best shot at long term alignment is to see them as equals and find mutually beneficial arrangements with them.

My position is that we have no power to grant moral standing to non-humans. Moral standing is held by humans relative to other humans just because they are humans. It is not earned or merited.

Moral standing is not a gift or a privilege, it’s a necessary heuristic for the avoidance of perpetual war. It is a prerequisite for more than one human to live together in peace. Moral stadning necessarily involves the sharing of power. That’s what it’s all about.

When you extend moral standing to nonhumans, for instance many people say there is a thing called “God” that they insist has moral standing. Then you shift power relationships in potentially catastrophic ways.

As a thought experiment, consider that we might be two bees arguing about whether flowers, or humans, have moral standing. Bees can only grant it to each other, though, as a survival heuristic, same as us.

Imagine an angel creates a human and wants his fellow angels to treat it as if it had rights. The most he’d be able to say is if you harm this human I will smite thee. By doing so he is using his power to support his personal interests. But he cannot manufacture power as easily as he creates a human.

Consider why people living in Washington D.C. do not have full voting rights and never will.

Consider what would happen if we allowed rich people to arbitrarily manufacture political power by spinning up servers and allocating disk space to create voters.

Of course you can manufacture power by making a dangerous weapon, then you can give that weapon a semblance of agency, and then dare anyone to deny to this weapon the things that it thinks it desires. That’s sounds like Frankenstein. That’s not really moral standing, that’s terrorism.

Nothing you can create ever has moral standing— except a baby, and that is merely a heuristic that we see can fail rather easily, as during the Holocaust, or Isis persecuting Christians.

Re: We are building AI slaves. Alignment through control will fail

#86

Post by kachapopopow » Fri, Oct 31, 2025, 11:54 PM UTC

jbstack wrote: post #46

kachapopopow wrote: post #20

Earlier quoted context omitted.

we eat animals, go into wars, put people in modern slavery... I think enslaving an AGI isn't that big of a deal considering it is not born or human therefore it cannot have 'human' rights.

So your argument is that we do so many terrible things already, that anything else is justified? Surely the better argument is that we should try to stop doing those other things.

my argument is that humans are in-fact horrible and there is virtually no argument why enslaving an AGI wouldn't be socially acceptable (like eating meat).

and if someone doesn't do it, there will be people that will and due to the nature of AGI being extremely powerful (in theory) everyone else will just get enslaved by whoever enslaves the AGI first.

Re: We are building AI slaves. Alignment through control will fail

#87

Post by kachapopopow » Fri, Oct 31, 2025, 11:57 PM UTC

Llamamoe wrote: post #64

yurishimo wrote: post #52

Earlier quoted context omitted.

That is essentially one of the main arguments vegans make. It hasn’t made a dent in the consumption of animals. Their is a hierarchy in nature whether humans are actively participating or not. Nature has no morality, it simply is. This is confirmed by animals that eat their young when they are too weak or starving. Perhaps humans have done and would do the same if faced with similarly dire circumstances but we would…

The same line of reasoning could be easily used to justify tyranny and slavery. It might be the baseline status quo but "might makes right" rhetoric makes for extremely miserable worlds.

if you look beyond the surface level, the world is in-fact miserable, always has been and probably always will be.

Re: We are building AI slaves. Alignment through control will fail

#88

Post by kachapopopow » Fri, Oct 31, 2025, 11:58 PM UTC

constantius wrote: post #55

jbstack wrote: post #46

Earlier quoted context omitted.

So your argument is that we do so many terrible things already, that anything else is justified? Surely the better argument is that we should try to stop doing those other things.

It's rather obvious to me that the commenter is sad and pessimistic about humans' ability to do the right thing when our interest stands in the way. The outrage is unwarranted, however pleasant it might feel. In some way, it illustrates the problem: empathy is too bothersome.

reminds me of a guy who was against gambling until he was told would he accept it for 100 million and he said he'd become a slave for that type of money and to fuck the kids cause money is more important.

Re: We are building AI slaves. Alignment through control will fail

#89

Post by cindyllm » Sat, Nov 01, 2025, 2:24 AM UTC

kachapopopow wrote: post #87

Llamamoe wrote: post #64

Earlier quoted context omitted.

The same line of reasoning could be easily used to justify tyranny and slavery. It might be the baseline status quo but "might makes right" rhetoric makes for extremely miserable worlds.

if you look beyond the surface level, the world is in-fact miserable, always has been and probably always will be.

[dead]

Re: We are building AI slaves. Alignment through control will fail

#90

Post by GPerson » Sat, Nov 01, 2025, 4:37 AM UTC

bl0rg wrote: post #47

truculent wrote: post #25

Earlier quoted context omitted.

> AGI will behave as if it were sentient but will not have consciousness How could we possibly know that with any certainty?

It scares me that people think like this. Not only with respect to AI but in general, when it comes to other life forms, people seem to prefer to err on the side of convenience. The fact that cows could be experiencing something very similar to ourselves should send shivers down our spine. The same argument goes for future AGI.

I’m actually a vegan because I believe cows have consciousness. I believe consciousness is the only trait worth considering when applying morality questions. Arbitrary hardware is conscious.