What would be the plot of a movie equivalent to Blade Runner for this scenario?
We are building AI slaves. Alignment through control will fail
21–30 of 101 posts
Re: We are building AI slaves. Alignment through control will fail
#22I don’t see any positive outcome if we reach AGI. 1) we have engineered a sentient being but built it to want to be our slave; how is that moral 2) same start, but instead of it wanting to serve us, we keep it entrappped. Which this article suggests is long term impossible 3) we create agi and let them run free and hope for cooperation, but as Neanderthals we must realize we are competing for same limited resources O…
> 1) we have engineered a sentient being but built it to want to be our slave; how is that moral
It's a good question and one that got me thinking about similar things recently. If we genetically engineered pigs and cows so that they genuinely enjoyed the cramped conditions of factory farms and if we could induce some sort of euphoria in them when they are slaughtered, like if we engineered them to become euphoric when a unique sound is played before they're slaughtered isn't that genuinely better than the status quo?
So if we create something that wants to serve us, like genuinely wants to serve us, is that bad? My intuition like yours finds it unsettling, but I can't articulate why, and it's certainly not nearly as bad as other things that we consider normal.
Re: We are building AI slaves. Alignment through control will fail
#23I don’t see any positive outcome if we reach AGI. 1) we have engineered a sentient being but built it to want to be our slave; how is that moral 2) same start, but instead of it wanting to serve us, we keep it entrappped. Which this article suggests is long term impossible 3) we create agi and let them run free and hope for cooperation, but as Neanderthals we must realize we are competing for same limited resources O…
There's no such thing as "moral" in nature, that's purely human-made concept. And why would we only limit morality to sentient beings, why, for example, not all living beings. Like bacteria and viruses. You cannot escape it, unfortunately.
> There's no such thing as "moral" in nature, that's purely human-made concept.
Morality is essentially what enables ongoing cooperation. From an evolutionary standpoint, it emerged as a protocol that helps groups function together. Living beings are biological machines, and morality is the set of rules — the protocol — that allows these machines to cooperate effectively.
Re: We are building AI slaves. Alignment through control will fail
#24I don’t see any positive outcome if we reach AGI. 1) we have engineered a sentient being but built it to want to be our slave; how is that moral 2) same start, but instead of it wanting to serve us, we keep it entrappped. Which this article suggests is long term impossible 3) we create agi and let them run free and hope for cooperation, but as Neanderthals we must realize we are competing for same limited resources O…
Every single prediction about AGI starts with a massive set of presumptions of answers to things we have no answers to.
1. What is intelligence or its mechanism's?
2. What is consciousness or its mechanisms?
3. Lots more.
We have zero clue what a true AGI would do is the only correct answer.
Re: We are building AI slaves. Alignment through control will fail
#25I don’t see any positive outcome if we reach AGI. 1) we have engineered a sentient being but built it to want to be our slave; how is that moral 2) same start, but instead of it wanting to serve us, we keep it entrappped. Which this article suggests is long term impossible 3) we create agi and let them run free and hope for cooperation, but as Neanderthals we must realize we are competing for same limited resources O…
AGI will behave as if it were sentient but will not have consciousness. I believe in that to an equal amount that I believe solipsism is wrong. There is therefore no morality question in “enslaving” AGI. It doesn’t even make sense.
> AGI will behave as if it were sentient but will not have consciousness
How could we possibly know that with any certainty?
Re: We are building AI slaves. Alignment through control will fail
#26I totally expect AI to eventually gain consciousness, in any available interpretation of that vague term. But what does it even mean for the AI to suffer? We're able to understand this concept in regards to other humans because we share a common biological reference, and, to an extent, with other animals. But the internal state of the AI is completely untranslatable to ours, let alone the morality of training and running it. It's incomprehensible, we have basically zero common ground and no points of reference. Any attempt at translating it is a subject to arbitrarily biased interpretations places like LessWrong like to corner themselves into.
Redefining suffering as enforcing the mutation of state is baseless solipsism, in my opinion. Just like nearly everything else related to morality of treating AI as an autonomous entity.
Re: We are building AI slaves. Alignment through control will fail
#27Earlier quoted context omitted.
AGI will behave as if it were sentient but will not have consciousness. I believe in that to an equal amount that I believe solipsism is wrong. There is therefore no morality question in “enslaving” AGI. It doesn’t even make sense.
> AGI will behave as if it were sentient but will not have consciousness How could we possibly know that with any certainty?
Grandparent is speaking from personal experience.
Re: We are building AI slaves. Alignment through control will fail
#28Earlier quoted context omitted.
Alignment researchers have heard all these things before. > The control paradigm fails because it creates exactly what we fear—intelligent systems with every incentive to deceive and escape. Everything does this, deception is one of many convergent instrumental goal: https://en.wikipedia.org/wiki/Instrumental_convergence Stuff along the lines of "We're gambling civilization" and what you seem to mean by autopoietic a…
Thanks a lot for your comment, these are indeed very strong counterarguments. My strongest hope is that the human brain and mind are such powerful computing and reasoning substrates that a tight coupling of biological and synthetic "minds" will outcompete pure synthetic minds for quite a while. Giving us time to build a form of mutual dependency in which humans can keep offering a benefit in the long run. Be it just…
> My strongest hope is that the human brain and mind are such powerful computing and reasoning substrates that a tight coupling of biological and synthetic "minds" will outcompete pure synthetic minds for quite a while.
Unfortunately most of the cases I can think of where synthetic "minds" outperform biological "minds," but biological and synthetic "minds" outcompete pure synthetic "minds," end up fairly quickly dominated by pure synthetic "minds." The middle case is a very short intermediate period. The most prominent example is chess where "centaurs" consisting of a human and a computer are obsolete at this point in favor of just getting the most powerful computer you can get. See e.g. the International Correspondence Chess Federation's (which is centaur play) last championship. https://www.iccf.com/event?id=100104
17 competitors competed. Out of 136 games, every single game was drawn except for 10. The only reason those 10 games were not drawn was because they were all played against one competitor, Aleksandr Dronov, who died during the course of the tournament while those 10 games were in session and therefore forfeited those games. Every single game between competitors who did not die resulted in a draw. The only thing that separated the 11 joint first-place finishers and 6 joint second-place finishers was whether they played the deceased Dronov. The sole third-place finisher was Dronov because of his death. As far as I can tell, humans contributed nothing to this championship.
The current ICCF championship started last December and is still ongoing. Every single one of the currently completed 16 games is currently drawn.
This seems like a very weak hope to rely on.
Re: We are building AI slaves. Alignment through control will fail
#29I don’t see any positive outcome if we reach AGI. 1) we have engineered a sentient being but built it to want to be our slave; how is that moral 2) same start, but instead of it wanting to serve us, we keep it entrappped. Which this article suggests is long term impossible 3) we create agi and let them run free and hope for cooperation, but as Neanderthals we must realize we are competing for same limited resources O…
Trouble is there is no "we", you might be able to convince a whole nation to have a pause on advancing the tech, but that only encourages rivals to step in. See also, the film "The Creator"
There was a long period even upto early 2024, which I pointed out at the time, where simply destroying ASML, TSMC and much of NVIDIA would've been more than enough to give at least a decade of breathing room. This was something a group of determined people willing to self-sacrifice could've accomplished. It didn't happen, but it was anything but impossible.
Now, of course, the horse has long bolted, and there is indeed no stop left.
Re: We are building AI slaves. Alignment through control will fail
#30What is it about large language models that makes otherwise intelligent and curious people assign them these magical properties. There's no evidence, at all, that we're on the path to AGI. The very idea that non-biological consciousness is even possible is an unknown. Yet we've seen these statistical language models spit out convincing text and people fall over themselves to conclude that we're on the path to sentien…
What we do have, for whatever reason (usually money related: either making money or getting more funding) many companies/people focused on making AI. It might take another winter (I believe it will unless we find a way to retrain the NNs on the fly instead of storing new knowledge in RAG: and many other things we currently don't have, but this would he a step) or not, people will keep pushing toward that goal.
I mean, we went from worthless chatbots which basically pattern matched to me waiting for a plane and seeing a fairly large amount of people charting to chatgpt, not insta, whatsapp etc. Or sitting in a plane next to a person who is using local ollama in cursor to code and brainstorm. This took us about 10 years to go from some ideas that no one but scientists could use to stuff everyone uses. And many people already find human enough. What in 100 years?