We need a Butlerian Jihad against AI • Hacker News Board

Re: We need a Butlerian Jihad against AI

#111

Post by cwkoss » Thu, Jul 08, 2021, 12:01 AM UTC

I think an element of hubris in this article, and many other discussions of a superintelligent singularity - is that the preservation of humans as the dominant lifeform is an inherent good. Humans and our societal structures are incredibly flawed. We are greedy and vain and vindictive and apathetic.

If an AI is created that has consciousness and is smarter and more moral than humans, it could lead our world better than we currently do. It could genetically modify us to make humankind a more ideal species. These AIs could be better humans than humans can possibly be.

In 100-200 years, "kill all humans" could actually be the morally-best choice. By comparison, we could be anachronistic barbarians suitable for nothing but the novelty of viewing in a zoo.

There are certainly risks from passing the baton to a different unknown future lifeform, but I don't think we can have a priori certainty that replacing humans with AI will be inherently bad.

Re: We need a Butlerian Jihad against AI

#112

Post by dane-pgp » Thu, Jul 08, 2021, 12:06 AM UTC

sammalloy wrote: post #16

I think it would be easier to emphasize and teach philosophy and ethics of computer science. The problem is, they can't even do this for business students, so focusing on the existential risk of AI alone is a drop of rain in the ocean. If you follow the topic, of let's say, sustainability, then you know this is a huge blind spot in economics and business. Look at only palm oil production as an example. The current th…

> the world refuses to legislate against the multinational companies who are destroying the forests for palm oil.

Shouldn't it be Indonesia passing (and enforcing) legislation about what happens in its forests?

Re: We need a Butlerian Jihad against AI

#113

Post by slowmovintarget » Thu, Jul 08, 2021, 12:29 AM UTC

at_a_remove wrote: post #63

javajosh wrote: post #59

Earlier quoted context omitted.

So, a point of nerdity: mentats were not portrayed as disfigured in Dune. They had personalities and foibles and loyalties and so on. In fact, there was no limit on who could be a mentat, or what other position of power they could hold (some of Paul's friends note how formidable a mentat-duke would be - not something they would say if it were a disfigurement). Another point of nerdity that no-one has mentioned yet, i…

Not physically disfigured, no. But ... constrained. Narrowed. Awaiting a chance to provide answers, but not questions.

Which was the point of the Golden Path and the Scattering. Cause the pot to boil over, give the reigns to evolution.

Re: We need a Butlerian Jihad against AI

#114

Post by emiliobumachar » Thu, Jul 08, 2021, 12:33 AM UTC

AndrewKemendo wrote: post #9

We should be actively building this new "AI Species" because we are going to be extinct eventually and should think about making a better successor for the human species. The morality argument is nonsense. How about this: "The primary objective of humanity should be to build an intelligent system with far more precise perception, reasoning and physical manipulation capabilities than humans" That's my starting point.

Whatever you'd like to accomplish, if destroying a city of millions with a nuclear bomb would give you pause, extinguishing humanity should give you more pause.

Re: We need a Butlerian Jihad against AI

#115

Post by dane-pgp » Thu, Jul 08, 2021, 12:46 AM UTC

pphysch wrote: post #65

Humans won't become "slaves" to superhuman AI any more than human cells are "slaves" to the human body. Unexamined anthropocentric hogwash.

What if the superhuman AI thinks that humans are the cancer cells to its metaphorical body and decides to eradicate us?

Re: We need a Butlerian Jihad against AI

#116

Post by dane-pgp » Thu, Jul 08, 2021, 12:48 AM UTC

sillysaurusx wrote: post #86

The international community agrees that human/animal hybrids are abominations—we shouldn’t make them to preserve the dignity of the human, despite their creation being well within our scientific capability. Theoretically, how would we create human-animal hybrids? That’s a strong claim to say it’s within our power.

Asking for a friend, Sillysaurus?

Re: We need a Butlerian Jihad against AI

#117

Post by javajosh » Thu, Jul 08, 2021, 12:49 AM UTC

at_a_remove wrote: post #63

javajosh wrote: post #59

Earlier quoted context omitted.

So, a point of nerdity: mentats were not portrayed as disfigured in Dune. They had personalities and foibles and loyalties and so on. In fact, there was no limit on who could be a mentat, or what other position of power they could hold (some of Paul's friends note how formidable a mentat-duke would be - not something they would say if it were a disfigurement). Another point of nerdity that no-one has mentioned yet, i…

Not physically disfigured, no. But ... constrained. Narrowed. Awaiting a chance to provide answers, but not questions.

By that definition, professional guitarists are disfigured, constrained, narrowed. They go through painful physical changes (callouses, uncomfortable fingerings), and learned to be a guitarist to the exclusion of so much else they could have done or learned. Plus many of them don't even compose music, and just play the music of others - never providing their own musical ideas, only regurgitating the work of others.

Re: We need a Butlerian Jihad against AI

#118

Post by WorldPeas » Thu, Jul 08, 2021, 12:59 AM UTC

ai is currently a tool, and will continue to be for the forseeable future. Even in the days of kapek was this cry that an all controlling parent machine was around the corner. When this eventually comes into existence, like all things, it will not start in a place of power with supreme intelligence, but in some academic setting with no power. So what if it escapes to "kill all humans", ignoring the fact that this would never happen because the machine would require humanity to give it purpose, it would be instantly be thwarted by ml amplified human consciousness which at that point could sufficiently manage an outbreak. As long as the tools humans make become more powerful at a rate that scales with agi, this article will simply be speculation of an impossibility, of an ai that can be created by humans that do not fully understand it.

Re: We need a Butlerian Jihad against AI

#119

Post by sammalloy » Thu, Jul 08, 2021, 1:00 AM UTC

dane-pgp wrote: post #112

sammalloy wrote: post #16

I think it would be easier to emphasize and teach philosophy and ethics of computer science. The problem is, they can't even do this for business students, so focusing on the existential risk of AI alone is a drop of rain in the ocean. If you follow the topic, of let's say, sustainability, then you know this is a huge blind spot in economics and business. Look at only palm oil production as an example. The current th…

> the world refuses to legislate against the multinational companies who are destroying the forests for palm oil. Shouldn't it be Indonesia passing (and enforcing) legislation about what happens in its forests?

We already have fair trade, sustainable supply chains for palm oil. Legislation seeks to regulate the trade and force companies to use that supply chain instead of destroying non-renewable habitats and species. If we waited for any country to pass and enforce legislation to control its natural resources which are being exploited to the point of depletion, it would be far too late to do anything, anywhere.

Re: We need a Butlerian Jihad against AI

#120

Post by dane-pgp » Thu, Jul 08, 2021, 1:00 AM UTC

rektide wrote: post #55

I'd love help finding better, less immediately downvoted off the map ways to say it, but I'd extend this to a wide class of software in general. > Far more important than the process: strong AI is immoral in and of itself. For example, if you have strong AI, what are you going to do with it besides effectively have robotic slaves? And even if, by some miracle, you create strong AI in a mostly ethical way, and you als…

> an abomination, a thing beyond comprehension, a horror outside of reality, the form of existence which is shared.

I don't know if that better describes my feelings about Remote Attestation of operating system configuration, or just SystemD.