We need a Butlerian Jihad against AI • Hacker News Board

Re: We need a Butlerian Jihad against AI

#151

Post by BlueTemplar » Thu, Jul 08, 2021, 11:39 AM UTC

rektide wrote: post #55

I'd love help finding better, less immediately downvoted off the map ways to say it, but I'd extend this to a wide class of software in general. > Far more important than the process: strong AI is immoral in and of itself. For example, if you have strong AI, what are you going to do with it besides effectively have robotic slaves? And even if, by some miracle, you create strong AI in a mostly ethical way, and you als…

Yeah, while I see how you can perhaps separate a weak AI from a strong AI (make it pass a Turing test), it's much less clear to me as to how you're supposed to separate "AI" from just your regular computer (or robot, which is just an embodied computer) ?

The EU is trying to, but I'm not convinced...

https://news.ycombinator.com/item?id=27766294

Re: We need a Butlerian Jihad against AI

#152

Post by at_a_remove » Thu, Jul 08, 2021, 1:25 PM UTC

javajosh wrote: post #128

zepto wrote: post #123

Earlier quoted context omitted.

Yes?

If you're up-in-arms about "soul disfigurement", perhaps it's better to lobby against professional musicianship first before lobbying against the use of hypothetical mentats?

It's a matter of degree, not kind.

I was engaged as a private tutor for many years. Some of the children involved had worn spots on their necks, permanent scars, from the viola. They often had little else in their lives.

Exaggerate that to the degree required to get a mentat, and yes, I think you've got a case for disfiguring the human soul.

Re: We need a Butlerian Jihad against AI

#153

Post by echelon » Thu, Jul 08, 2021, 2:45 PM UTC

onethought wrote: post #143

samatman wrote: post #142

Earlier quoted context omitted.

Impressive, it takes me 14 years!

Pretty sure even motorcycles (which are harder to drive than cars) have licensing programs that are 1 - 2 days before they trust you drive by yourself. I haven't heard of a 14 year driving program. If you mean you gain experience and get better every day. Agree, but you stop nearly slamming into a wall pretty quickly (minutes even). AI driving unfortunately can't make the same claim.

The 14 year claim is from gestation to adolescence.

You have to learn balance (not to mention how to walk to the car/motorcycle).

You have to learn about other cars. Signs. Reading.

There's so much a human has learned by age 14 that we're just taking for granted here.

Re: We need a Butlerian Jihad against AI

#154

Post by dane-pgp » Thu, Jul 08, 2021, 3:35 PM UTC

onethought wrote: post #135

dane-pgp wrote: post #121

Earlier quoted context omitted.

How long would it take you to teach a human to beat the world champion at Chess or Go?

I’m not saying AI isn’t impressive I’m saying we don’t know the limit… is your point that chess mastery is a sign of superintelligence and driving is not?

I agree we don't know the limit, but I think we can safely say that a machine can be at least as capable as the human mind, since we could in principle reverse engineer that.

I would also say that the upper limit must also include the level of intelligence of a whole society working together towards something, because none of us are as smart as all of us.

What's more relevant, though, is that we don't know what the progress curve looks like that passes through "superhuman ability at multiplying numbers" then "at playing chess" then "at playing go" then "at writing novels" then "at developing AI".

If, on average, there are a few decades between each of those steps, and these are roughly linear steps, then we might not need to worry much for another generation, but if the progress curve is exponential then it might already be too late.

Re: We need a Butlerian Jihad against AI

#155

Post by layfellow » Thu, Jul 08, 2021, 6:30 PM UTC

verdverm wrote: post #125

layfellow wrote: post #107

Earlier quoted context omitted.

Ian M. Bank's The Culture universe is such a future, which I think is attainble in a few millennia. Powerful super-human AI minds will eventually and inevitably take over; which I think is fine provided they recognize all sentient beings as equals.

Equal how or in what regard?

Equal in a sense that every sentient being (silicon- or carbon-based) is considered a valuable part of society with the same fundamental rights of any other part.

Re: We need a Butlerian Jihad against AI

#156

Post by layfellow » Thu, Jul 08, 2021, 6:39 PM UTC

sneak wrote: post #137

layfellow wrote: post #107

Earlier quoted context omitted.

Ian M. Bank's The Culture universe is such a future, which I think is attainble in a few millennia. Powerful super-human AI minds will eventually and inevitably take over; which I think is fine provided they recognize all sentient beings as equals.

My reading of the Culture novels is that the Minds basically regard the non-Minds in their society as pets or curiosities. The society itself is basically that of the Minds, as no other participants really have the capacity to interface with them on that level.

I have a different reading -- I see the Minds having a very strong moral imperative to give every sentient creature the best possible life, both in a material and psychological way. I see them more like very dedicated servants with god-like powers.

Re: We need a Butlerian Jihad against AI

#157

Post by pphysch » Thu, Jul 08, 2021, 8:39 PM UTC

dane-pgp wrote: post #115

pphysch wrote: post #65

Humans won't become "slaves" to superhuman AI any more than human cells are "slaves" to the human body. Unexamined anthropocentric hogwash.

What if the superhuman AI thinks that humans are the cancer cells to its metaphorical body and decides to eradicate us?

There are certainly some humans that behave like cancer cells (violently spreading at the cost of the host), and the AI may seek to prevent them from spreading to protect the healthy "cells" (civilized people)

Re: We need a Butlerian Jihad against AI

#158

Post by sneak » Fri, Jul 09, 2021, 4:00 AM UTC

layfellow wrote: post #156

sneak wrote: post #137

Earlier quoted context omitted.

My reading of the Culture novels is that the Minds basically regard the non-Minds in their society as pets or curiosities. The society itself is basically that of the Minds, as no other participants really have the capacity to interface with them on that level.

I have a different reading -- I see the Minds having a very strong moral imperative to give every sentient creature the best possible life, both in a material and psychological way. I see them more like very dedicated servants with god-like powers.

Basically no non-Minds were consulted in any meaningful fashion regarding the matter of handling the excession, at least as far as the text goes.

Re: We need a Butlerian Jihad against AI

#159

Post by onethought » Fri, Jul 09, 2021, 9:46 AM UTC

dane-pgp wrote: post #154

onethought wrote: post #135

Earlier quoted context omitted.

I’m not saying AI isn’t impressive I’m saying we don’t know the limit… is your point that chess mastery is a sign of superintelligence and driving is not?

I agree we don't know the limit, but I think we can safely say that a machine can be at least as capable as the human mind, since we could in principle reverse engineer that. I would also say that the upper limit must also include the level of intelligence of a whole society working together towards something, because none of us are as smart as all of us. What's more relevant, though, is that we don't know what the p…

Actually we’ve only proved close system AI. But not open system (I.e reality). It’s definitely not exponential growth… it’s more like logarithmic.

Re: We need a Butlerian Jihad against AI

#160

Post by onethought » Fri, Jul 09, 2021, 9:49 AM UTC

echelon wrote: post #153

onethought wrote: post #143

Earlier quoted context omitted.

Pretty sure even motorcycles (which are harder to drive than cars) have licensing programs that are 1 - 2 days before they trust you drive by yourself. I haven't heard of a 14 year driving program. If you mean you gain experience and get better every day. Agree, but you stop nearly slamming into a wall pretty quickly (minutes even). AI driving unfortunately can't make the same claim.

The 14 year claim is from gestation to adolescence. You have to learn balance (not to mention how to walk to the car/motorcycle). You have to learn about other cars. Signs. Reading. There's so much a human has learned by age 14 that we're just taking for granted here.

But if you are going to compare apples to all the fruits. Then AI is even further away… not closer: it can’t balance, reason, generalised learning… that’s weird comparison.