We need a Butlerian Jihad against AI • Hacker News Board

Re: We need a Butlerian Jihad against AI

#61

Post by tudorw » Wed, Jul 07, 2021, 10:32 PM UTC

subroutine wrote: post #56

The problem is, we don't really know how consciousness works (I assume consciousness the the part the author takes issue with; most of our cognitive faculties in isolation are not that special). We don't even have a great definition of consciousness, or good tests for it, or know whether it is a linear spectrum, or if it emerges abruptly with the evolution of certain reasoning and attention faculties, and we don't kn…

Also, if we model a brain effectively, then study that model, would our brain not develop new skills from reflecting on such a model and therefore develop again beyond the model?

Re: We need a Butlerian Jihad against AI

#62

Post by at_a_remove » Wed, Jul 07, 2021, 10:34 PM UTC

johnvaluk wrote: post #52

at_a_remove wrote: post #11

One of the problems the Butlerian Jihad ran up against, aside from the inevitable skirting of the lines from Richese and Ix (many machines on Ix, new machines), is that it runs directly counter to "Thou shalt not disfigure the soul." Replacement of AI with Mentats (as well as other narrow specialities) has done nothing but disfigure the soul. We see few Mentats -- aside from Paul and eventually another -- who are not…

Well put. But isn't the concern here with some utilities becoming ... people? Either way could result in disfigured souls. Is AI simply a pursuit of slavery without guilt?

That is a whole 'nother ball of wax.

Consider someone wanting an AI. What exactly do they want? Well, is it a mind? Because we have seven billion of those and we can make more on demand. Takes a bit but they're pretty flexible.

Once you start asking questions about what kind of mind you would like, aside from the pathological types who want a trapped and helpless mind to torture (and don't think that there won't be people who would get their jollies that way), most people seem to have a kind of subconscious archetype of an old-fashioned butler (I assure you I did not pick the profession based on irony).

Your butler -- knows your business but rarely contradicts, perhaps corrects. Slides into the background when not needed, simply ... minding things. Perhaps not watched over by machines of loving grace, as Brautigan would have it, but tended to, looked out for. Without needs or drives or goals of their own to interfere with our individual or collective desires.

Yes, the idea of AI does seem to converge on a fantastically intelligent p-zed in a nice suit, a less bloodthirsty form of some of the minds encountered in Watts' Blindsight, unencumbered by interior experience, desires and attachment, or what arises from thwarted desire and attachment, suffering.

Re: We need a Butlerian Jihad against AI

#63

Post by at_a_remove » Wed, Jul 07, 2021, 10:35 PM UTC

javajosh wrote: post #59

at_a_remove wrote: post #11

One of the problems the Butlerian Jihad ran up against, aside from the inevitable skirting of the lines from Richese and Ix (many machines on Ix, new machines), is that it runs directly counter to "Thou shalt not disfigure the soul." Replacement of AI with Mentats (as well as other narrow specialities) has done nothing but disfigure the soul. We see few Mentats -- aside from Paul and eventually another -- who are not…

So, a point of nerdity: mentats were not portrayed as disfigured in Dune. They had personalities and foibles and loyalties and so on. In fact, there was no limit on who could be a mentat, or what other position of power they could hold (some of Paul's friends note how formidable a mentat-duke would be - not something they would say if it were a disfigurement). Another point of nerdity that no-one has mentioned yet, i…

Not physically disfigured, no. But ... constrained. Narrowed. Awaiting a chance to provide answers, but not questions.

Re: We need a Butlerian Jihad against AI

#64

Post by [deleted] » Wed, Jul 07, 2021, 10:37 PM UTC

johnvaluk wrote: post #52

at_a_remove wrote: post #11

One of the problems the Butlerian Jihad ran up against, aside from the inevitable skirting of the lines from Richese and Ix (many machines on Ix, new machines), is that it runs directly counter to "Thou shalt not disfigure the soul." Replacement of AI with Mentats (as well as other narrow specialities) has done nothing but disfigure the soul. We see few Mentats -- aside from Paul and eventually another -- who are not…

Well put. But isn't the concern here with some utilities becoming ... people? Either way could result in disfigured souls. Is AI simply a pursuit of slavery without guilt?

[deleted]

Re: We need a Butlerian Jihad against AI

#65

Post by pphysch » Wed, Jul 07, 2021, 10:38 PM UTC

Humans won't become "slaves" to superhuman AI any more than human cells are "slaves" to the human body.

Unexamined anthropocentric hogwash.

Re: We need a Butlerian Jihad against AI

#66

Post by Dylan16807 » Wed, Jul 07, 2021, 10:38 PM UTC

pshc wrote: post #46

Dylan16807 wrote: post #35

Earlier quoted context omitted.

> Rather rich coming from one of the self driving car market leaders. You don't need anything close to strong AI to do a reasonable job of driving a car. It seems like 90% of the problem is object recognition, and even in terms of brain-equivalent logic that's a really low bar.

It’s the 10% that gets you. To safely operate a car in all reasonable situations might very well require Strong AI. The car needs to be able to problem solve and make inferences about road conditions up ahead.

What kind of inferences need strong AI? How often does a driver need to figure out something with logic that couldn't be handled by current technology?

Level 4 self driving is fine and I really don't think it needs strong AI.

Re: We need a Butlerian Jihad against AI

#67

Post by jonstaab » Wed, Jul 07, 2021, 10:38 PM UTC

The crux of his argument, and its downfall, at least in the short term is:

> All to say: discussions about controlling or stopping AI research should be deontological—an actual moral theory or stance is needed

I don't see this happening in the near future, at least in the West. We're in the middle of total epistemological meltdown, and only capable of reasoning from utility, or some insane framework like critical theory. If we get to Strong AI in our lifetimes, we're just going to spawn a bunch of reductive, racist robots.

Re: We need a Butlerian Jihad against AI

#68

Post by tick_tock_tick » Wed, Jul 07, 2021, 10:40 PM UTC

AndrewKemendo wrote: post #9

We should be actively building this new "AI Species" because we are going to be extinct eventually and should think about making a better successor for the human species. The morality argument is nonsense. How about this: "The primary objective of humanity should be to build an intelligent system with far more precise perception, reasoning and physical manipulation capabilities than humans" That's my starting point.

> extinct eventually and should think about making a better successor for the human species

Citation needed. As it currently stands it seems incredibly unlikely we won't expand to most of our local group making extinction incredible unlikely.

Re: We need a Butlerian Jihad against AI

#69

Post by killingtime74 » Wed, Jul 07, 2021, 10:40 PM UTC

bpodgursky wrote: post #25

killingtime74 wrote: post #13

Earlier quoted context omitted.

lol never heard of nukes or mutually assured destruction? Can’t even stop Iran or North Korea and you think China can be stopped?

That's exactly my point. I did, explicitly, say it was unpleasant, and something you should only do in extremis -- when you genuinely believe that extinction a likely alternative. But the US could absolutely glass every semiconductor fab (and datacenter, and research facility) in mainland China using a combination of conventional + atomic kinetic options.

I guess we agree then. That would lead to the end of the human race

Re: We need a Butlerian Jihad against AI

#70

Post by thrower123 » Wed, Jul 07, 2021, 10:41 PM UTC

javajosh wrote: post #59

at_a_remove wrote: post #11

One of the problems the Butlerian Jihad ran up against, aside from the inevitable skirting of the lines from Richese and Ix (many machines on Ix, new machines), is that it runs directly counter to "Thou shalt not disfigure the soul." Replacement of AI with Mentats (as well as other narrow specialities) has done nothing but disfigure the soul. We see few Mentats -- aside from Paul and eventually another -- who are not…

So, a point of nerdity: mentats were not portrayed as disfigured in Dune. They had personalities and foibles and loyalties and so on. In fact, there was no limit on who could be a mentat, or what other position of power they could hold (some of Paul's friends note how formidable a mentat-duke would be - not something they would say if it were a disfigurement). Another point of nerdity that no-one has mentioned yet, i…

Every mentat depicted was addicted to nootropics, like sapho juice or melange, which long-term abuse of caused physiological changes. Not as extreme as the Navigators, but it's well beyond physical dependance.