Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We need a Butlerian Jihad against AI

erikhoel.substack.com

Post reply on HN
Print view View original

11–20 of 161 posts

Re: We need a Butlerian Jihad against AI

#11

Post by at_a_remove »

One of the problems the Butlerian Jihad ran up against, aside from the inevitable skirting of the lines from Richese and Ix (many machines on Ix, new machines), is that it runs directly counter to "Thou shalt not disfigure the soul."

Replacement of AI with Mentats (as well as other narrow specialities) has done nothing but disfigure the soul. We see few Mentats -- aside from Paul and eventually another -- who are not constricted. Similarly, if you practice medicine, well, you get the Imperial Conditioning. Certainly, a sign of trust ... but also a sign that the person's actions are no longer completely free.

Now, I am not touting the Heinlein "A human being should be able to change a diaper, plan an invasion, butcher a hog, conn a ship ..." line, exactly, but the alternative to AI is the kind of stagnation we see in Dune, millennia of locked down ritual, honed again and again, with some people becoming ... utilities.

Before we begin this jihad, we must examine the alternative futures.

Re: We need a Butlerian Jihad against AI

#12

Post by vngzs »

ampdepolymerase wrote: post #7

> To slim results. Elon Musk explained his new nihilism about the possibility of stopping AI advancement on the Joe Rogan podcast, when he said: “I tried to convince people to slow down. Slow down AI. To regulate AI. This was futile. I tried for years. Nobody listened. Nobody listened.” Rather rich coming from one of the self driving car market leaders. It certainly makes business sense to mislead academia and the po…

I do believe it's reasonable to draw a distinction between what we're doing now (which is essentially just "statistics") and what Musk warns about.

It's a mistake to believe strong AI will just be a more powerful iteration of today's weak AI. He is arguing to slow developments toward general intelligence, not developments in any narrow field.

Re: We need a Butlerian Jihad against AI

#13

Post by killingtime74 »

bpodgursky wrote: post #3

Yes. The inevitable defeatism that will show up in these comments is "Oh, but China will do it anyway, so there's no point." Which is pretty easily counterable: 1) We don't know if China would cooperate with a ban. We haven't tried. China is very complicated, and if you think you can predict what Xi will do, you are wrong. 2) If AI is truly a global existential and moral crisis, the US _could_ absolutely shut down Ch…

lol never heard of nukes or mutually assured destruction? Can’t even stop Iran or North Korea and you think China can be stopped?

Re: We need a Butlerian Jihad against AI

#14

Post by killingtime74 »

erikhoel wrote: post #10
bpodgursky wrote: post #3

Yes. The inevitable defeatism that will show up in these comments is "Oh, but China will do it anyway, so there's no point." Which is pretty easily counterable: 1) We don't know if China would cooperate with a ban. We haven't tried. China is very complicated, and if you think you can predict what Xi will do, you are wrong. 2) If AI is truly a global existential and moral crisis, the US _could_ absolutely shut down Ch…

Agreed, these are both good replies to that (which is probably the most common response for some reason)

It’s naïve in the extreme. Also way too late

Re: We need a Butlerian Jihad against AI

#16

Post by sammalloy »

I think it would be easier to emphasize and teach philosophy and ethics of computer science. The problem is, they can't even do this for business students, so focusing on the existential risk of AI alone is a drop of rain in the ocean. If you follow the topic, of let's say, sustainability, then you know this is a huge blind spot in economics and business. Look at only palm oil production as an example. The current threat to the environment of Indonesia has been recognized for decades, yet the world refuses to legislate against the multinational companies who are destroying the forests for palm oil. Again, this is only a small part of the problem, and it's deeply connected to many other problems, such as the profitable harvesting of Indonesian wood from this destruction, which recently showed up in Japan as a source for the Olympic Games infrastructure.

Re: We need a Butlerian Jihad against AI

#17

Post by devindotcom »

Regardless of the merit of this particular piece, the portion of Butler's Erewhon comprising "The Book of the Machines" makes for very interesting and forward thinking reading.

You can skip directly to it here:

https://www.gutenberg.org/files/1906/1906-h/1906-h.htm#chap2...

But the context in the story is fairly important - it takes place in a society that has essentially already carried out its own Butlerian Jihad and taken it too far. Wonderful book by the way.

I'm starting to think that the religious sects in the U.S. that laboriously evaluate a technology before incorporating it into their communities have a pretty good thing going. Sadly it's not really practical at a larger scale, and the suffering that could be avoided by adopting something early rather than late is difficult to estimate. Ah well!

Re: We need a Butlerian Jihad against AI

#18

Post by bpodgursky »

ampdepolymerase wrote: post #7

> To slim results. Elon Musk explained his new nihilism about the possibility of stopping AI advancement on the Joe Rogan podcast, when he said: “I tried to convince people to slow down. Slow down AI. To regulate AI. This was futile. I tried for years. Nobody listened. Nobody listened.” Rather rich coming from one of the self driving car market leaders. It certainly makes business sense to mislead academia and the po…

Elon wasn't calling for a hard halt like this article. He's talking about the dangers of general AI (AGI). I don't think most people would consider a self-driving car a likely path to accidental superintelligence; it's a highly-targeted application like a chess engine.

OpenAI's GPT-X engines OTOH, IMO, have a lot more potential danger because it's very unclear what they'll be used for.

Re: We need a Butlerian Jihad against AI

#20

Post by dr_dshiv »

Aren't we inappropriately reifying AI? AI doesn't really exist, other than as an academic field of research. For instance, here is how the European Union is trying to define AI for regulation purposes. Note how broad it is!

"artificial intelligence system (AI system) means software that is developed with one or more of the techniques and approaches listed in Annex I and can, for a given set of human-defined objectives, generate outputs such as content, predictions, recommendations, or decisions influencing the environments they interact with"

"Annex 1: Machine learning approaches, including supervised, unsupervised and reinforcement learning, using a wide variety of methods including deep learning; (b)Logic- and knowledge-based approaches, including knowledge representation, inductive (logic) programming, knowledge bases, inference and deductive engines, (symbolic) reasoning and expert systems; (c) Statistical approaches, Bayesian estimation, search and optimization methods."

Post reply on HN