Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Dune's Butlerian Jihad and the Future of AI

technology.inquirer.net

Post reply on HN
Print view View original

51–60 of 84 posts

Re: Dune's Butlerian Jihad and the Future of AI

#51

Post by GlibMonkeyDeath »

Sol- wrote: post #19

Doesn't it seem implausible that in the world of Dune they reached their level of technology without AGI? Perhaps all their tech is pre Butlerian Jihad, but even that seems questionable. Seems like if it was adopted as a policy, there would have to be many gray areas where people pretend that they aren't really using AI - the ship's computer is just a domain specific finetuned version, after all, etc. Additionally, t…

The Bene Gesserit tried to solve alignment through their breeding schedule, but Lady Jessica threw a wrench into it by bearing a son (Paul) out of love for Duke Leto. Paul wasn't aligned with the Bene Gesserit, and hilarity ensued.

So I agree, I don't see why thinking machines are a worse alignment problem than enhanced humans. See also the Wrath of Khan.

I try not to think too deeply about it, or I won't enjoy the story. How could a society have gravity suspensors, "shields", and lasguns, but no advanced computing capability somewhere? And why use swords/knives for fighting, when projectile weapons are so much more deadly? I still love the books, but I have friends who can't enjoy the Dune world due to its many contradictions.

Edit: I realize that "shields" supposedly made ordinary projectile weapons ineffective - but they had advanced drone technology (e.g. the assassin drone used against Paul, which was also stupid)

Re: Dune's Butlerian Jihad and the Future of AI

#52

Post by keiferski »

netsharc wrote: post #49
keiferski wrote: post #12

Earlier quoted context omitted.

Man it annoys me to no end that this basic fact is so under discussed. It’s as if calling something AI is all that’s needed to justify it being the same thing as Skynet, Dune automation, Terminators, Neuromancer, and every other cultural trope using the label of AI. The ultimate fact is that LLMs are a pretty powerful technology that has little to nothing to do with most fictional depictions of intelligent machines.…

Yesterday's plane making a turnaround because of a Bluetooth speaker with the name "bomb" (media says "a 4 letter word") makes me think the Skynet-level destruction is a risk, because humans are already too chicken-shit to take responsibility for decisions. They'd rather feel safe by just following the procedure, however stupid it is. And in the the future the procudre might be "Do whatever the AI says". My thought a…

Absolutely. The real danger is that people treat LLMs like actually intelligent entities and stop making decisions themselves. Not that the LLMs will be conscious and evil in the way Sci-fi portrays.

Re: Dune's Butlerian Jihad and the Future of AI

#53

Post by GeoAtreides »

>the Bene Gesserit sisterhood with their powers of prescience

no, the Bene Gesserit doesn't have prescience, the opposite maybe (postcognition)

>Spice melange, which can only be found on the planet Arrakis (aka Dune), is the drug that enables this enhanced mental cognition

no, for anyone not called Atreides spice just prolongs life; some mentats take sapho juice, but they don't need drugs. .

Re: Dune's Butlerian Jihad and the Future of AI

#54

Post by defrost »

GeoAtreides wrote: post #53

>the Bene Gesserit sisterhood with their powers of prescience no, the Bene Gesserit doesn't have prescience, the opposite maybe (postcognition) >Spice melange, which can only be found on the planet Arrakis (aka Dune), is the drug that enables this enhanced mental cognition no, for anyone not called Atreides spice just prolongs life; some mentats take sapho juice, but they don't need drugs. .

As I recall, perhaps erroneously, Guild Navigators are nominal humans who owe their ability to move ships to being spiced to their gills.

Re: Dune's Butlerian Jihad and the Future of AI

#55

Post by sph »

tomaytotomato wrote: post #8

LLMs cannot think, they just guess with some magic with the help of things like softmax ( https://en.wikipedia.org/wiki/Softmax_function ), that makes it seem intelligent. We aren't enslaved or at any risk just yet with this form of "thinking machine", but we do need to culutrally and emotionally come to terms with them. Let's not have any other "jihad" just yet, there's plenty of other jihads going on right now.

> We aren't enslaved

The people complaining about becoming stupider because of their AI usage might have something to say. Those that wish they didn't have to use AI but are coerced by their workplace might disagree. What of the day something negative happens to you based on the wrong, unreviewed decision without appeal by an AI agent? Do you truly believe this day will never come?

You don't need literal chains to be a slave.

Re: Dune's Butlerian Jihad and the Future of AI

#56

Post by GeoAtreides »

Sol- wrote: post #19

Doesn't it seem implausible that in the world of Dune they reached their level of technology without AGI? Perhaps all their tech is pre Butlerian Jihad, but even that seems questionable. Seems like if it was adopted as a policy, there would have to be many gray areas where people pretend that they aren't really using AI - the ship's computer is just a domain specific finetuned version, after all, etc. Additionally, t…

>Doesn't it seem implausible that in the world of Dune they reached their level of technology without AGI?

This implies their technology is unfathomably complex, too complex for the human mind to understand. Doesn't seem to be the case.

>Seems like if it was adopted as a policy, there would have to be many gray areas where people pretend that they aren't really using AI - the ship's computer is just a domain specific finetuned version, after all, etc.

Ships are using Navigators, who are trained in "higher mathematics" and are also prescient. They have no need for AIs, nor do they wish to court destruction if they are found out (I can only imagine the BG will have the Spacing Guild by the balls if they knew).

> Why would the dangers of an incomprehensible machine mind be worse than the dangers of a transhuman being that was bred and optimized for generations? Who solved the alignment problem for these superhumans?

The books address these questions, especially the one about a certain God-Emperor...

Re: Dune's Butlerian Jihad and the Future of AI

#57

Post by sph »

keiferski wrote: post #12
tomaytotomato wrote: post #8

LLMs cannot think, they just guess with some magic with the help of things like softmax ( https://en.wikipedia.org/wiki/Softmax_function ), that makes it seem intelligent. We aren't enslaved or at any risk just yet with this form of "thinking machine", but we do need to culutrally and emotionally come to terms with them. Let's not have any other "jihad" just yet, there's plenty of other jihads going on right now.

Man it annoys me to no end that this basic fact is so under discussed. It’s as if calling something AI is all that’s needed to justify it being the same thing as Skynet, Dune automation, Terminators, Neuromancer, and every other cultural trope using the label of AI. The ultimate fact is that LLMs are a pretty powerful technology that has little to nothing to do with most fictional depictions of intelligent machines.…

We don't have "Skynet, Dune automation, Terminators, Neuromancer, and every other cultural trope using the label of AI" today simply because LLM technology doesn't seem to directly lead to those, but the end goal of AI research is the creation of those.

It annoys me to no end that people keep looking at the finger while pointing at the moon. No, we don't have Skynet today, but you can believe Sam Altman would have loved to see Skynet appear out of ChatGPT. That is the end goal, and it's worth discussing now, not when it's too late.

Re: Dune's Butlerian Jihad and the Future of AI

#58

Post by the8472 »

thinking_cactus wrote: post #22
the8472 wrote: post #3

Earlier quoted context omitted.

I guess top mathematicians are also mere word generators? https://openai.com/index/model-disproves-discrete-geometry-c...

Being a human may be sufficient (with important exceptions) to play chess, but playing chess well is not sufficient to be a human. Likewise with math, it turns out, it appears. Likewise with most other things we classify as needing intelligence, I bet. It turns out that intelligence is not the same as humanity or sentience etc..

But they're gaining more and more formerly exclusively human capabilities, increasingly advanced ones. The future isn't here yet, but good policy requires extrapolating and considering predictable consequences before they happen.

Re: Dune's Butlerian Jihad and the Future of AI

#59

Post by Ekaros »

decimalenough wrote: post #28
Kiro wrote: post #23

Earlier quoted context omitted.

Not a very deep justification. Probably the easiest thing to grasp for when thinking "how can I make this religious theme fit the narrative?".

It seems exactly the opposite to me: Herbert didn't want robots etc in Dune, so he came up with a religious figleaf to justify this.

Also reminds me of the shield technology and laser weapons. Making them go boom when put together is a way to get rid of them and ensure certain kind of setting. Even if the whole thing is just weird if think about physics.

Re: Dune's Butlerian Jihad and the Future of AI

#60

Post by GeoAtreides »

GlibMonkeyDeath wrote: post #51
Sol- wrote: post #19

Doesn't it seem implausible that in the world of Dune they reached their level of technology without AGI? Perhaps all their tech is pre Butlerian Jihad, but even that seems questionable. Seems like if it was adopted as a policy, there would have to be many gray areas where people pretend that they aren't really using AI - the ship's computer is just a domain specific finetuned version, after all, etc. Additionally, t…

The Bene Gesserit tried to solve alignment through their breeding schedule, but Lady Jessica threw a wrench into it by bearing a son (Paul) out of love for Duke Leto. Paul wasn't aligned with the Bene Gesserit, and hilarity ensued. So I agree, I don't see why thinking machines are a worse alignment problem than enhanced humans. See also the Wrath of Khan. I try not to think too deeply about it, or I won't enjoy the s…

They had advanced computing, a whole region was full of it: IX.

The rest of the empire was too feudal and too religious to dabble into advanced technology. And the other players (the BG, the Tleilalxu, the Guild) were against it as well.

What good are drones, except as weapons of assassination? Can't use them in an conventional war, there are shields. And they're very expensive, remember, the Great Houses don't have advanced manufacturing capabilities (that's IX). And, finally, there are no great wars in the empire, most houses keep to their planetary fief; the Baron's attack on the Atreides' forces on Dune was an exception. It was also mindbogglingly expensive, 80 years of spice profit.

Post reply on HN