Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Dune's Butlerian Jihad and the Future of AI

technology.inquirer.net

Post reply on HN
Print view View original

61–70 of 84 posts

Re: Dune's Butlerian Jihad and the Future of AI

#61

Post by GeoAtreides »

defrost wrote: post #54
GeoAtreides wrote: post #53

>the Bene Gesserit sisterhood with their powers of prescience no, the Bene Gesserit doesn't have prescience, the opposite maybe (postcognition) >Spice melange, which can only be found on the planet Arrakis (aka Dune), is the drug that enables this enhanced mental cognition no, for anyone not called Atreides spice just prolongs life; some mentats take sapho juice, but they don't need drugs. .

As I recall, perhaps erroneously, Guild Navigators are nominal humans who owe their ability to move ships to being spiced to their gills.

Yes, I forgot about the Guild Navigators. Technically they don't move the ships, they see the safe path to the destination.

Re: Dune's Butlerian Jihad and the Future of AI

#62

Post by keiferski »

sph wrote: post #57
keiferski wrote: post #12

Earlier quoted context omitted.

Man it annoys me to no end that this basic fact is so under discussed. It’s as if calling something AI is all that’s needed to justify it being the same thing as Skynet, Dune automation, Terminators, Neuromancer, and every other cultural trope using the label of AI. The ultimate fact is that LLMs are a pretty powerful technology that has little to nothing to do with most fictional depictions of intelligent machines.…

We don't have "Skynet, Dune automation, Terminators, Neuromancer, and every other cultural trope using the label of AI" today simply because LLM technology doesn't seem to directly lead to those, but the end goal of AI research is the creation of those. It annoys me to no end that people keep looking at the finger while pointing at the moon. No, we don't have Skynet today, but you can believe Sam Altman would have lo…

We’ll have to agree to disagree then. Personally I think treating sci-fi tropes as likely outcomes is a major distraction from the actual real likely and near term outcomes from LLMs. There are enough real problems there, we don’t need to indulge everyone’s fictional scenarios too.

Re: Dune's Butlerian Jihad and the Future of AI

#63

Post by the8472 »

Sol- wrote: post #19

Doesn't it seem implausible that in the world of Dune they reached their level of technology without AGI? Perhaps all their tech is pre Butlerian Jihad, but even that seems questionable. Seems like if it was adopted as a policy, there would have to be many gray areas where people pretend that they aren't really using AI - the ship's computer is just a domain specific finetuned version, after all, etc. Additionally, t…

Humans come vaguely prealigned due to whatever is encoded in genes and also due to limitations of human bodies that put important constraints on individuals (e.g. no infinitely copyable trusted subagents). Even if you made them superhuman in some aspects a lot of that would still remain. It seems unlikely that minds constructed by a different process would end up humanlike because they lack the evolutionary path-dependencies that shaped humans. Current models appear somewhat human due to imitation learning/pretraining, but A) this could be deceptive as we don't know what's going on inside B) history has shown that that imitation learning becomes unnecessary once RL becomes good enough (e.g. AlphaGo -> AlphaZero), meaning we might end up with minds created from random initialization.

Re: Dune's Butlerian Jihad and the Future of AI

#64

Post by GlibMonkeyDeath »

GeoAtreides wrote: post #60
GlibMonkeyDeath wrote: post #51

Earlier quoted context omitted.

The Bene Gesserit tried to solve alignment through their breeding schedule, but Lady Jessica threw a wrench into it by bearing a son (Paul) out of love for Duke Leto. Paul wasn't aligned with the Bene Gesserit, and hilarity ensued. So I agree, I don't see why thinking machines are a worse alignment problem than enhanced humans. See also the Wrath of Khan. I try not to think too deeply about it, or I won't enjoy the s…

They had advanced computing, a whole region was full of it: IX. The rest of the empire was too feudal and too religious to dabble into advanced technology. And the other players (the BG, the Tleilalxu, the Guild) were against it as well. What good are drones, except as weapons of assassination? Can't use them in an conventional war, there are shields. And they're very expensive, remember, the Great Houses don't have…

>What good are drones, except as weapons of assassination? Can't use them in an conventional war, there are shields.

Presumably a drone could get close enough to push a blade through the shield (maybe use a rocket booster or some other advanced "suspensor" tech) at low-ish speed (a knife would have to travel through the shield on the order of a human reaction time, otherwise even knives wouldn't work. That's about 0.1s/0.1 m, so 1 m/s - ish.)

Re: Dune's Butlerian Jihad and the Future of AI

#65

Post by Sol- »

GlibMonkeyDeath wrote: post #51
Sol- wrote: post #19

Doesn't it seem implausible that in the world of Dune they reached their level of technology without AGI? Perhaps all their tech is pre Butlerian Jihad, but even that seems questionable. Seems like if it was adopted as a policy, there would have to be many gray areas where people pretend that they aren't really using AI - the ship's computer is just a domain specific finetuned version, after all, etc. Additionally, t…

The Bene Gesserit tried to solve alignment through their breeding schedule, but Lady Jessica threw a wrench into it by bearing a son (Paul) out of love for Duke Leto. Paul wasn't aligned with the Bene Gesserit, and hilarity ensued. So I agree, I don't see why thinking machines are a worse alignment problem than enhanced humans. See also the Wrath of Khan. I try not to think too deeply about it, or I won't enjoy the s…

You're right about the suspension of disbelief necessary to enjoy the story, of course. I just felt like nitpicking because people seem to take the Butlerian Jihad seriously as a possible way to contend with AI in our real world, which I think asks too much of the story.

> Paul wasn't aligned with the Bene Gesserit, and hilarity ensued.

Hah, great perspective and I agree. Even normal humans are so difficult to align, no wonder it went wrong with a guy with prescient powers.

Re: Dune's Butlerian Jihad and the Future of AI

#66

Post by GeoAtreides »

GlibMonkeyDeath wrote: post #64
GeoAtreides wrote: post #60

Earlier quoted context omitted.

They had advanced computing, a whole region was full of it: IX. The rest of the empire was too feudal and too religious to dabble into advanced technology. And the other players (the BG, the Tleilalxu, the Guild) were against it as well. What good are drones, except as weapons of assassination? Can't use them in an conventional war, there are shields. And they're very expensive, remember, the Great Houses don't have…

>What good are drones, except as weapons of assassination? Can't use them in an conventional war, there are shields. Presumably a drone could get close enough to push a blade through the shield (maybe use a rocket booster or some other advanced "suspensor" tech) at low-ish speed (a knife would have to travel through the shield on the order of a human reaction time, otherwise even knives wouldn't work. That's about 0.…

if it moves slow enough to pass a shield, it moves slow enough to be easily dispatched. And remember, no autonomous behaviour, there has to be human operator. And it's very expensive. It just doesn't scale.

Re: Dune's Butlerian Jihad and the Future of AI

#67

Post by maleldil »

thih9 wrote: post #35
willis936 wrote: post #14

Earlier quoted context omitted.

I really despise when the term technocrat is misused. It has a very specific meaning in politics and is nowhere near "broligarch", "plutocrat from the tech industry", or any of the thousand different, legitimate ways to describe what people sometimes mean when they mistakenly say "technocrat". How can a political opinion be taken seriously when this extremely basic level of literacy on the topic is missing?

Not to "well ackchyually" your "well ackchyually", but language evolves. I guess in a few years we'll consider it just another instance of semantic shift, perhaps with a splash of folk etymology[1]. [1]: https://en.wikipedia.org/wiki/Folk_etymology

> but language evolves

Totally, but we can try to resist it. Technocrat is a perfectly fine word, and I'd hate to see it go away permanently from its current meaning.

Re: Dune's Butlerian Jihad and the Future of AI

#69

Post by sph »

keiferski wrote: post #62
sph wrote: post #57

Earlier quoted context omitted.

We don't have "Skynet, Dune automation, Terminators, Neuromancer, and every other cultural trope using the label of AI" today simply because LLM technology doesn't seem to directly lead to those, but the end goal of AI research is the creation of those. It annoys me to no end that people keep looking at the finger while pointing at the moon. No, we don't have Skynet today, but you can believe Sam Altman would have lo…

We’ll have to agree to disagree then. Personally I think treating sci-fi tropes as likely outcomes is a major distraction from the actual real likely and near term outcomes from LLMs. There are enough real problems there, we don’t need to indulge everyone’s fictional scenarios too.

Sci-fi tropes are not black and white. It's a spectrum from what exists today to the impossible. 20 years ago many would have claimed being able to have a machine generate decent code from a sentence is science fiction.

When talking about artificial superintelligences, you don't want to study their potential effect on humanity the day they appear, because it is too late.

Re: Dune's Butlerian Jihad and the Future of AI

#70

Post by [deleted] »

maleldil wrote: post #67
thih9 wrote: post #35

Earlier quoted context omitted.

Not to "well ackchyually" your "well ackchyually", but language evolves. I guess in a few years we'll consider it just another instance of semantic shift, perhaps with a splash of folk etymology[1]. [1]: https://en.wikipedia.org/wiki/Folk_etymology

> but language evolves Totally, but we can try to resist it. Technocrat is a perfectly fine word, and I'd hate to see it go away permanently from its current meaning.

[deleted]
Post reply on HN