He was also warning against the followers, myths, institutions, and power structures that turn a talented person into an untouchable savior. Sounds familiar?
Dune's Butlerian Jihad and the Future of AI
21–30 of 84 posts
Re: Dune's Butlerian Jihad and the Future of AI
#22Seems a bit premature given the current state of word generators.
I guess top mathematicians are also mere word generators? https://openai.com/index/model-disproves-discrete-geometry-c...
Being a human may be sufficient (with important exceptions) to play chess, but playing chess well is not sufficient to be a human.
Likewise with math, it turns out, it appears. Likewise with most other things we classify as needing intelligence, I bet. It turns out that intelligence is not the same as humanity or sentience etc..
Re: Dune's Butlerian Jihad and the Future of AI
#23Of course the article mixes things up in order to push a cheap narrative. Herbert's Butlerian Jihad was a rehash of Islamic ban on images of people that is supposed to prevent idolatry and preserve uniqueness of God. It's purely religious in nature and has nothing to do with "tools of oppression". Not to mention that absolutely any automation, even the simplest kind, was considered a thinking machine and was banned,…
The tools of oppression reading is backed directly by the books: "But that only permitted other men with machines to enslave them" Is it also true that Fremen are canonically descendants of Zensunni practitioners, which is a mix of Zen and Sunni Islam Many other restrict machines, and the dogma is encoded on a religious text called the Orange Catholic Bible. And, this might be a big jump but, some of the specific ant…
Not a very deep justification. Probably the easiest thing to grasp for when thinking "how can I make this religious theme fit the narrative?".
Re: Dune's Butlerian Jihad and the Future of AI
#24People talk about a Butlerian Jihad against AI as if you could just ban LLMs and be done. I bet some govermenst would like to do that for ordinary people. They can ban visible products (chatbots, public APIs, GPAI services), and laws are already targeting those. but you cant ban the math! the same transformer/attention ideas work great as compressors, function approximators, and surrogate models in physics, chemistry…
> Would be interesting whether it is possible to write a LLM-like program just using compression and function interpolation algoritms.
gzip can be used as a (not very good) LLM-like text and image generator: https://arxiv.org/abs/2309.10668
Re: Dune's Butlerian Jihad and the Future of AI
#25Of course the article mixes things up in order to push a cheap narrative. Herbert's Butlerian Jihad was a rehash of Islamic ban on images of people that is supposed to prevent idolatry and preserve uniqueness of God. It's purely religious in nature and has nothing to do with "tools of oppression". Not to mention that absolutely any automation, even the simplest kind, was considered a thinking machine and was banned,…
This is a flattening of Herbert's angle - it is explicitly stated in the quoted section in the article that part of the reason for the ban is that the people who control thinking machines would exert outsized control on those who depend on them. This is a recurring theme in Dune, as we see with anxieties about dependency on mentats and Bene Gesserit truthsayers (the latter of which are in fact exerting hidden influen…
> The mentats are theoretically safer because of their imperial conditioning, but even that can be tampered with, as we see with Dr Yueh
Dr Yueh was not a mentat, but a suk doctor who was subject to conditioning. (Which was broken by the Piter de Vries, via the pain amplifiers applied to his wife, Wanna.)
Paul himself was being trained to be a mentat, and there were no hints of conditioning there, neither with Paul, Hawat, or de Vries (albeit he was described as a "twisted mentat", whatever that means).
Re: Dune's Butlerian Jihad and the Future of AI
#26Earlier quoted context omitted.
When "guess with some magic" can solve long-standing problems in mathematics that no human had been able to, it seems fair to ask whether it's approaching risky levels of intelligence.
I mean we've been using technology to solve long-standing problems in mathematics that no human had been able to do for years. Is Deep Blue sentient because it can beat any human at chess?
Deep Blue's strength was leveraging massive compute to execute a task-specific, human-written algorithm. The problems which LLMs are tackling don't elude mathematicians because they require too much number-crunching, but because they demand creative problem solving. The latter seems more profound, even if it doesn't imply sentience.
Re: Dune's Butlerian Jihad and the Future of AI
#27I think it is unlikely that people will act to completely abolish technology once they are used to it in their everyday lives.
This is called technological momentum.
Re: Dune's Butlerian Jihad and the Future of AI
#28Earlier quoted context omitted.
The tools of oppression reading is backed directly by the books: "But that only permitted other men with machines to enslave them" Is it also true that Fremen are canonically descendants of Zensunni practitioners, which is a mix of Zen and Sunni Islam Many other restrict machines, and the dogma is encoded on a religious text called the Orange Catholic Bible. And, this might be a big jump but, some of the specific ant…
Not a very deep justification. Probably the easiest thing to grasp for when thinking "how can I make this religious theme fit the narrative?".
It seems exactly the opposite to me: Herbert didn't want robots etc in Dune, so he came up with a religious figleaf to justify this.
Re: Dune's Butlerian Jihad and the Future of AI
#29Earlier quoted context omitted.
This is a flattening of Herbert's angle - it is explicitly stated in the quoted section in the article that part of the reason for the ban is that the people who control thinking machines would exert outsized control on those who depend on them. This is a recurring theme in Dune, as we see with anxieties about dependency on mentats and Bene Gesserit truthsayers (the latter of which are in fact exerting hidden influen…
> The mentats are theoretically safer because of their imperial conditioning, but even that can be tampered with, as we see with Dr Yueh Dr Yueh was not a mentat, but a suk doctor who was subject to conditioning. (Which was broken by the Piter de Vries, via the pain amplifiers applied to his wife, Wanna.) Paul himself was being trained to be a mentat, and there were no hints of conditioning there, neither with Paul,…
Hey, you're right, I don't think mentats get the conditioning as part of their training. I must have misremembered Thufir having the diamond tattoo.
Re: Dune's Butlerian Jihad and the Future of AI
#30The article had a great opportunity to at least reference the newest encyclical from the Pope.
With a bit of click bait, this might be the first step of "Butlerian Jihad" -> "AI Crusade" and a founding text for the Orange CATHOLIC Bible.
I saw a take that you can now cite religious reasons to refuse working on and with AI - if more people try this, I wonder how it'd play out.
Interesting times.