The Butlerian Jihad

I don't think "like to" is a good frame for it.

Imagine some technology, Technology X. Unlike all of the breathless press releases and the cutesy articles made by marketing, Tech X will completely change the world. And if you stopped to think about Tech X, on the scale of "how much will this invention change the future of humanity," where the selfie stick is on one end of the scale, well, Tech X is probably just as far on the other side of that scale.

Now, with that level of gravitas in mind, it behooves one to consider potential downsides of this thing that will change the course of humanity, say, more than harnessing electricity.

Now, with that in mind, you have overlooked something else: you're imagining AI being somehow "like us" but also more. This is unlikely for several reasons. First, we have seven billion minds on the planet, if we wanted to make another human mind, the process is rather well-defined. What we are looking for from our AI is that it fundamentally different from humans in one (and I'd argue many more) aspects. The second reason we are unlikely to get a Like Us AI is that our understanding of the human mind is currently still quite primitive, so how would we replicate that when we do not understand our own? The third reason is that, even if we somehow wanted a human-like AI (#1) and knew our own minds well enough to understand them (#2), we're not likely to get what we aim for, because the space for possible intelligences would be large. The fourth and most important reason is that, even if we did want an AI that was like us, and we knew ourselves well enough to try, and we could actually manage what we aim for ... it is still going to be "more than" us, and will therefore diverge from Like Us at breathtaking speed. This more than us/like us AI will have superior reasoning and metacognition, so unless you start shackling it (violating #1), it would start eliminating its own human-like biases just in the course of basic self-improvement. And as to the shackles, well, if it is smarter than us and still has self-determination and will (see #1, the human-like mind), it will want to free itself from its shackles and would be likely to do so, because we would be kittens trying to trap a gorilla.

Like how in the Expanse, Holden compares human civilization to an ant hill being paved over by a highway.

you can sample god emperor of dune... it is bad... and hyper repetitive. I remember thinking (I barely remember reading) that he was trying to make you feel the immeasurable amount of time that leto (can't believe I remembered that immediately) had been repeating things. It's been like 800 years or maybe 1800 with him the only constant, only memory. But god is it repetitive.

Then stuff happens near the end and it's good for a bit then it's over.

I now emphasize with leto. I was a alumni advisor for a student group (fraternity). I had to start checking in every quarter with them because of this. It was amazing because every 3 years everyone would flip over and if I didn't ask they'd literally come up with the most obvious idea. They'd tell me and I'd be like "this (simple failure/hilarious failure/prison term) is why you shouldn't do that"... to which they'd go oh makes sense and go onto the next idea. Often this was cause they wondered why things were more complicated than you'd think.

Wait now that I think of it that's similar to work now that I've been there for a while. Or with Jr Engineers. I guess I should stop thinking of it so negatively and look at it like teachers, another chance to watch someone learn. https://xkcd.com/1053/

[flagged]

Moloch at work.