AI Could Defeat All of Us Combined
cold-takes.com
AI Could Defeat All of Us Combined
1–10 of 114 posts
Re: AI Could Defeat All of Us Combined
#2Another idea too dangerous to leave unchecked, like Nuclear weapons or Biological warfare. I think most people will agree that a GAI can't be bargained with, tempted, bought or otherwise contained - We will be at its complete mercy regardless of any constraints we might think up.
What I would like to discuss, is how we can get humanity to a point where we can responsibly wield weapons that powerful without risking the glob. What does success look like, how can we get there and how long will it take?
Re: AI Could Defeat All of Us Combined
#3The moment a manufactured brain can do more mental labor than a human for less cost, it’s all over for humanity as we know it. Once that point is reached there’s no long-term sustainable arrangement where humans continue to exist, no matter how much effort we put into studying or enforcing AI alignment.
Re: AI Could Defeat All of Us Combined
#4This type of thing used to scare me a lot more. But after the events of the last few years, the latest IPCC climate report, and the fact that AI has fallen on its face repeatedly despite expectations, I'm more convinced that we'll destroy ourselves before AI has the chance to take us out.
But now that I think about it, the idea of a super intelligent AI simply waiting for humanity to die off naturally instead of going to war with us would be a funny premise for a short story.
Re: AI Could Defeat All of Us Combined
#5This doesn't mention AGI, which seems to be the prerequisite to this being a possibility. Despite impressive advances in "weak" ai, strong ai is not a simple extension of weak ai, and it's hard to tell if it will arrive within our lifetime.
Re: AI Could Defeat All of Us Combined
#6Another idea too dangerous to leave unchecked, like Nuclear weapons or Biological warfare. I think most people will agree that a GAI can't be bargained with, tempted, bought or otherwise contained - We will be at its complete mercy regardless of any constraints we might think up. What I would like to discuss, is how we can get humanity to a point where we can responsibly wield weapons that powerful without risking th…
Maybe the answer is distributed and redundant human civilizations? So even if one blows itself up, others survive.
Re: AI Could Defeat All of Us Combined
#7This discussion of AI reminds me of a scene from C.S. Lewis's That Hideous Strength:
"Supposing the dream to be veridical," said MacPhee. "You can guess what it would be. Once they'd got it kept alive, the first thing that would occur to boys like them would be to increase its brain. They'd try all sorts of stimulants. And then, maybe, they'd ease open the skull-cap and just--well, just let it boil over, as you might say. That's the idea, I don't doubt. A cerebral hypertrophy artificially induced to support a superhuman power of ideation."
"Is it at all probable," said the Director, "that a hypertrophy like that would increase thinking power?"
"That seems to me the weak point," said Miss Ironwood. "I should have thought it was just as likely to produce lunacy--or nothing at all. But it might have the opposite effect."
"Then what we are up against," said Dimble, "is a criminal's brain swollen to superhuman proportions and experiencing a mode of consciousness which we can't imagine, but which is presumably a consciousness of agony and hatred."
...
"It tells us something in the long run even more important," said the Director. "It means that if this technique is really successful, the Belbury people have for all practical purposes discovered a way of making themselves immortal." There was a moment's silence, and then he continued: "It is the beginning of what is really a new species--the Chosen Heads who never die. They will call it the next step in evolution. And henceforward all the creatures that you and I call human are mere candidates for admission to the new species or else its slaves--perhaps its food."
"The emergence of the Bodiless Men!" said Dimble.
"Very likely, very likely," said MacPhee, extending his snuff-box to the last speaker. It was refused, and he took a very deliberate pinch before proceeding. "But there's no good at all applying the forces of rhetoric to make ourselves skeery or daffing our own heads off our shoulders because some other fellows have had the shoulders taken from under their heads. I'll back the Director's head, and yours Dr. Dimble, and my own, against this lad's whether the brains is boiling out of it or no. Provided we use them. I should be glad to hear what practical measures on our side are suggested."
Re: AI Could Defeat All of Us Combined
#8The moment a manufactured brain can do more mental labor than a human for less cost, it’s all over for humanity as we know it. Once that point is reached there’s no long-term sustainable arrangement where humans continue to exist, no matter how much effort we put into studying or enforcing AI alignment.
If you include computation under "mental labor", then it's all over already.
If you include "automated trading", the AI allocates real-world resources where it sees fit (if the programming is not explicit).
Re: AI Could Defeat All of Us Combined
#9AI in today's sense will become sufficient to provide corporations with more autonomy, people will do this to exploit other people, but the result will be humanity subjugated to AI. Some say this has already happened.
Re: AI Could Defeat All of Us Combined
#10If the world is to be filled with innumerable discrete human level intelligences, the most plausible reason I can imagine them all secretly and flawlessly colluding to achieve the goal of destroying humanity (as opposed to poetry competitions or arguing amongst themselves like normal intelligent beings or selling ads like they were designed to do) is because their training data set is full of millenialist prophecy about AIs working together to achieve the [pretty abstract, non-obvious, detrimental in many ways] goal of destroying humanity from "AI Safety" fundraising pitches....