The New Yorker did an article on Bostrom and the Future of Humanity Institute last year:
http://www.newyorker.com/magazine/2015/11/23/doomsday-invent...
Superintelligence: The Idea That Eats Smart People
471–480 of 607 posts
Re: Superintelligence: The Idea That Eats Smart People
#472It puzzles me that people think that the nanosecond some superintelligence comes into being, that it also has the capacity to destroy humans/earth/whatever. No thought is given as to how this intellect gets said capacity, bar 'launch the nukes'.
Seriously, if "it" turned against "us", we'd have the upper hand. For example, quality electronics are hard to make without sending African children down into mines and having African adults shoot each other over the results (ie: conflict minerals). If a superintelligence is reliant on us to propagate its physical-world interactions, we're going to be just fine.
I mean come on, we all work in IT, and we all know just how difficult it is to keep hardware running securely, safely, and in good order. Stuff fails all the time. Similarly, we all know people who are really intelligent, but this doesn't translate to success in life for them.
In short, "being intelligent" isn't enough - the entity also needs ways to effectively work the world around them.
edit: heh, just saw another article on the front page: "IQ is only a minor factor in success"
Re: Superintelligence: The Idea That Eats Smart People
#473Earlier quoted context omitted.
Exactly. The "Terminator" scenario of a rogue malfunctioning AI is a silly distraction from the real AI threat, which is military AIs that don't malfunction. They will give their human masters practically unlimited power over everyone else. And AI is not the only technology with the potential to worsen inequality in the world.
Human beings have been extremely easy to kill for our entire existence. No system of laws can possibly keep you alive if your neighbors are willing to kill you, and nothing can make them actually unable to kill you. Your neighbor could walk over and put a blade in your jugular, you're dead. They could drive into you at 15MPH with their car, you're dead. They could set your house on fire while you're asleep, you're de…
That's not true, if I know my neighbors are coming I'm loading my Remington 870 and I'm waiting for that door to open. In America we're allowed to bear arms for protection.
Re: Superintelligence: The Idea That Eats Smart People
#474Why is everyone afraid of artificial intelligence? I'm more afraid of natural infinite human stupidity. Superintelligence would just balance that out ;-) . But seriously here is my critique of Nick Bostroms arguments about superintelligence: https://asmaier.blogspot.de/2015/06/superintelligence-critiq...
Re: Superintelligence: The Idea That Eats Smart People
#475While I agree with Maciej's central point, I think the inside arguments he presents are pretty weak. I think that AI risk is not a pressing concern even if you grant the AI risk crowd's assumptions. Elided from https://alexcbecker.net/blog.html#against-ai-risk : The real AI risk isn't an all-powerful savant which misinterprets a command to "make everyone on Earth happy" and destroys the Earth. It's a military AI that…
No doubt that his "inside arguments" have been rebutted extensively by the strong AI optimists and their singularity priests. After all, dreaming up scenarios in which robotic superintelligence dominates humanity is their version of saving the world.
That's why I found the "outside arguments" here equally important and compelling.
> The outside view doesn't care about content, it sees the form and the context, and it doesn't look good.
If it sounds like and acts like a cult, why should we treat it any differently from a cult? Even if the people in it are all very smart, wealthy, well-dressed, and appear very rational, they're still preaching the end of the world on a certain date. All of those groups have only one thing in common: they're all wrong.
The best rebuttals to all this are the least engaging.
"Dude, are you telling me you want to build Skynet?"
Re: Superintelligence: The Idea That Eats Smart People
#476I am one of those who is convinced that AI will destroy humanity and Elons/others efforts will not help.
Its hard for me to argue against the fact that this idea isn't eating me.
Re: Superintelligence: The Idea That Eats Smart People
#477While I agree with Maciej's central point, I think the inside arguments he presents are pretty weak. I think that AI risk is not a pressing concern even if you grant the AI risk crowd's assumptions. Elided from https://alexcbecker.net/blog.html#against-ai-risk : The real AI risk isn't an all-powerful savant which misinterprets a command to "make everyone on Earth happy" and destroys the Earth. It's a military AI that…
No doubt that his "inside arguments" have been rebutted extensively by the strong AI optimists and their singularity priests. After all, dreaming up scenarios in which robotic superintelligence dominates humanity is their version of saving the world. That's why I found the "outside arguments" here equally important and compelling. > The outside view doesn't care about content, it sees the form and the context, and it…
> If it sounds like and acts like a cult, why should we treat it any differently from a cult? Even if the people in it are all very smart, wealthy, well-dressed, and appear very rational, they're still preaching the end of the world on a certain date. All of those groups have only one thing in common: they're all wrong.
Occasionally the crazies are right. Remember when the idea that the NSA was recording everyone's emails was paranoid conspiracy theory talk, except turns out they were actually doing this the whole time.
The fact that the world hasn't ended tells us virtually nothing about how likely the end of the world is, for the simple reason that if the world had ended we wouldn't be here to talk about it. So we can't take it as evidence, at least, not conclusive evidence. Note also that the same argument works just as well against global warming as it does against AI risk.
Turn it around. Suppose there was a genuinely real risk of destroying the world. How would you tell? How would you distinguish between the groups that had spotted the real danger and the run-of-the-mill end-of-the-world cults?
Re: Superintelligence: The Idea That Eats Smart People
#478Shouldn't a superintelligence be smart enough to become another Gandhi? Why not?
Re: Superintelligence: The Idea That Eats Smart People
#479This article explicitly endorses argument ad hominem: "These people are wearing funny robes and beads, they live in a remote compound, and they speak in unison in a really creepy way. Even though their arguments are irrefutable, everything in your experience tells you you're dealing with a cult. Of course, they have a brilliant argument for why you should ignore those instincts, but that's the inside view talking. Th…
OK, so I ignore Einstein, and I miss general relativity. And then what? If it's proven true before I die, then I accept it; if it isn't, or if it is and I continue to ignore it, I die anyway. And then it's 2015 and it's being taught to schoolchildren. High-school-educated people who don't really know the first damn thing about physics, like non-hypothetical me, still have a rough idea what relativity is, and the repe…
If it were just a disagreement about physics then it would be safe to ignore.
But in this case, if they're right then we're about to wipe out humanity. That's not safe to ignore.
Re: Superintelligence: The Idea That Eats Smart People
#480Earlier quoted context omitted.
No doubt that his "inside arguments" have been rebutted extensively by the strong AI optimists and their singularity priests. After all, dreaming up scenarios in which robotic superintelligence dominates humanity is their version of saving the world. That's why I found the "outside arguments" here equally important and compelling. > The outside view doesn't care about content, it sees the form and the context, and it…
> If it sounds like and acts like a cult, why should we treat it any differently from a cult? Even if the people in it are all very smart, wealthy, well-dressed, and appear very rational, they're still preaching the end of the world on a certain date. All of those groups have only one thing in common: they're all wrong. Occasionally the crazies are right. Remember when the idea that the NSA was recording everyone's e…
The point that I was trying to bring attention to is that one's perception of the AI risk movement changes substantially once you turn your focus from content to form and context. He brings many examples of this (the cult in the robes, Art Bell, etc.)
I wouldn't call suspecting the government of surveilling people "crazy." Claiming you know the timeframe for the apocalypse with precision is different. I am from North Carolina -- try a Google Image search for "May 21, 2011."
Ray Kurzweil believes he will never die. Balancing content with form and context, what we have here is clearly an atheistic, scientist version of "May 21, 2011."