Earlier quoted context omitted.
It is not a tool if you can't control it. Politicians fool countries delivering empty promises about better health, education and security. A supraintelligent AI could promise making humans rich, healthy and powerful, to then break its promise and dominate the world. https://en.wikipedia.org/wiki/AI_box
Well, it would have to have a motivation to do so. Evolution has put complex motivations into human beings for billions of years, self preservation being chief among them. Even if we put motivation for self preservation into an AI, we might not do as well as nature did, leaving the AI open to self destruction or shutdown by humans - simply because the AI has no motivation not to allow humans to turn it off. Human des…
Humans are going to program their AI's to try to make as much money as possible. Many corporations are already mindless and reckless amoral machines that relentlessly try to optimize profits despite any externalities. Try to imagine Exxon, Wal-Mart, and Amazon run by an intelligence beyond human understanding or accountability.