Stop Fearing Artificial Intelligence

>It has an “I,” a sense of self distinct from others.

I deny this is required for a program to be dangerous.

>It has the intellectual capacity to step outside of the boundaries of its intended purpose and programming to form radically new goals for itself (the “I”).

Deny this as well; even carrying out programmed goals can be harmful. In the words of Superintelligence (copied from https://intelligence.org/2015/01/08/brooks-searle-agi-voliti...):

>[W]e cannot blithely assume that a superintelligence with the final goal of calculating the decimals of pi (or making paperclips, or counting grains of sand) would limit its activities in such a way as not to infringe on human interests. An agent with such a final goal would have a convergent instrumental reason, in many situations, to acquire an unlimited amount of physical resources and, if possible, to eliminate potential threats to itself and its goal system.

>It has access to resources on a global scale to carry out the plan.

This sounds reasonable. As http://slatestarcodex.com/2015/04/07/no-physical-substrate-n... points out, humans have made billions anonymously, just by being connected to the internet. http://lesswrong.com/lw/k4h/request_for_concrete_ai_takeover... gives some ways an AI could takeover.

Sure, humans exist, and we can already make humans by combining DNA from a man and a woman :) That method does not lead to superhuman intelligence.

What makes you think we could devise some other method to make human intelligence? Have we demonstrated any progress toward building humans?

What we have demonstrated is extremely rapid progress in making computers and software systems that can take in a huge amount of data and make decisions that look intelligent (but are still brittle and trivially broken by humans). I think this path will lead us to many great things in the future, and it will have a lot of risks that are more important to think about then whether computers will overthrow us.

Computers and humans are completely different things. If you watch a child grow up, this is obvious. A child can do things that are impossible for a computer, while it can't speak, read, or write. A computer can do things that no human could ever do. They are qualitatively different.

I also want to bring up a related point: AI advocates always seem to bring up "simulating the universe" on a computer. Nobody has shown that this is possible. In fact most evidence suggests it is unlikely. The universe is not a computer.

Computers, human brains, and the universe are just different things, and I see a deluge of sloppy thinking connecting them. There are of course relationships between them, but they should be made precise. To do so often require considerable technical expertise, like the author of this article apparently has.

As always, Peter Thiel has a great contrarian viewpoint in his book "Zero to One". The future is in human + machine intelligence. They both do different things, and the sum of them is very powerful. I predict this will be the story for the next 50 years, and general AI / strong AI won't enter into it.

We already have this problem, where decisions that should be made by the humans involved are restricted under the "policy" excuse of bureaucracy, blind to the reality of unusual situations that require an exception in the usual routine. I fear AI because it is harder to fight complicated systems. I fear AI because - like the dubious "studies" that came before it, it will become a tool to obfuscate and hide bias and prejudice.

It's strange to me to think as humans we will remain at the top of the food chain forever. As such, it would seem a fault of our genetic makeup thinking nothing would capable of knocking us off that top rung of the evolutionary ladder.

It's like we can't fathom building something that would undermine our own existence, or downplay the notion that AI would have some sort of bad intentions since we built it.