Transcending Complacency on Superintelligent Machines

We get superhuman performance on most tasks we use computers for now; people forget that easily. Not too long ago, before electronic calculators, we used people as calculators; it was a good job for which you needed a brain; they would consider computers now superhuman. A Watson hooked up to Asimo presented a few 100 years back (I don't think you actually would have to go back that far; my grandparents would not see it as less than human, probably more) would be considered god himself.

And we are on HN, mostly smart people here who vastly over estimate 'normal people'. It's nice that we (me included) assume a human can be taught to be able to do anything other humans can (with some margins), but for now this is not true either. And if we want this empirical evidence thing going on; if a (kind of) Turing test would be done with a large part of the population who have not been told they are, for instance, we let a human with earplugs walking around a village in Arkansas and walk up to an average person and play the human interaction for Watson (or something like it), it would usually succeed. It would in my village for 100% sure; I could actually make a knowledge based script for talking to a lot of people and they would not see the difference. So I understand what you mean, but I don't think in a chinese room kind of way (and that experiment, as many have shown, doesn't matter) we are not that far off. When we reach your level of input/output you will 'see it' but still, because you don't have a definition, will deny it. I would wager that we are there are already in the 'fuzzy' sense of at least 40% (I think it's a lot more) of the population. My grandparents, bless them, definitely think they are talking to a human when they call they book a railway ticket (which has been a steadily improving AI for a little under 20 years now); for their 'fuzzy' it's been solved and strong AI exists.

I'm no expert in this area so I welcome corrections to my rough numbers below.

    Approximate number of human neurons:    1.0e11
    Approximate number synapses in a human: 1.0e14

These are big numbers, but not impossibly big numbers. There are different kinds of neurons, and signals on synapses are not simply binary. However, even with these complications, the hardware needed to reach these scales isn't hard to imagine.

    Transistors in XBox One:                1.0e09

Brains are biological computers so they suffer from very slow switching speeds at the neural level. Neurons run in parallel, but they are not fast:

    Approximate neural switching speed:     1.0e03/sec

Even if all of the synapeses could sustain this rate in parallel (they can't) and even if all of the brain was 100% occupied with solving a single task (it isn't) this would mean that we absolutely can't compute faster than:

    Speed * synapses (brain ops per second): 1.0e17/sec

For comparison, the fastest bitcoin hardware I see is advertised to operate at the following speed:

    Minerscube 15 (hashes per second):      1.5e12/sec

And a regular GPU is capable of simple instructions that run at the following speed:

    AMD Radeon HD 6990 32-bit instructions: 2.6e12/sec

From this we can see that hardware is catching up with the raw computing ability of the human brain. Now consider the problem of programming a brain. It isn't necessary to program every synapse. The brain learns, and essentially programs itself. To see why this is true consider the programming that we are born with:

    Bits of information in human genome:    1.0e10

This is far less than the number of synapses that we have. Therefore, the brain must program itself, somehow.

Now, to address the argument about evolution taking millions of years. First, we can evolve programs much faster than nature can evolve humans. There have been, perhaps, 100 million generations of humans. Even if it takes six seconds of computing time to run an evolutionary computation for a single generation it will take no more than 20 years to run over 100 million generations.

Brute force evolution isn't the only way to build strong AI. A program can exhibit behavior that we don't anticipate. I've written simple programs that beat me easily at games such as Othello or Freecell.

Finally, once machines get smart enough to design other machines there may be a rapid acceleration of progress in this area as we employ them in designing subsequent generations.

I feel that strong AI may pose a significant risk to humans; consequently, we should proceed with caution. Here is a thought experiment. If a chimpanzee could be taught to drive, would you trust it to pick your kids up from school? What sort of value judgements would it make in the case of an impending emergency? Would you let an elephant baby sit for you? Even if was much "smarter" than a normal elephant?

Strong AI will not be like us. It will learn and develop without a human body, and it will not interact with the world and society as we do and may end up being very foreign to us. Will it be sociopathic? Or will it be like whales, intelligent, but mysterious, perhaps spending all its time singing AI songs to other AIs.

No, I'm not one of those people. And I agree with basically everything you've said here (except for the part about the possibilities of AI and the limitations of mystery - but we've covered that!). I find the closed-mindedness of many people so disappointing. I think asking these questions is so important. I also find others' experiences with religion fascinating. Thank you for sharing. My experience was extremely different - and the community I was (and am) part of doesn't strike me as closed-minded. As you note, there are many religious people who don't see a conflict between science and religion. I count myself as one of them.