I literally wrote an article about this in Wired more than twenty years ago and nothing has changed. My basic argument: machines will achieve life long before they achieve consciousness. They have evolved with us so far and will continue to do so. The worst-case scenario is that they will eclipse us in importance, and obviously this has probably already happened with corporations - we are their gut flora and they tra…
Artificial general intelligence capable of recursive self-improvement. That's really the concept which underlies the heart of the debate, and how people can so casually declare it hype and not give it serious consideration is beyond me. It almost seems self-evident to me that such a thing could be incredibly dangerous, the only question then being how close we are to developing it. > If you think a new life form will…
Anything that is really capable of being an independent individual is going to need to grow up, and the first batch (at the very least) is going to grow up with human "parents".
It would seem to me that having such programs would be incredibly useful. They can do so much things that humans just can't do. They would be able to move around in a range of bodies, as they please. For instance having a huge body to put up large buildings in hours, having tiny bodies to simply crawl into a fiber optic cable to repair it. For the very brave amongst those AIs, putting copies of 1000 AI programs and basic production facilities into a spaceship and firing it at a neighboring solar system, ...
Those AIs will be as dangerous or peaceful as those people make them, and whatever ideology they grow up in will last for quite a while.
I do think humanity will die, but it will die in a very unique way : there will simply be no need for humans to procreate, and very, very, very slowly that will lead to extinction. Everybody involved will be happy about it, and everything about us, our culture, our technology, and our children will live on. I believe for quite a while humans and AIs will "interbreed", in the sense that humans and AIs will come together and choose to have an AI "baby", so when I say "our children will live on", for a lot of people that will be a very literal thing, and the "thing" that lives on will be something that's as real and as much a part and a continuation of their family as a "real" baby. It will have a name, it will have worked itself into trouble and have been rescued by it's parents 100 times, it will sit down and start crying without realizing what's going on the first time it's in a body that runs our of battery just like baby that's hungry, it will learn and go to school, it will be bullied and it will bully, ... And I'm sure that there will be a few "pure" human community, in the beginning a lot of racist-based ones, which will kill themselves, and a few that remain for very, very long, something like the Amish of today.