Machine intelligence, part 1

It's kind of unfortunate that the spectre of "AI doom" has blown up in the popular press the way it has, in that it's forced a lot of top people to push back against caricatured, sensationalist arguments in a way that distracts from the more serious conversations that ought to be happening.

Most of these people you cited actually have more nuanced views on the future of AI, e.g., LeCun and Bengio are (along with many others) signatories to the "Research priorities for robust and beneficial artificial intelligence" document (http://futureoflife.org/misc/open_letter, http://futureoflife.org/static/data/documents/research_prior...), which encourages research into AI's social and economic effects as well as research into technical questions of designing provably valid, secure, and controllable AI systems. Unfortunately when the conversation becomes "Stephen Hawking says we shut down AI research now before it dooms humanity", it becomes harder to express the belief that there are potential consequences of increasing machine intelligence that are worth understanding and planning for, even if we are not currently anywhere close to building superhuman general intelligences.

For what it's worth, expert opinion is not unified on AI risk in the way that it is on, say, climate change. First of all, most AI researchers don't spend their days thinking about long-term risks, so their main reaction is "the stuff I'm working on right now is such a toy that it has no chance of ever becoming a threat to humanity", which is probably true, but doesn't really make them an expert on the topic. Of the researchers who actually do think about this stuff, Stuart Russell is at least one example of someone who believes that there are serious questions to be asked (http://www.cs.berkeley.edu/~russell/research/future/, http://edge.org/conversation/the-myth-of-ai#26015):

"None of this proves that AI, or gray goo, or strangelets, will be the end of the world. But there is no need for a proof, just a convincing argument pointing to a more-than-infinitesimal possibility. ... No one in the field is calling for regulation of basic research; given the potential benefits of AI for humanity, that seems both infeasible and misdirected. The right response seems to be to change the goals of the field itself; instead of pure intelligence, we need to build intelligence that is provably aligned with human values."

Hopefully once the popular fearmongering dies down a bit, the field can move towards more of a sane, thoughtful conversation regarding what if anything we should be doing to prepare for the potential arrival of very intelligent (specialized or general) machines in our lives.

Yeah, I'm not talking about morality at all here. Our technological prowess, resulting from the application of our intelligence, has enabled us to wipe out entire species.

Here is the thing about all of those, none of them give concrete step by step examples of how we would get to "unfriendly" or "friendly" AGI respectively. Now, you might say - well of course because we don't know how an AGI will be built yet. Right! That is exactly the point.

At this point all of the MIRI (Former SIAI) staffers have stopped trying to explain paths to unfriendly AGI and have been just assuming them for their solutions. Just look at Yudkowski's work a few years ago on CEV and you'll see how any attempt at formalization falls apart.

Superintelligence just takes from the starting point they seeded in Global Catastrophic Risks, but doesn't actually address it in any more detail IMO. "Our Final Invention" tried also to do this but failed as well.

To be clear the AGI community understands thoroughly the risks inherent in AGI - I would venture to say far more-so than the non-researcher. So they are worth exploring, but at a certain point you have to say "We aren't sure if the atmosphere will catch on fire or not but lets try it anyway because it is the next logical step."

At some point we need to have the hard conversation about what humans will do in a world where we are less relevant.

Or just stop pursuing it altogether and put a moratorium on AGI research. A terrible terrible idea in my opinion.

How about Jeremy Howard, former President of crowd-ML-competition site Kaggle and now founder of ML-medicine company Enlitic?

http://www.reddit.com/r/Futurology/comments/2p6k20/im_jeremy...

The opinions of leading-edge researchers are valuable but not necessarily dispositive. Often, it benefits a researcher to be oblivious to the larger or darker implications of their work. Boundless optimism and a maniacal focus on only-what-is-knowable-right-now delivers publishable/actionable results, all the other 'speculation' is genuinely a "distraction" for them. That doesn't mean it's a distraction for us.

Often the speculators and entrepreneurs ("billionaires and pundits") are more talented than researchers at projecting trends and social-economic interactions.

> I hope you aren't a researcher

Andrej is a brilliant researcher, currently doing his PhD. He has a bright career ahead.

Maybe you should actually read his comment instead of dismissing it crudely, and likewise for the thoughts of the likes of LeCun, Ng, Bengio, etc. These are the people I would listen to, not the Nostradamus pundits.