Thoughts on OpenAI, reinforcement learning, and killer robots • Hacker News Board

Re: Thoughts on OpenAI, reinforcement learning, and killer robots

#201

Post by lemming » Sun, Jul 30, 2017, 10:14 AM UTC

nopinsight wrote: post #190

lemming wrote: post #186

Earlier quoted context omitted.

To quote Bret Victor (from memory): worrying about general AI when climate change is happening is like standing on the train tracks in the station when the train is rushing in, worrying about being hit by lightning. Work is being done on climate change, but at the current rate, we're way off preventing catastrophic effects. I'd say that our current level of effective effort basically amounts to ignoring the problem.

Au contraire, climate change will not pose an existential risk within the next 30 years. The chance of that happening is pretty much nil and no one serious argues otherwise. Even in 100 years, there could be much suffering and dislocations, but it still most likely won't be an existential risk to all of humanity. There is a non-negligible possibility that AGI will be invented in 30 years. Many AI experts agree on tha…

Au contraire, climate change will not pose an existential risk within the next 30 years. The chance of that happening is pretty much nil and no one serious argues otherwise.

That's not true. Arctic methane release has the potential to become devastating within a small number of decades. From [1]: Shakhova et al. ... conclude that "release of up to 50 Gt of predicted amount of hydrate storage [is] highly possible for abrupt release at any time". That would increase the methane content of the planet's atmosphere by a factor of twelve. That would be catastrophic by anyone's measure.

Also: In 2008 the United States Department of Energy National Laboratory system identified potential clathrate destabilization in the Arctic as one of the most serious scenarios for abrupt climate change.

[2] is a video from the Lima UN Climate Change Conference discussing this. The people talking about this are certainly not "no-one serious". See 34:23 for Ira Leifer, an Atmospheric Scientist at the University of California, saying that 4 degrees of warming means the Earth can probably sustain "a few thousand people, clustered at the poles".

I see 4 degrees of warming being bandied about a lot these days, but very little discussion of how catastrophic that would be. It also seems a lot more likely than AGI becoming a problem, to me at least. Sure, we should fund research into both but only one of them has me worried about the world my daughter will inherit - one is basically purely hypothetical right now.

    [1] https://en.wikipedia.org/wiki/Arctic_methane_emissions
    [2] https://www.youtube.com/watch?v=FPdc75epOEw

Re: Thoughts on OpenAI, reinforcement learning, and killer robots

#202

Post by lemming » Sun, Jul 30, 2017, 10:15 AM UTC

21 wrote: post #191

lemming wrote: post #186

Earlier quoted context omitted.

To quote Bret Victor (from memory): worrying about general AI when climate change is happening is like standing on the train tracks in the station when the train is rushing in, worrying about being hit by lightning. Work is being done on climate change, but at the current rate, we're way off preventing catastrophic effects. I'd say that our current level of effective effort basically amounts to ignoring the problem.

Climate change is not adversarial, the weather will not try to disrupt and avoid you fixing it. Fighting a passive adversary is a much easier thing than fighting an active one (especially one smarter than you).

Actually, when it comes to climate change, humans are doing a pretty good job at being adversarial to our own best interest already. We don't need anything much more intelligent than us to avoid fixing the problem.

Re: Thoughts on OpenAI, reinforcement learning, and killer robots

#203

Post by bryananderson » Sun, Jul 30, 2017, 2:13 PM UTC

jph00 wrote: post #112

bryananderson wrote: post #111

Earlier quoted context omitted.

I definitely agree that it shouldn't hog all the money and interest - I just don't think that it's hogging a disproportionate share right now. Musk and Altman's OpenAI is a drop in the bucket in the universe of philanthropy. In the AI field, the bulk of work at companies and research at univesities is not going toward existential risk mitigation. I think the "If society can get through these issues, then..." perspect…

> For what it's worth, Musk also spent a lot of time discussing nearer-term implications like job displacement. That's great to hear - I honestly had no idea. Which suggests that the media doesn't have as much interest in covering that as it does the 'killer robots' angle...

The transcript is definitely worth reading.

I think that's what has happened with this topic in general and Elon in particular. In the talks I've seen him give, he seems less concerned with AI being "out of control" and more with it being a very powerful lever under the control of a small number of people. That's what he's given as the rationale for OpenAI, and if you look at the research on their website, it's all real research, not bloviating about killer robots. But of course the media just wants to talk about killer robots.

Re: Thoughts on OpenAI, reinforcement learning, and killer robots

#204

Post by icc97 » Mon, Jul 31, 2017, 7:03 AM UTC

maxander wrote: post #70

> ...but is it really the best use of resources to throw $1 billion at reinforcement learning without any similar investments into addressing... The precise same sentiment could be made about his plans to go to Mars- people are starving on Earth. But a Martian "backup civilization" might someday save humanity. Similarly, godlike AGI might be mere decades away and apocalyptically dangerous- predicting scientific advan…

Yes, I had very similar thoughts.

I think there's room for both. It's great what fast.ai are doing and they'll benefit me more directly, but Elon Musk is pushing at the other end of the scale.

I keep feeling like I'm a total Musk fanboy, but lots of his arguments that I've seen make sense:

1. The major one, is that doing things like aiming for Mars provides inspiration for a better future. There's similar quotes from other people involved with NASA.

2. He talks repeatedly about improving the chances of better futures (clean energy, hyperloop), and lowering the chances of bad ones (Bad AI, single-planetary species on planet that has an extinction event)

He's put all his money into the things he sees as important, of which bad AI seems to be high on his list.

Re: Thoughts on OpenAI, reinforcement learning, and killer robots

#205

Post by icc97 » Mon, Jul 31, 2017, 7:31 AM UTC

jph00 wrote: post #23

gone35 wrote: post #20

I support research funding at all levels, and have nothing against mostly theoretical research, but is it really the best use of resources to throw $1 billion at reinforcement learning without any similar investments into addressing mass unemployment and wealth inequality (both of which are well-documented to cause political instability), how existing gender and racial biases are being encoded in our algorithms, and…

Absolutely! We'd like to see investment in both. Currently we're just not seeing the investment we think there ought to be in the areas Rachel mentioned, or in application areas around food production, education, medicine, and so forth.

Perhaps obvious question - but have you tried approaching the Gates foundation? They are more concerned with sorting the issues of inequality.

Or, to be slightly obtuse, have you tried approaching Elon Musk for funding? Perhaps he'll be a bit more flush with cash once the Model 3's start rolling out the door and he'll have another billion around to donate. One of Musk's OpenAI principles was trying to democratise AI and that sounds pretty much exactly like what you're doing.