Open Philanthropy Project awards a grant of $30M to OpenAI • Hacker News Board

Re: Open Philanthropy Project awards a grant of $30M to OpenAI

#131

Post by backpropaganda » Sat, Apr 01, 2017, 2:38 PM UTC

nbouscal wrote: post #89

backpropaganda wrote: post #84

Earlier quoted context omitted.

That's a strawman. People who scoff at the "AI risk" cult don't believe that "AI will be totally fine". They just believe that it doesn't deserve the undue amount of attention it gets, which distracts from other more important and related risks, such as what Cambridge Analytica did for Trump, fake news due to internet, Facebook's bubbles, etc. The belief is to solve problems we have right now, or will have in 10-20 y…

I've been personally told by multiple different people who scoff at AI risk that it will be totally fine, so it's not a straw man. The median expert estimate for when we'll be 10% likely to have human-level AI is ~10 years. AI risk research didn't receive a penny of funding until the last few years, and is still funded at way lower levels than a lot of things that have dramatically less impact. In nearly every debate…

You're piling strawman on top of strawman, while suffering from confirmation bias. You think the AI risk guys are making well-hedge arguments, because you believe in that thesis.

Andrew Ng, Yann LeCun, and many other people who have ACTUALLY worked in AI are the ones who scoff at it. They don't need to make arguments because what do you respond to a young earth creationist?

All of the arguments in Supreintelligence or otherwise are simply that AI will eventually exist. The only argument for that it will come by 2050 is a badly conducted survey of non-experts.

Should we worry about all sorts of existential risk which could arrive in any undetermined time in the future?

The whole project is so absurd, it's hard even to begin to make any counter arguments, because none of the arguments make any sense.

Re: Open Philanthropy Project awards a grant of $30M to OpenAI

#132

Post by icebraining » Sat, Apr 01, 2017, 2:42 PM UTC

nbouscal wrote: post #130

icebraining wrote: post #113

Earlier quoted context omitted.

That number didn't pass my sniff test, so I went looking. It seems to have come from here[1], which aggregates a series of surveys. I first opened the "FHI Winter Intelligence" report: it's an informal survey made to 35 participants of a conference, of which only 8 work on AI at all (let alone be an expert in AGI). I then looked at the "Kruel interviews", which the site reports as giving a prediction of "2025" for 10…

Sorry, gave a citation in another comment on the thread but not in this one. I was referencing http://sophia.de/pdf/2014_PT-AI_polls.pdf

Are they actually experts, though? From that paper:

  “Concerning the above questions, how would you describe your own expertise?”
  (0 = none, 9 = expert)
  − Mean 5.85

  “Concerning technical work in artificial intelligence, how would you describe your own expertise?”
  (0 = none, 9 = expert)
  − Mean 6.26

Also, the whole methodology of aggregating the opinions of random conference attendees seems suspect to me. Attending a conference doesn't make you an expert.

Re: Open Philanthropy Project awards a grant of $30M to OpenAI

#133

Post by andrewem » Sat, Apr 01, 2017, 2:50 PM UTC

jaibot wrote: post #104

idlewords wrote: post #57

Scroll to the end. This is a $30M grant to the guy's roommate and future brother-in-law. Unbelievable.

I want to make sure I fully understand the accusation here. You're saying that the Open Philanthropy Fund - which is funded by an $8.3 billion grant from Dustin Moskovitz and Cari Tuna, also close associates - is funneling $30M money to an organization that pays below market rates ( https://www.quora.com/What-is-compensation-like-at-the-non-p... ), run by people who have dedicated their professional careers and milli…

You make a lot of points. For the moment I'll take issue with only one of them. You claim OpenAI is "an organization that pays below market rates", and give a link to support your claim. The first thing in that link is "OpenAI compensation is similar to industry compensation.", which means it's basically "market rate". Later that link says "OpenAI does not pay absolute top of market", but arguing from that sentence that OpenAI doesn't pay market rate is like saying for example Apple doesn't pay market rates because there's a hedge fund which pays more.

So I believe the source you cited indicates the opposite of what you claim it does.

Re: Open Philanthropy Project awards a grant of $30M to OpenAI

#134

Post by nbouscal » Sat, Apr 01, 2017, 3:23 PM UTC

backpropaganda wrote: post #131

nbouscal wrote: post #89

Earlier quoted context omitted.

I've been personally told by multiple different people who scoff at AI risk that it will be totally fine, so it's not a straw man. The median expert estimate for when we'll be 10% likely to have human-level AI is ~10 years. AI risk research didn't receive a penny of funding until the last few years, and is still funded at way lower levels than a lot of things that have dramatically less impact. In nearly every debate…

You're piling strawman on top of strawman, while suffering from confirmation bias. You think the AI risk guys are making well-hedge arguments, because you believe in that thesis. Andrew Ng, Yann LeCun, and many other people who have ACTUALLY worked in AI are the ones who scoff at it. They don't need to make arguments because what do you respond to a young earth creationist? All of the arguments in Supreintelligence o…

Is this a badly conducted survey of non-experts? http://sophia.de/pdf/2014_PT-AI_polls.pdf

Edit: In particular, the TOP100 subgroup.

Re: Open Philanthropy Project awards a grant of $30M to OpenAI

#135

Post by nbouscal » Sat, Apr 01, 2017, 3:26 PM UTC

icebraining wrote: post #132

nbouscal wrote: post #130

Earlier quoted context omitted.

Sorry, gave a citation in another comment on the thread but not in this one. I was referencing http://sophia.de/pdf/2014_PT-AI_polls.pdf

Are they actually experts, though? From that paper: “Concerning the above questions, how would you describe your own expertise?” (0 = none, 9 = expert) − Mean 5.85 “Concerning technical work in artificial intelligence, how would you describe your own expertise?” (0 = none, 9 = expert) − Mean 6.26 Also, the whole methodology of aggregating the opinions of random conference attendees seems suspect to me. Attending a co…

You can restrict your attention to the TOP100 group if you prefer.

Re: Open Philanthropy Project awards a grant of $30M to OpenAI

#136

Post by backpropaganda » Sat, Apr 01, 2017, 3:34 PM UTC

nbouscal wrote: post #134

backpropaganda wrote: post #131

Earlier quoted context omitted.

You're piling strawman on top of strawman, while suffering from confirmation bias. You think the AI risk guys are making well-hedge arguments, because you believe in that thesis. Andrew Ng, Yann LeCun, and many other people who have ACTUALLY worked in AI are the ones who scoff at it. They don't need to make arguments because what do you respond to a young earth creationist? All of the arguments in Supreintelligence o…

Is this a badly conducted survey of non-experts? http://sophia.de/pdf/2014_PT-AI_polls.pdf Edit: In particular, the TOP100 subgroup.

Yes. The other 3 groups were not AI researchers. And the TOP100 group had a response rate of 29%, which adds self-selection to the process. People who are interested in AI risk et al. are more likely to respond to such a survey, adding bias. Andrew Ng or Yann LeCun or anyone actually working in AI would have refused (and probably did refuse) the invitation. Also, this TOP100 group of people are also more likely to be GOFAI folks who pretty much have no idea about the current data-driven deep learning-based AI.

Even Stuart Russel, the only CS guy in the AI risk camp, doesn't actually believe that AGI is anywhere near. But he works on it simply because he thinks we can do solve some of the problems like learning from demonstrations instead of (possibly faulty) rewards. That's actually a core AI research topic, not a AI ethics/values/blahblah topic. Oh, and also because this allows him to have a differentiated research program, and thus directs any funding on this niche to him.

Re: Open Philanthropy Project awards a grant of $30M to OpenAI

#137

Post by icebraining » Sat, Apr 01, 2017, 3:36 PM UTC

nbouscal wrote: post #135

icebraining wrote: post #132

Earlier quoted context omitted.

Are they actually experts, though? From that paper: “Concerning the above questions, how would you describe your own expertise?” (0 = none, 9 = expert) − Mean 5.85 “Concerning technical work in artificial intelligence, how would you describe your own expertise?” (0 = none, 9 = expert) − Mean 6.26 Also, the whole methodology of aggregating the opinions of random conference attendees seems suspect to me. Attending a co…

You can restrict your attention to the TOP100 group if you prefer.

Yeah, but then you just have 29 responses in total.

Re: Open Philanthropy Project awards a grant of $30M to OpenAI

#138

Post by backpropaganda » Sat, Apr 01, 2017, 3:57 PM UTC

icebraining wrote: post #137

nbouscal wrote: post #135

Earlier quoted context omitted.

You can restrict your attention to the TOP100 group if you prefer.

Yeah, but then you just have 29 responses in total.

From the first paragraph of Turing's famous essay on AI:

I propose to consider the question, "Can machines think?" This should begin with definitions of the meaning of the terms "machine" and "think." The definitions might be framed so as to reflect so far as possible the normal use of the words, but this attitude is dangerous, If the meaning of the words "machine" and "think" are to be found by examining how they are commonly used it is difficult to escape the conclusion that the meaning and the answer to the question, "Can machines think?" is to be sought in a statistical survey such as a Gallup poll.

Re: Open Philanthropy Project awards a grant of $30M to OpenAI

#139

Post by nbouscal » Sat, Apr 01, 2017, 4:13 PM UTC

icebraining wrote: post #137

nbouscal wrote: post #135

Earlier quoted context omitted.

You can restrict your attention to the TOP100 group if you prefer.

Yeah, but then you just have 29 responses in total.

The word "just" feels awfully out of place in this context. If you were doing some kind of broad-based polling of public opinion, of course you'd want a bigger sample size, but 29 of the top 100 researchers in a field sounds like a hell of a good sample to me, and well worth listening to.

Re: Open Philanthropy Project awards a grant of $30M to OpenAI

#140

Post by nbouscal » Sat, Apr 01, 2017, 4:14 PM UTC

backpropaganda wrote: post #136

nbouscal wrote: post #134

Earlier quoted context omitted.

Is this a badly conducted survey of non-experts? http://sophia.de/pdf/2014_PT-AI_polls.pdf Edit: In particular, the TOP100 subgroup.

Yes. The other 3 groups were not AI researchers. And the TOP100 group had a response rate of 29%, which adds self-selection to the process. People who are interested in AI risk et al. are more likely to respond to such a survey, adding bias. Andrew Ng or Yann LeCun or anyone actually working in AI would have refused (and probably did refuse) the invitation. Also, this TOP100 group of people are also more likely to be…

Seems pretty clear neither of us will convince the other in this venue, so I'm going to leave it here. Thanks for the discussion :)