Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Open Philanthropy Project awards a grant of $30M to OpenAI

openphilanthropy.org

Post reply on HN
Print view View original

81–90 of 186 posts

Re: Open Philanthropy Project awards a grant of $30M to OpenAI

#81

Post by rspeer »

nbouscal wrote: post #70
joshuamorton wrote: post #63

Earlier quoted context omitted.

You don't need to prove anything about an AGI system if you can prove things about its I/O capabilities. I recognize that proving things about machine learning models is very hard, I tried to do some research in that area and got practically no where. And I think you and I have very different definitions of Rigorous. Like I said, unless you'd take Wilbur Wright's thoughts on a mars mission seriously, I don't think yo…

I really, really don't think modern AI capabilities and AGI are as dissimilar as airplanes and rockets. Neither do many AI researchers. You're obviously welcome to disagree, but you don't just get to declare their opinions invalid because you think they're out of their depth.

I don't think AI researchers are "out of their depth" (I am one myself). And yes, there is room for different opinions.

However, I believe that the ones who say sensational things about AI doomsday are the ones who are disproportionately quoted by the media.

Re: Open Philanthropy Project awards a grant of $30M to OpenAI

#83

Post by backpropaganda »

jonmc12 wrote: post #27

When OpenAI was announced, they mentioned having $1B in funding. Why the additional $30M?

The $1B was just promised funding, not immediate. This $30M could be immediate. This makes me believe that perhaps the original funding is being dropped due to lack of results.

Re: Open Philanthropy Project awards a grant of $30M to OpenAI

#84

Post by backpropaganda »

nbouscal wrote: post #72
idlewords wrote: post #59

Earlier quoted context omitted.

The cult of 'AI risk' is a religious belief that brooks no dissent, and unfortunately has its claws in a lot of otherwise smart people.

The cult of 'AI will be totally fine' is a religious belief that brooks no dissent, and unfortunately has its claws in a lot of otherwise smart people.

That's a strawman. People who scoff at the "AI risk" cult don't believe that "AI will be totally fine". They just believe that it doesn't deserve the undue amount of attention it gets, which distracts from other more important and related risks, such as what Cambridge Analytica did for Trump, fake news due to internet, Facebook's bubbles, etc. The belief is to solve problems we have right now, or will have in 10-20 years, not the problems we'd have in 100 years.

Re: Open Philanthropy Project awards a grant of $30M to OpenAI

#85

Post by joshuamorton »

rspeer wrote: post #81
nbouscal wrote: post #70

Earlier quoted context omitted.

I really, really don't think modern AI capabilities and AGI are as dissimilar as airplanes and rockets. Neither do many AI researchers. You're obviously welcome to disagree, but you don't just get to declare their opinions invalid because you think they're out of their depth.

I don't think AI researchers are "out of their depth" (I am one myself). And yes, there is room for different opinions. However, I believe that the ones who say sensational things about AI doomsday are the ones who are disproportionately quoted by the media.

This is also true. You see all of these things like Hawking and Musk talking about the AI Apocalypse (when they aren't even experts in the field), and it gets people scared when, at this point, there's very little reason to be.

On the other hand, I don't actually know Hinton's opinion on these things, and he might agree with me (in which case, you absolutely should listen to him!). But instead the loudest voices are perhaps the most ridiculous.

That said, I do think that if I asked you to make a bet with me on when we would approach with even 50% confidence, your error bars would be on the order of a century.

Re: Open Philanthropy Project awards a grant of $30M to OpenAI

#86

Post by Smaug123 »

joshuamorton wrote: post #50
nbouscal wrote: post #37

Earlier quoted context omitted.

That's… not at all true? There have been numerous thorough attempts at coming up with as reasonable an estimate as possible, involving in-depth conversations with most of the experts in the field. Maybe it's not strictly a "calculation", but it's a hell of a lot more than just an appeal to the idea of AGI.

Except that there's no such thing as the field of "AGI" (or if there is, its a subfield of philosophy). Asking modern ML and Deep Learning researchers about their thoughts on AGI is like asking the wright brothers about a mars mission. Sure, they're the closest thing we have to "experts", but there's not just one, but likely 5 or 10 more field, if not world-changing leaps we need to make before the technologies we ha…

"We can formally prove what a system is capable of."

This is not true. By Rice's theorem, either the system is too dumb to be useful, or we can't prove what it can do.

Re: Open Philanthropy Project awards a grant of $30M to OpenAI

#87

Post by joshuamorton »

Smaug123 wrote: post #86
joshuamorton wrote: post #50

Earlier quoted context omitted.

Except that there's no such thing as the field of "AGI" (or if there is, its a subfield of philosophy). Asking modern ML and Deep Learning researchers about their thoughts on AGI is like asking the wright brothers about a mars mission. Sure, they're the closest thing we have to "experts", but there's not just one, but likely 5 or 10 more field, if not world-changing leaps we need to make before the technologies we ha…

"We can formally prove what a system is capable of." This is not true. By Rice's theorem, either the system is too dumb to be useful, or we can't prove what it can do.

An airgapped computer is not going to be capable of taking over the world.

Further, Turing completeness is not required to be "useful". You can get to the moon without Turing completeness.

Re: Open Philanthropy Project awards a grant of $30M to OpenAI

#88

Post by Noos »

idlewords wrote: post #59
mankash666 wrote: post #3

I think there are more important causes than "reducing potential risks from advanced AI". Honest to god, $30M will go a long way in saving lives TODAY. Flint, MI anyone?

The cult of 'AI risk' is a religious belief that brooks no dissent, and unfortunately has its claws in a lot of otherwise smart people.

You can't get rid of human religious longing easily. Instead of the rapture and armageddon, we have transhumanism and ai risk.

Re: Open Philanthropy Project awards a grant of $30M to OpenAI

#89

Post by nbouscal »

backpropaganda wrote: post #84
nbouscal wrote: post #72

Earlier quoted context omitted.

The cult of 'AI will be totally fine' is a religious belief that brooks no dissent, and unfortunately has its claws in a lot of otherwise smart people.

That's a strawman. People who scoff at the "AI risk" cult don't believe that "AI will be totally fine". They just believe that it doesn't deserve the undue amount of attention it gets, which distracts from other more important and related risks, such as what Cambridge Analytica did for Trump, fake news due to internet, Facebook's bubbles, etc. The belief is to solve problems we have right now, or will have in 10-20 y…

I've been personally told by multiple different people who scoff at AI risk that it will be totally fine, so it's not a straw man.

The median expert estimate for when we'll be 10% likely to have human-level AI is ~10 years.

AI risk research didn't receive a penny of funding until the last few years, and is still funded at way lower levels than a lot of things that have dramatically less impact.

In nearly every debate on the topic I've seen (with a few exceptions), the people concerned about AI risk have carefully considered the topic, are aware of the areas where there's still a lot of uncertainty, and make clear and well-hedged arguments that acknowledge that uncertainty; meanwhile the people who scoff at it haven't read any of the arguments (not even in popular book form in Superintelligence), haven't thought about most of the considerations, and have a general air of "assuming things will probably be fine". That's not a straw man, that's just direct observation of the state of the debate. People are doing serious academic work on the topic and have thought about it very deeply; the standard HN middlebrow dismissal is both common and inappropriate.

Re: Open Philanthropy Project awards a grant of $30M to OpenAI

#90

Post by riffraff »

joshuamorton wrote: post #49
sillysaurus3 wrote: post #46

Earlier quoted context omitted.

1 mil / year per expert Is this realistic? I don't think it's impossible for a dev to be pulling $1M/yr in total comp, but it seemed more likely happen at Google or FB rather than AI.

Note that this says "per expert". I'm not sure what exactly you'd consider an "expert", but I think a reasonable definition of "expert" could result in your average expert being a million-dollar-a-year engineer.

Does a thing like a million a year engineer exist?

I feel like this number would make more sense for a pool of researchers with a strong lead than a single person.

Post reply on HN