Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Open Philanthropy Project awards a grant of $30M to OpenAI

openphilanthropy.org

Post reply on HN
Print view View original

61–70 of 186 posts

Re: Open Philanthropy Project awards a grant of $30M to OpenAI

#61

Post by idlewords »

Analemma_ wrote: post #60
idlewords wrote: post #57

Scroll to the end. This is a $30M grant to the guy's roommate and future brother-in-law. Unbelievable.

idlewords, this comment has tremendous negative utility for humanity. By pointing out obvious nepotism and corruption in the AI risk charity field, you make people less likely to donate in the future, thus increasing the likelihood of a takeover from the robots. You must delete this comment; the fate of the world depends on it! /s

I'll believe it when I hear it from a future superintelligence that has pre-committed to simulating 3^^^^^3 copies of me.

Re: Open Philanthropy Project awards a grant of $30M to OpenAI

#62

Post by idlewords »

jaibot wrote: post #35
mankash666 wrote: post #3

I think there are more important causes than "reducing potential risks from advanced AI". Honest to god, $30M will go a long way in saving lives TODAY. Flint, MI anyone?

"We have to defuse this bomb or it will destroy the city in one hour!" "But there are people suffering right now !"

Twist: the bomb is a cartoon someone drew on a napkin

Re: Open Philanthropy Project awards a grant of $30M to OpenAI

#63

Post by joshuamorton »

nbouscal wrote: post #54
joshuamorton wrote: post #50

Earlier quoted context omitted.

Except that there's no such thing as the field of "AGI" (or if there is, its a subfield of philosophy). Asking modern ML and Deep Learning researchers about their thoughts on AGI is like asking the wright brothers about a mars mission. Sure, they're the closest thing we have to "experts", but there's not just one, but likely 5 or 10 more field, if not world-changing leaps we need to make before the technologies we ha…

Your opinion about how far off AGI is is a totally reasonable one, and some AI experts agree with it. Other AI experts disagree, and think that AGI is basically tractable with current research methods plus a breakthrough or two. Yes, there's some additional uncertainty about whether you're even asking the right people. But you can take account of that uncertainty, both by widening your error bars and by asking people…

You don't need to prove anything about an AGI system if you can prove things about its I/O capabilities. I recognize that proving things about machine learning models is very hard, I tried to do some research in that area and got practically no where.

And I think you and I have very different definitions of Rigorous. Like I said, unless you'd take Wilbur Wright's thoughts on a mars mission seriously, I don't think you should give much thought to Geoff Hinton's thoughts about when we'll get AGI, and I say that having enormous respect for him and his achievements. (and using him as a simple example)

Its pseudoscience. We're notoriously bad about predicting the future. I don't see any reason to trust people going on about the dangers of AGI any more than the futurologists of my parents generation, who predicted flying cars and interstellar travel, but missed smartphones.

Re: Open Philanthropy Project awards a grant of $30M to OpenAI

#64

Post by btown »

idlewords wrote: post #61
Analemma_ wrote: post #60

Earlier quoted context omitted.

idlewords, this comment has tremendous negative utility for humanity. By pointing out obvious nepotism and corruption in the AI risk charity field, you make people less likely to donate in the future, thus increasing the likelihood of a takeover from the robots. You must delete this comment; the fate of the world depends on it! /s

I'll believe it when I hear it from a future superintelligence that has pre-committed to simulating 3^^^^^3 copies of me.

By the time a simulation of you hears that it will already be too late.

Re: Open Philanthropy Project awards a grant of $30M to OpenAI

#65

Post by idlewords »

dilemma wrote: post #53
Moshe_Silnorin wrote: post #51

Earlier quoted context omitted.

Technological advancement has been responsible for most of the increases in human welfare, most notably allowing us to (at least temporally) escape the Malthusian condition. It's not implausible that technical research likely to be neglected by markets and academia could be a better use of money than even the most efficient African charity. But I drink the AI-is-very-likely-to-change-everything-we-really-mean-it-this…

OpenAI still isn't open source, or open in any other sense of the word. It is a private organization that will own any technology developed. Again, OpenAI isn't Open.

Nor is it AI.

Re: Open Philanthropy Project awards a grant of $30M to OpenAI

#66

Post by BaronSamedi »

nbouscal wrote: post #37
Avshalom wrote: post #25

Earlier quoted context omitted.

>>high impact / low probability EV calculations in other domains Yeah but there isn't a calculation of the risk/probability of AGI there's just the appeal to the idea.

That's… not at all true? There have been numerous thorough attempts at coming up with as reasonable an estimate as possible, involving in-depth conversations with most of the experts in the field. Maybe it's not strictly a "calculation", but it's a hell of a lot more than just an appeal to the idea of AGI.

Experts in the field of AI have a long history of being wrong. "AI is 20 years" away has become the standard joke in the field. There is no reason to grant expert opinion on this topic any weight.

Re: Open Philanthropy Project awards a grant of $30M to OpenAI

#67

Post by schoen »

joshuamorton wrote: post #50
nbouscal wrote: post #37

Earlier quoted context omitted.

That's… not at all true? There have been numerous thorough attempts at coming up with as reasonable an estimate as possible, involving in-depth conversations with most of the experts in the field. Maybe it's not strictly a "calculation", but it's a hell of a lot more than just an appeal to the idea of AGI.

Except that there's no such thing as the field of "AGI" (or if there is, its a subfield of philosophy). Asking modern ML and Deep Learning researchers about their thoughts on AGI is like asking the wright brothers about a mars mission. Sure, they're the closest thing we have to "experts", but there's not just one, but likely 5 or 10 more field, if not world-changing leaps we need to make before the technologies we ha…

> Air gap the computer and control the inputs and outputs. We can formally prove what a system is capable of. That fixes the problem.

Debates about superhuman AI have focused quite a lot on what it would mean to "control the inputs and outputs" while still being able to get some kind of benefit from the AI.

You can indeed formally prove that a computer will or won't do certain things, so you could use that for isolation purposes. But in order to be useful, the AI needs to interact with people and/or the world in some way. Otherwise it might as well be switched off or never have been built in the first place.

If it's really a superhuman intelligence with superhuman knowledge about the world, then interacting with people is where the risk creeps back in, because the AI could make suggestions, recommendations, requests, offers, promises, or threats. Although there are plenty of ideas about limiting the nature of questions and answers, having some kind of separate person or machine judge whether information from the AI's communications should be used or how, or limiting what the AI is programmed to attempt to do or how, none of these measures are straightforward to formally prove correct in the way that simpler isolation properties are.

If we made contact with intelligent aliens, would formal proofs of correctness of the computers through which we (say, exclusively) communicate with them guarantee that they couldn't massively disrupt our society by means of what they had to say?

Re: Open Philanthropy Project awards a grant of $30M to OpenAI

#68

Post by schoen »

joshuamorton wrote: post #63
nbouscal wrote: post #54

Earlier quoted context omitted.

Your opinion about how far off AGI is is a totally reasonable one, and some AI experts agree with it. Other AI experts disagree, and think that AGI is basically tractable with current research methods plus a breakthrough or two. Yes, there's some additional uncertainty about whether you're even asking the right people. But you can take account of that uncertainty, both by widening your error bars and by asking people…

You don't need to prove anything about an AGI system if you can prove things about its I/O capabilities. I recognize that proving things about machine learning models is very hard, I tried to do some research in that area and got practically no where. And I think you and I have very different definitions of Rigorous. Like I said, unless you'd take Wilbur Wright's thoughts on a mars mission seriously, I don't think yo…

> You don't need to prove anything about an AGI system if you can prove things about its I/O capabilities.

But then you can't benefit from it, or you can only benefit from it except in narrowly predefined ways.

Re: Open Philanthropy Project awards a grant of $30M to OpenAI

#69

Post by nbouscal »

BaronSamedi wrote: post #66
nbouscal wrote: post #37

Earlier quoted context omitted.

That's… not at all true? There have been numerous thorough attempts at coming up with as reasonable an estimate as possible, involving in-depth conversations with most of the experts in the field. Maybe it's not strictly a "calculation", but it's a hell of a lot more than just an appeal to the idea of AGI.

Experts in the field of AI have a long history of being wrong. "AI is 20 years" away has become the standard joke in the field. There is no reason to grant expert opinion on this topic any weight.

What's your suggested alternative? It's really easy to criticize, but a lot harder to do any better. If you aren't suggesting any other methods, you're implicitly saying we should just ignore the problem. I really don't think that will end well.

Re: Open Philanthropy Project awards a grant of $30M to OpenAI

#70

Post by nbouscal »

joshuamorton wrote: post #63
nbouscal wrote: post #54

Earlier quoted context omitted.

Your opinion about how far off AGI is is a totally reasonable one, and some AI experts agree with it. Other AI experts disagree, and think that AGI is basically tractable with current research methods plus a breakthrough or two. Yes, there's some additional uncertainty about whether you're even asking the right people. But you can take account of that uncertainty, both by widening your error bars and by asking people…

You don't need to prove anything about an AGI system if you can prove things about its I/O capabilities. I recognize that proving things about machine learning models is very hard, I tried to do some research in that area and got practically no where. And I think you and I have very different definitions of Rigorous. Like I said, unless you'd take Wilbur Wright's thoughts on a mars mission seriously, I don't think yo…

I really, really don't think modern AI capabilities and AGI are as dissimilar as airplanes and rockets. Neither do many AI researchers. You're obviously welcome to disagree, but you don't just get to declare their opinions invalid because you think they're out of their depth.
Post reply on HN