Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Introducing OpenAI

openai.com

Post reply on HN
Print view View original

271–280 of 392 posts

Re: Introducing OpenAI

#271

Post by Houshalter »

zxcvvcxz wrote: post #196

Man I dunno about some of this media hype surrounding the topic of AI. I understand how powerful ML/AI algorithms are for general pattern matching (with a big enough computer, gradient descent can learn a lot of things...), but this whole skynet/doomsday fear thing seems ridiculous. I guess the risk is embedding into systems that manage missiles or something. But you don't need sophisticated algorithms for that to be…

Survey's of AI experts give a median prediction that we will have human level AI within 30 years. And a non-trivial probability of it happening in 10-20 years. They are almost unanimous in predicting it will happen within this century.

Re: Introducing OpenAI

#272

Post by Houshalter »

karmacondon wrote: post #123
mikepalmer wrote: post #99

Earlier quoted context omitted.

'The "evil" AI and safety stuff is just science fiction, but whatever.' umm... you can offer proof that we have nothing to worry about? Does the proof go like: Just as all people are inherently good, therefore all AIs will be inherently good? Or is it more like: since we can now safely contain all evil people, therefore we will be able to safely contain evil AIs? Sounds to me like there is some risk, no?

As I've said many times on HN over the years, there is currently no clear path to science fiction like "AI". To return your question, hopefully without being rude, is there any proof that AI capable of having a moral disposition will ever exist? Andrew Ng (I believe) compared worrying about evil AI to worrying about overpopulation on Mars. Which is to say, the problem is so far off that it's rather silly to be consid…

How are you estimating how far away AI is so accurately that you can disregard it entirely? The best we can do to predict these things is survey experts, and the results aren't too comforting: http://www.nickbostrom.com/papers/survey.pdf

>We thus designed a brief questionnaire and distributed it to four groups of experts in 2012/2013. The median estimate of respondents was for a one in two chance that highlevel machine intelligence will be developed around 2040-2050, rising to a nine in ten chance by 2075. Experts expect that systems will move on to superintelligence in less than 30 years thereafter. They estimate the chance is about one in three that this development turns out to be ‘bad’ or ‘extremely bad’ for humanity.

Re: Introducing OpenAI

#273

Post by andreyf »

richardw wrote: post #187

How to prevent Future ISIS from getting Future AI, or do we just shift from us trying to out-think them to our AI trying to out-think their AI? If the answer to the latter is "resources" then we're back where we started. Whoever has the biggest AI wins. The picture seems to be of many AI's all keeping each other in check, but that outcome seems less likely to result in the AI-UN and more like a primordial soup of com…

ISIS does not have access to many CS researchers nor server farms, as far as I am aware.

Re: Introducing OpenAI

#274

Post by johann28 »

jychang wrote: post #208
johann28 wrote: post #170

Earlier quoted context omitted.

> OpenAI's research director is Ilya Sutskever, one of the world experts in machine learning. Our CTO is Greg Brockman, formerly the CTO of Stripe. The group's other founding members are world-class research engineers and scientists: Trevor Blackwell, Vicki Cheung, Andrej Karpathy, Durk Kingma, John Schulman, Pamela Vagata, and Wojciech Zaremba. Pieter Abbeel, Yoshua Bengio, Alan Kay, Sergey Levine, and Vishal Sikka…

> Abbeel is at Stanford's AI lab. Uhhh.... https://www.google.com/search?q=Pieter+Abbeel That's a lot of results showing how he's been a professor at Berkeley since 2008. He received his PhD at Stanford, then went to be a professor at Berkeley.

His website didn't load for some reason so I just went with the Google hit's title. Maybe that was his old page.

Re: Introducing OpenAI

#275

Post by Jach »

astrofinch wrote: post #219
vonnik wrote: post #150

> Musk: I think the best defense against the misuse of AI is to empower as many people as possible to have AI. If everyone has AI powers, then there’s not any one person or a small set of individuals who can have AI superpower. In a sense, we have no other defense. AI is just math and code, and I know of no way to distinguish good linear algebra from evil linear algebra. The barriers to putting that math and code tog…

It might be useful to differentiate between technological equilibria that favor either attackers or defenders. * Two guys with pistols in a crowded bar: Attacker is favored. * Trench warfare during WWI: Defender is favored. * Nukes: Attacker is favored, although with the invention of nuclear submarines that could lurk under the ocean and offer credible retaliation even in the event of a devastating strike, the attack…

Have a read through Military Nanotechnology: Potential Applications and Preventive Arms Control by Jürgen Altmann. If you have 30 mins, maybe check out this talk as an optional prelude: http://www.youtube.com/watch?v=MANPyybo-dA

I suspect that after reading you'll be convinced if you aren't already that in the realms of biological and chemical warfare (special cases of nanotech warfare) nature overwhelmingly favors offense. We've been able to worldwide keep research and development on those limited, and there's incentive to avoid it anyway since if word gets out others will start an arm's race so they can at least try and maybe get to a MAD equilibrium, but it's not nearly as stable as the one with nukes. An additional incentive against is that the only purpose of such weaponry is to annihilate rather than destroy enough to achieve a more reasonable military objective.

But molecular nanotech is on a completely different playing field. Fortunately it's still far out, but as it becomes more feasible, there is a huge incentive to be the first entity to build and control a universal molecular assembler or in general self-replicating devices. Arms control over this seems unlikely.

Giving everyone their own AGI is like giving everyone their own nation state, which implies being like giving everyone their own nuke plus the research personnel to develop awesome molecular nanotech which as a special case enables all the worst possibilities of biological and chemical and non-molecular-nanotech nanotech warfare, and then more with what you can do with self replication, most of which are graver existential threats than nukes. Absent a global authority or the most superintelligent Singleton to monitor everything that situation is in no way safe for long.

Re: Introducing OpenAI

#276

Post by whazor »

nicolashahn wrote: post #222
athenot wrote: post #183

Earlier quoted context omitted.

> If he believes in the potential harm of AI, then supporting its widespread use doesn't seem logical. If you take the quote above, and substitute the word "guns" for "AI", you basically have the NRA, and the NRA is not making the world a safer place. Guns are not exactly good at healing, making or creating. A better comparison would be knives. Knives can be used for stabbing and killing but also for sustenance (cook…

You're right, guns are pure evil. Clearly, we should take them out of the hands of cops, bodyguards, hunters, and civilians defending themselves.

He does not state that guns are evil. Also, where I live bodyguards and civilians are not allowed to have guns. Many cops do not even carry guns. So for me it is hard to identify the comparison to guns.

Re: Introducing OpenAI

#277

Post by bengpertzel »

mark_l_watson wrote: post #68

Sounds great. I was hoping for OpenCog to be a good open source AI framework but is is difficult to work with (good team; I have worked with several of them in the past, no criticism intended). I look forward to seeing how OpenAI uses outside contributions, provides easy to use software and documentation, etc.

Hi Mark, I think OpenAI is an exciting initiative.

OpenCog is a bit different because it's founded on a specific approach to building AGI. I realize OpenCog is kind of a pain to work with at present, and we hope to fix that during the coming year....

But I see OpenCog and OpenAI as complementary initiatives, really.... OpenAI's mandate is more broad and generic in terms of fostering and funding open-source AGI research, which is wonderful ... but OpenAI does not come along with a specific, coherent AGI design from what I can see.... Quite possibly OpenAI will end up funding stuff that is used in OpenCog, or even funding some work on OpenCog directly, down the road...

For that matter, if I had a billion dollars, I wouldn't put it all into OpenCog, I would also fund a variety of OSS projects in AI and other important domains of science and engineering...

Interesting times ;)

Re: Introducing OpenAI

#278

Post by sageikosa »

gmac wrote: post #259
nicolashahn wrote: post #222

Earlier quoted context omitted.

You're right, guns are pure evil. Clearly, we should take them out of the hands of cops, bodyguards, hunters, and civilians defending themselves.

Ironically, you're absolutely right. Cops, bodyguards and civilians defending themselves generally only need guns because their adversaries have guns. Just take them out of everyone's hands. I know this works, if you can make it happen, because I've seen the gun death statistics for countries with effective gun control. Hunters are a different case, but their weapons are rather different too. To be honest I wouldn't…

We'll ask nicely. Please don't shoot.

Re: Introducing OpenAI

#279

Post by tim333 »

zxcvvcxz wrote: post #196

Man I dunno about some of this media hype surrounding the topic of AI. I understand how powerful ML/AI algorithms are for general pattern matching (with a big enough computer, gradient descent can learn a lot of things...), but this whole skynet/doomsday fear thing seems ridiculous. I guess the risk is embedding into systems that manage missiles or something. But you don't need sophisticated algorithms for that to be…

>skynet/doomsday fear thing seems ridiculous

Maybe but better safe than sorry. Here's a scenario that OpenAI could protect against - super intelligent AI is first built by the US military, patented and backdoored by the NSA to protect us against terrorism. Then some one evil hacks them and turns them against us. Super intelligent AI being open source would reduce such risks.

Re: Introducing OpenAI

#280

Post by astrofinch »

cwp wrote: post #262
Moshe_Silnorin wrote: post #207

Earlier quoted context omitted.

Ecosystems are hellish malthusian processes rife with extinction. I'm unsure why they think an ecosystem of superhuman agents will be any different. What sort of selective pressures would exist in such a world that would ensure the survival of humanity? It's pretty naive to think a competitive ecology would select for anything more than very intelligent entities that value replication.

Physics. You're right that ecosystems are brutal. That's exactly why I'm not worried about AI as an existential threat to humanity. A few years back Bill Joy was sounding the alarm on nanotechnology. He sounded a lot like Elon Musk does today. Nanobots could be a run-away technology that would reduce the world to "grey goo". But nothing like that will ever happen. The world is already awash in nanobots. We call them…

Advanced nanotechnology is not the only possible way to achieve power: http://slatestarcodex.com/2015/04/07/no-physical-substrate-n...
Post reply on HN