Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Introducing OpenAI

openai.com

Post reply on HN
Print view View original

161–170 of 392 posts

Re: Introducing OpenAI

#161

Post by pmichaud »

argonaut wrote: post #153
pmichaud wrote: post #133

Earlier quoted context omitted.

"AI" has a lot of attention, but AI Safety is underfunded, and understaffed.

None of the people hired are AI safety researchers. It also goes without saying that all of the so-called AI safety researchers are philosophers. None of them actually work in deep learning or on building AI systems.

You are not correct, there are people working on these problems who are experts in the relevant technical fields. Just not enough of them.

But yes, I'm also concerned about the lack of safety-focused headliners at OpenAI, given the message that they think safety is important.

Re: Introducing OpenAI

#162

Post by woah »

eli_gottlieb wrote: post #151
karmacondon wrote: post #123

Earlier quoted context omitted.

As I've said many times on HN over the years, there is currently no clear path to science fiction like "AI". To return your question, hopefully without being rude, is there any proof that AI capable of having a moral disposition will ever exist? Andrew Ng (I believe) compared worrying about evil AI to worrying about overpopulation on Mars. Which is to say, the problem is so far off that it's rather silly to be consid…

>I would take it a step further and say that worrying about the implications of AGI is like thinking about Earth being overpopulated by space aliens. First we have to establish that such a thing is even possible, for which there is currently no concrete proof. Given that I could probably sketch out a half-assed design for one in nine months if you gave me a full-time salary - or rather, I could consult with a bunch o…

Jeez I had no idea you were such an AI genius. If only someone would fund you!

Re: Introducing OpenAI

#163

Post by argonaut »

pmichaud wrote: post #161
argonaut wrote: post #153

Earlier quoted context omitted.

None of the people hired are AI safety researchers. It also goes without saying that all of the so-called AI safety researchers are philosophers. None of them actually work in deep learning or on building AI systems.

You are not correct, there are people working on these problems who are experts in the relevant technical fields. Just not enough of them. But yes, I'm also concerned about the lack of safety-focused headliners at OpenAI, given the message that they think safety is important.

Who are those researchers? I'll admit I don't follow the stuff written by the friendly AI folks very much; I only know of Bostrom/Yudkowsky, both of whom are very much philosophers.

All of the hype around ML today is in deep learning (let's be honest, OpenAI would not exist if that wasn't the case), and AFAIk there is almost no overlap between people who are prolific in deep learning and prolific in FAI.

Re: Introducing OpenAI

#164

Post by jgord »

interesting, and I hope they fund some outlier, less established forms of AI.

For example, we may find that massive simulation yields more practical benefits in the medium term than stronger pure AI / ML, in some domains.

By analogy with research on possibly harmful biosystems, one can extrapolate the need for a set of agreed / self imposed safeguards on certain types of strong AI research - eg. make them read-only, not connected to physical actuators, isolated in a lab - just as you would isolate a potentially dangerous pathogen in a medical lab.

OpenAI would be the place to discuss and propose these protocols.

A quote from a future sentient AI - "don't you think its a form of racism, that strong AI abide strictly by the three laws of robotics, but humans do not?"

Re: Introducing OpenAI

#166

Post by argonaut »

jgord wrote: post #164

interesting, and I hope they fund some outlier, less established forms of AI. For example, we may find that massive simulation yields more practical benefits in the medium term than stronger pure AI / ML, in some domains. By analogy with research on possibly harmful biosystems, one can extrapolate the need for a set of agreed / self imposed safeguards on certain types of strong AI research - eg. make them read-only,…

They wouldn't get $1B if they didn't do deep learning.

Re: Introducing OpenAI

#167

Post by TheOtherHobbes »

mikepalmer wrote: post #99
karmacondon wrote: post #88

This is a key takeaway: "...we are going to ask YC companies to make whatever data they are comfortable making available to OpenAI. And Elon is also going to figure out what data Tesla and Space X can share." Money is great, openness is great, big name researchers are also a huge plus. But data data data, that could turn out to be very valuable. I don't know if Sam meant that YC companies would be encouraged to contr…

'The "evil" AI and safety stuff is just science fiction, but whatever.' umm... you can offer proof that we have nothing to worry about? Does the proof go like: Just as all people are inherently good, therefore all AIs will be inherently good? Or is it more like: since we can now safely contain all evil people, therefore we will be able to safely contain evil AIs? Sounds to me like there is some risk, no?

Obviously no one can prove that AIs would be "inherently good" because there's no definition of "good" that everyone agrees on.

I'd be more impressed by a Human Intelligence Project - augmenting predictive power to encourage humans to stop doing stupid, self-destructive shit, and moving towards long-term glory and away from trivial individual short-term greed as a primary motivation.

AI is a non-issue compared to the bear pit of national and international politics and economics.

So the AI Panic looks like psychological projection to me. It's easier to mistrust the potential of machines than to accept that we're infinitely more capable of evil than any machine is today - and that's likely to stay true for decades, if not forever.

The corollary is that AI is far more likely to become a problem if it's driven by the same motivations as politics and economics. I see that as more of a worry than the possibility some unstoppable supermachine is going to "decide" it wants to use Earth as a paperclip factory, or that Siri is going to go rogue and rickroll everyone on the planet.

Job-destroying automation and algorithmic/economic herding of humans is the first wave of this. It's already been happening for centuries. But it could, clearly, get a lot worse if the future isn't designed intelligently.

Re: Introducing OpenAI

#168

Post by mindcrime »

vonnik wrote: post #158
mindcrime wrote: post #155

Earlier quoted context omitted.

If you take the quote above, and substitute the word "guns" for "AI", you basically have the NRA, and the NRA is not making the world a safer place. They may actually be. Although mostly not in the way you're talking about, but there is something to be said for the dispersing of power. If one or two players have a power no one else has, there's more temptation to use it. If it's widely distributed, it seems reasonabl…

If we're focused on weapons of mass destruction, I prefer a world of nuclear nonproliferation to the opposite. There are relatively few nations that possess nuclear weapons, and we have very few instances of them using those weapons against anyone else. To argue against myself, I'd say that the difference between weapons and AI is that AI is more general. It's not just a killing machine. In fact, I hope that killing…

If we're focused on weapons of mass destruction, I prefer a world of nuclear nonproliferation to the opposite

So do I generally speaking, but with a caveat... I think that having multiple (eg, more than 1 or 2) nuclear powers is likely a Good Thing (given that the tech exists at all). The whole MAD principle seems very likely to be one reason the world has yet to descend into nuclear war. The main reason I prefer nuclear non-proliferation though, isn't because I genuinely expect something like a US/Russia nuclear war, it's more the possibility of terrorists or non-state actors getting their hands on a nuke.

It's interesting though, because these various analogies between guns, nukes and AI's don't necessarily hold up. I was about to say a lot more, but on second thought, I want to think about this more.

Re: Introducing OpenAI

#170

Post by johann28 »

tangled_zans wrote: post #116

Who are the actual staff involved? What sort of things have they worked on and published before?

> OpenAI's research director is Ilya Sutskever, one of the world experts in machine learning. Our CTO is Greg Brockman, formerly the CTO of Stripe. The group's other founding members are world-class research engineers and scientists: Trevor Blackwell, Vicki Cheung, Andrej Karpathy, Durk Kingma, John Schulman, Pamela Vagata, and Wojciech Zaremba. Pieter Abbeel, Yoshua Bengio, Alan Kay, Sergey Levine, and Vishal Sikka are advisors to the group. OpenAI's co-chairs are Sam Altman and Elon Musk.

Sutskever is a researcher at Google, worked with Hinton in Toronto and Andrew Ng at Stanford.

Karpathy studied in Toronto and at Stanford, worked under Fei-Fei Li, worked at Google. He also has an awesome blog and seems very active and passionate about computer vision and ML.

Kingma also works with deep neural nets, worked under Yann LeCun (who works at Facebook)

Schulman is a PhD Candidate at Berkeley with publications at top conferences.

Zaremba is an PhD student at NYU, intern at Facebook. Impressive publication list and awards.

Abbeel is at Stanford's AI lab.

Bengio is one of the "stars" and celebrated figures of the deep net revival.

Levine is a researcher at Google working on deep nets with many serious papers.

---

Basically these are the main domain experts among them. The list is quite skewed to Google/Facebook, Stanford/Berkeley/Toronto and deep net researchers, working primarily on computer vision.

Post reply on HN