Introducing OpenAI • Hacker News Board

Re: Introducing OpenAI

#131

Post by johann28 » Fri, Dec 11, 2015, 11:31 PM UTC

_delirium wrote: post #102

johann28 wrote: post #71

Earlier quoted context omitted.

Sutskever, Schulman, Karpathy and Kingma are experts in machine learning. And yes, AI will definitely be used for all sorts of purposes including hostile means. Just like anything else, really. Financial manipulation, spying, intelligent military devices, cracking infrastructure security, etc. These are realistic concerns, we shouldn't fall for the Skynet red herring. We can have problems with ethical AI use, even if…

I hope I'm wrong, but given the composition of the donors I'd be surprised if they really put much scrutiny on near-term corporate/government misuse of AI, apart perhaps from military robots. There are definitely interesting ethics questions already arising today around how large tech companies, law enforcement, etc. are starting to use AI, whether it's Palantir, the FBI, Google, or Facebook, so no argument that it's…

I share your concerns. It also worries me that the brightest ML researchers choose to work at companies like Facebook, Google and Microsoft instead of public universities. One reason is probably that academia and public grants are too sluggish to accommodate this fast paced field. Another is that these companies have loads of data that these researchers can use to test their ideas.

The downside is that much of the research is probably held secret for business advantages. The public releases are more of a PR and hiring strategy than anything else in my opinion. By sending papers to conferences, Google's employees can get to know the researchers and attract them to Google.

Others say there's nothing to worry about, Google and Facebook are just today's equivalent of Bell Labs, which gave numerous contributions to computer technologies without causing much harm.

Re: Introducing OpenAI

#132

Post by rl3 » Fri, Dec 11, 2015, 11:31 PM UTC

>Musk: I think the best defense against the misuse of AI is to empower as many people as possible to have AI. If everyone has AI powers, then there’s not any one person or a small set of individuals who can have AI superpower.

This is essentially Ray Kurzweil's argument. Surprising to see both Musk and Altman buy into it.

If the underlying algorithms used to construct AGI turn out to be easily scalable, then the realization of a dominant superintelligent agent is simply a matter of who arrives first with sufficient resources. In Bostrom's Superintelligence, a multipolar scenario was discussed, but treated as unkikely due to the way first-arrival and scaling dynamics work.

In other words, augmenting everyone's capability or intelligence doesn't necessarily preclude the creation of a dominant superintelligent agent. On the contrary, if there's any bad or insufficiently careful actors attempting to construct a superintelligence, it's safe to assume they'll be taking advantage of the same AI augments everyone else has, thus rendering the dynamic not much different from today (i.e. a somewhat equal—if not more equal—playing field).

I would argue that in the context of AGI, an equal playing field is actually undesirable. For example, if we were discussing nuclear weapons, I don't think anyone would be arguing that open-source schematics is a great idea. Musk himself has previously stated that [AGI] is "potentially more dangerous than nukes"—and I tend to agree—it's just that we do not know the resource or material requirements yet. Fortunately with nuclear weapons, they at least require highly enriched materials, which render them mostly out of reach to anyone but nation states.

To be clear, I think the concept of opening up normal AI research is fantastic, it's just that it falls apart when viewed in context of AGI safety.

Re: Introducing OpenAI

#133

Post by pmichaud » Fri, Dec 11, 2015, 11:31 PM UTC

argonaut wrote: post #79

I find it a bit disappointing that despite originally stating that YC Research would target underfunded/underserved areas of research, they've decided to fund and dive into one of the most-hyped, well-funded areas of research: deep learning, an area of research where companies are hiring like crazy and even universities are hiring faculty like crazy. I'm reasonably sure all the research scientists had multiple job of…

"AI" has a lot of attention, but AI Safety is underfunded, and understaffed.

Re: Introducing OpenAI

#134

Post by JoshTriplett » Fri, Dec 11, 2015, 11:36 PM UTC

baconner wrote: post #56

JoshTriplett wrote: post #50

Earlier quoted context omitted.

The idea is not that we should build and (try to) suppress a sentient AI; that would be a bad idea for numerous reasons. However, we don't necessarily need to build a sentient AI in the first place; we can build a process that has reasoning capabilities far above human without actually having agency of its own.

See I think that's exactly where it becomes complicated. Can an entity with reasoning capabilities far beyond that of humans have its agency suppressed successfully? And is it ethical to do so or is that internally designed suppression somehow ethically different from the external suppression applied against human slaves? If you could engineer a human being with his/her agency removed so that you could use their reas…

The way you're asking the question implies that reasoning inherently has agency/sentience that needs suppressing. It doesn't need to; there's nothing to "suppress".

Re: Introducing OpenAI

#135

Post by hacker_9 » Fri, Dec 11, 2015, 11:39 PM UTC

johann28 wrote: post #128

hacker_9 wrote: post #84

Earlier quoted context omitted.

What? The brain has not only solved the vision problem (reconstructing depth from still images, recognising the objects in the scene, and filling in the occluded parts), it has also solved the motion problem of coordinated the movement of our ~300 muscles (given constraints, how do I move from A to B, or pickup the cup, or do a handstand), as well as solved the memory problem (basically infinite memory, with some sor…

The brain doesn't "solve" tasks. That's cart before horse thinking. Our whole concept of "vision" only exists because eyes and visual cortices exist. I know it seems like a philosophical nitpick, but saying that the visual cortex is good at vision is like saying that water is good at being wet or being surprised that your soup is perfectly fitting the shape of your bowl. Now, the human brain is definitely a complicat…

I know, I was approaching it from your angle, the current state of ML, and explaining it from that context. You support ML but then refer to humans as 'just apes' in a derogatory fashion. I was just pointing out that in fact these dumb apes solved all your ML problems a very long time ago.

Re: Introducing OpenAI

#136

Post by w1ntermute » Fri, Dec 11, 2015, 11:44 PM UTC

samstave wrote: post #19

_sentient wrote: post #5

$1B in committed funding. Just, wow. Side note: I wonder if the Strong AI argument can benefit from something akin to Pascal's Wager, in that the upside of being right is ~infinite with only a finite downside in the opposing case.

semi-off-topic: after Google invested $1B in Uber, I knew they were doing it for the self-driving car long play. How much of that 1B is directly going to self-driving AI at Uber?

The vast majority is no doubt going to subsidizing rides in an attempt to achieve market dominance.

Re: Introducing OpenAI

#137

Post by strong_ai » Fri, Dec 11, 2015, 11:45 PM UTC

hacker_9 wrote: post #3

This is about 100 years too early. Seriously why do people think neural networks are the answer to AI? They are proven to be stupid outside of their training data. We have such a long way to go. This fear-mongering is pointless.

What do you see as the downside of creating this organization now, as opposed to in 100 years? Artificial Intelligence in its current state has shown itself to be incredibly useful and effective at small tasks. I see no harm in researching and expanding this field (besides opportunity cost).

Also, I see a distinct lack of "fear-mongering" in this post.

Re: Introducing OpenAI

#138

Post by LesZedCB » Fri, Dec 11, 2015, 11:55 PM UTC

miguelrochefort wrote: post #111

vox_mollis wrote: post #85

Earlier quoted context omitted.

Given Musk's public comments about existential threats, I assume the focus will be on friendly AI theory and implementations, akin to what MIRI does.

Friendly AI? That's no AI! I don't understand how Yudkowsky came up with such a ridiculous idea. That's simply not a constraint you can apply to true AI. Even if friendly AI was possible, it wouldn't make sense to have it, nor would any form of regulation enforce it.

You should read this! [1] Unfortunately, the problem is not nearly as simple as you make it seem to be, otherwise this thread wouldn't be here. :)

[1] http://waitbutwhy.com/2015/01/artificial-intelligence-revolu...

Re: Introducing OpenAI

#139

Post by skndr » Fri, Dec 11, 2015, 11:59 PM UTC

karmacondon wrote: post #123

mikepalmer wrote: post #99

Earlier quoted context omitted.

'The "evil" AI and safety stuff is just science fiction, but whatever.' umm... you can offer proof that we have nothing to worry about? Does the proof go like: Just as all people are inherently good, therefore all AIs will be inherently good? Or is it more like: since we can now safely contain all evil people, therefore we will be able to safely contain evil AIs? Sounds to me like there is some risk, no?

As I've said many times on HN over the years, there is currently no clear path to science fiction like "AI". To return your question, hopefully without being rude, is there any proof that AI capable of having a moral disposition will ever exist? Andrew Ng (I believe) compared worrying about evil AI to worrying about overpopulation on Mars. Which is to say, the problem is so far off that it's rather silly to be consid…

Improvements in AI aren't linear, though. Artificial Superintelligence, after reaching AGI, might happen in the span of minutes or days. I imagine the idea here is to guide progress so that on the day that AGI is possible, we've already thoroughly considered what happens after that point.

Secondly - it's not necessarily about 'evil' AI. It's about AI indifferent to human life. Have a look at this article, it provides a better intuition for how slippery AI could be: https://medium.com/@LyleCantor/russell-bostrom-and-the-risk-...

Re: Introducing OpenAI

#140

Post by jtolmar » Sat, Dec 12, 2015, 12:01 AM UTC

karmacondon wrote: post #123

mikepalmer wrote: post #99

Earlier quoted context omitted.

'The "evil" AI and safety stuff is just science fiction, but whatever.' umm... you can offer proof that we have nothing to worry about? Does the proof go like: Just as all people are inherently good, therefore all AIs will be inherently good? Or is it more like: since we can now safely contain all evil people, therefore we will be able to safely contain evil AIs? Sounds to me like there is some risk, no?

As I've said many times on HN over the years, there is currently no clear path to science fiction like "AI". To return your question, hopefully without being rude, is there any proof that AI capable of having a moral disposition will ever exist? Andrew Ng (I believe) compared worrying about evil AI to worrying about overpopulation on Mars. Which is to say, the problem is so far off that it's rather silly to be consid…

is there any proof that AI capable of having a moral disposition will ever exist?

Why does an AI need to be capable of moral reasoning to perform actions we'd consider evil?

The concern is that computers will continue to do what they're programmed to do, not what we want them to do. We will continue to be as bad at getting those two things to line up as we've always been, but that will become dangerous when the computer is smarter than its programmers and capable of creatively tackling the task of doing something other than what we wanted it to do. Any AI programmed to maximize a quantity is particularly dangerous, because that quantity does not contain a score for accurately following human morality (how would you ever program such a score).

If you're willing to believe that an AI will some day be smarter than an AI researcher (and assuming that's not possible applies a strange special-ness to humans), then an AI will be capable of writing AIs smarter than itself, and so forth up to whatever the limits of these things are. Even if that's not its programmed goal, you thought making something smarter than you would help with your actual goal, and it's smarter than you so it has to realize this too. And that's the bigger danger - at some unknown level of intelligence, AIs suddenly become vastly more intelligent than expected, but still proceed to do something other than what we wanted.