Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Thoughts on OpenAI, reinforcement learning, and killer robots

fast.ai

Post reply on HN
Print view View original

161–170 of 205 posts

Re: Thoughts on OpenAI, reinforcement learning, and killer robots

#161

Post by nopinsight »

unityByFreedom wrote: post #154
nopinsight wrote: post #140

Earlier quoted context omitted.

Unlike airplanes, AGI possesses agency and the ability to cause widespread harm and damage in today's computer-penetrated world. So if we wait until it is invented, there is no guarantee that there won't be danger or even catastrophes. Wouldn't you even agree that there is a 1% chance that AGI is possible and that it could harm us once invented? Also, the goalpost for identifying something as a challenging cognitive…

> Wouldn't you even agree that there is a 1% chance that AGI is possible and that it could harm us once invented? There's a small chance aliens could find us, too. Yet, we don't know any more about that than we do about AI. The most imminent threat to humans is humans. In the nearer future, it seems more likely that someone will use machine learning technology in weaponry to cause harm than create AGI. Or, that human…

For subgoals, there have been a lot of research results on that, mostly in the symbolic AI tradition.

Even if an AI does not determine its own ultimate goal(s), it could still cause us a lot harm by creating subgoals that do not align with human values, which no one can articulate in full yet.

Overall, the controversy is worth it because the force for progress is strong enough that another AI winter is highly unlikely if useful applications continue to be developed. Without the warnings, research on Provably Safe AI might not have sufficient resources especially talented people working on it.

Re: Thoughts on OpenAI, reinforcement learning, and killer robots

#162

Post by eli_gottlieb »

visarga wrote: post #101
daly wrote: post #60

I've worked in a lot of AI-related projects and was around when the AI winter arrived. These various techniques that currently work by training, either supervised or self-training, can have fatal flaws. Take, for example, some high-tech camera technology. Use it on a drone to take pictures of warships from thousands of angles. You take pictures of U.S. warships, Russian warships, and Chinese warships. You achieve 100…

This problem is well studied - there are ways to make a neural net explain what parts of the input most influenced the decision. Another solution would be to use autoencoders or GANs to create a latent code from the input image. By construction, these codes need to carry the most important features about the input, because otherwise they couldn't reconstruct it. And regarding analysis - a lot of groups are attempting…

> This problem is well studied - there are ways to make a neural net explain what parts of the input most influenced the decision.

That's a new area of research, actually.

Re: Thoughts on OpenAI, reinforcement learning, and killer robots

#163

Post by eli_gottlieb »

tanilama wrote: post #137
nopinsight wrote: post #91

Earlier quoted context omitted.

The point is whether we should be complacent and dismiss concerns just because we don't know when it will happen. Once someone builds it, stopping it might be very difficult. Here's why: https://youtu.be/4l7Is6vOAOA (less than 9 minutes and very clearly explained). Dismissing even a 10% chance of possible catastrophic risks is not what we practice in any other domains. Would you dismiss a concern over airplanes that…

Point is, nobody knows whether AGI really exists or not, based on our current approach. As I have answered in another comment section, natural language understanding haven't been cracked, at all. I mean at all. So the reality is, it is like ancient human knows birds can fly with wings, but people can only fly in their dreams. Now, we know people can think intelligently with their brain, but no one knows how to make c…

You already have Joshua Tenenbaum and Karl Friston. You're just ignoring them in favor of celebrities who talk about AI rather than mere cognition, and have billions in funding and years of professional self-marketing to throw around.

Re: Thoughts on OpenAI, reinforcement learning, and killer robots

#164

Post by halwa »

moravak1984 wrote: post #159
visarga wrote: post #101

Earlier quoted context omitted.

This problem is well studied - there are ways to make a neural net explain what parts of the input most influenced the decision. Another solution would be to use autoencoders or GANs to create a latent code from the input image. By construction, these codes need to carry the most important features about the input, because otherwise they couldn't reconstruct it. And regarding analysis - a lot of groups are attempting…

"This problem is well studied - there are ways to make a neural net explain what parts of the input most influenced the decision." Would you mind providing some reference here? I am interested and not familiar with any such way.

While I wouldn't say that the problem has been "well studied", the research community has been paying interest, and some progress has been made, most notably- as pointed out by benjaminjackman- LIME[1]. Roughly speaking, LIME learns a locally approximate model which can be interpreted. It will work will any black box model, not just neural networks.

[1] https://arxiv.org/abs/1602.04938

Re: Thoughts on OpenAI, reinforcement learning, and killer robots

#165

Post by uptownfunk »

jacquesm wrote: post #67
daly wrote: post #60

I've worked in a lot of AI-related projects and was around when the AI winter arrived. These various techniques that currently work by training, either supervised or self-training, can have fatal flaws. Take, for example, some high-tech camera technology. Use it on a drone to take pictures of warships from thousands of angles. You take pictures of U.S. warships, Russian warships, and Chinese warships. You achieve 100…

That's called 'data leakage'. http://machinelearningmastery.com/data-leakage-machine-learn...

Actually this is a clear case of overfitting leading to a high false positive rate resulting in the misidentification of a US ship for chinese.

Re: Thoughts on OpenAI, reinforcement learning, and killer robots

#166

Post by unityByFreedom »

nopinsight wrote: post #160
unityByFreedom wrote: post #153

Earlier quoted context omitted.

> Most humans do not really come up with our goals on our own either I disagree. I believe we have free will, in addition to being influenced by our environment/genes. This is my general point, that contemplating AI is a way for us to investigate how we ourselves operate. You have your theory and I have mine. Nobody has shown how we can be replicated. EDIT > We can imagine an alien who is highly capable of any tasks…

Unfortunately, no one can really articulate the full set of human values yet, especially since they are often in conflict in real-world situations. Which value(s) should be prioritized over others? How about differing values between cultures and human groups within each culture? You might have heard of the Trolley Problem. There are many and more complicated variations of that. This Harvard session on Justice (The Mo…

> Which value(s) should be prioritized over others? How about differing values between cultures and human groups within each culture?

Don't kill humans sounds like a good start.

Honestly, this problem seems a lot simpler to me than creating AGI itself.

I agree it is challenging. The most imminent question is probably, "should a self driving car be allowed to kill 10 people on the sidewalk to avoid a head-on collision, or should it accept the head-on and let the driver die?"

These domain specific problems will come up before the AGI one does, and we can address them as they become relevant.

You've noted that regulating AGI is difficult, and it sounds like you don't have any other solution in mind.

Re: Thoughts on OpenAI, reinforcement learning, and killer robots

#167

Post by mindcrime »

_qbjt wrote: post #74
jacquesm wrote: post #64

Earlier quoted context omitted.

Now you're being (even more) silly. Obviously I can't turn that one off any more than I can turn off the refrigerator downstairs from where I'm sitting. But there are people there that can turn it off, and if I was physically present there then I could turn it off.

You're working under the assumption that no one will attempt to use AI to assert power over others, and that the human race will come together to pull the plug when it becomes a problem. Looking at our current geopolitical situation, I'd rather be silly than naive.

Humans + nukes already equal an existential threat to the human race, so I'm not seeing how AI adds anything new to the equation in that scenario. "Humans using AI + nukes" vs "Humans using nukes", either one probably ends human life on Earth.

I thought the problem everybody was fretting about was the idea of a nominally beneficent AI going off on it's own and doing something malicious / destructive.

Re: Thoughts on OpenAI, reinforcement learning, and killer robots

#168

Post by unityByFreedom »

nopinsight wrote: post #161
unityByFreedom wrote: post #154

Earlier quoted context omitted.

> Wouldn't you even agree that there is a 1% chance that AGI is possible and that it could harm us once invented? There's a small chance aliens could find us, too. Yet, we don't know any more about that than we do about AI. The most imminent threat to humans is humans. In the nearer future, it seems more likely that someone will use machine learning technology in weaponry to cause harm than create AGI. Or, that human…

For subgoals, there have been a lot of research results on that, mostly in the symbolic AI tradition. Even if an AI does not determine its own ultimate goal(s), it could still cause us a lot harm by creating subgoals that do not align with human values, which no one can articulate in full yet. Overall, the controversy is worth it because the force for progress is strong enough that another AI winter is highly unlikel…

> the force for progress is strong enough that another AI winter is highly unlikely if useful applications continue to be developed.

Really disagree here. All it takes is a few well positioned people, like those at Nnaisense, to create companies promising AGI and sucking up investment dollars. We all know greed is rampant in finance, and AI tech is a big word in startups/innovation these days that can land big VC dollars. There is corruption/snake oil in tech too, and you'll find the greediest float around the most hyped tech.

I think companies like Nnaisense are the wrong place to invest, and that there is much more practical work to be done to advance humanity.

Re: Thoughts on OpenAI, reinforcement learning, and killer robots

#169

Post by nopinsight »

unityByFreedom wrote: post #166
nopinsight wrote: post #160

Earlier quoted context omitted.

Unfortunately, no one can really articulate the full set of human values yet, especially since they are often in conflict in real-world situations. Which value(s) should be prioritized over others? How about differing values between cultures and human groups within each culture? You might have heard of the Trolley Problem. There are many and more complicated variations of that. This Harvard session on Justice (The Mo…

> Which value(s) should be prioritized over others? How about differing values between cultures and human groups within each culture? Don't kill humans sounds like a good start. Honestly, this problem seems a lot simpler to me than creating AGI itself. I agree it is challenging. The most imminent question is probably, "should a self driving car be allowed to kill 10 people on the sidewalk to avoid a head-on collision…

It is often not practical for something dumber to regulate something far more intelligent. (Humans win against lions despite our physical weakness.) So the best solution I have heard of is to create a Provably Safe AGI (or Friendly AI in Yudkowsky's term) and have them help us regulate other AI efforts. A moral core that aligns with human values need to be part of this Safe AGI.

It is definitely very challenging to create one, and more challenging than creating a random AGI. The morality also needs to be integrated into the Safe AGI as its core that is not susceptible to self-modification ability that an AGI could have. Thus, we need to work on that aspect of AGI now.

Stuart Russell has outlined his preliminary thought on the topic: https://www.ted.com/talks/stuart_russell_how_ai_might_make_u...

Re: Thoughts on OpenAI, reinforcement learning, and killer robots

#170

Post by tanilama »

nopinsight wrote: post #140
tanilama wrote: post #137

Earlier quoted context omitted.

Point is, nobody knows whether AGI really exists or not, based on our current approach. As I have answered in another comment section, natural language understanding haven't been cracked, at all. I mean at all. So the reality is, it is like ancient human knows birds can fly with wings, but people can only fly in their dreams. Now, we know people can think intelligently with their brain, but no one knows how to make c…

Unlike airplanes, AGI possesses agency and the ability to cause widespread harm and damage in today's computer-penetrated world. So if we wait until it is invented, there is no guarantee that there won't be danger or even catastrophes. Wouldn't you even agree that there is a 1% chance that AGI is possible and that it could harm us once invented? Also, the goalpost for identifying something as a challenging cognitive…

For NLU, to convince me the potential, one important evidence I would be particularly interested, is the demonstration of reasoning. Unlike the current black-box model, I would love to see the model give explanation, along with the answer, what it takes to lead such conclusion, be it a rule listed in the manual or case that happened before.

This itself presents several big challenges: how do we represent the concept of reasoning, in mathematical form? Knowledge required or not? What is the representation of such knowledge?

A truly intelligent machine, should be able to take what human takes, a piece of text, then build its knowledge base from there, then answer a question just like human, then give explanation when asked for it.

Post reply on HN