Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Thoughts on OpenAI, reinforcement learning, and killer robots

fast.ai

Post reply on HN
Print view View original

171–180 of 205 posts

Re: Thoughts on OpenAI, reinforcement learning, and killer robots

#171

Post by lern_too_spel »

tanilama wrote: post #137
nopinsight wrote: post #91

Earlier quoted context omitted.

The point is whether we should be complacent and dismiss concerns just because we don't know when it will happen. Once someone builds it, stopping it might be very difficult. Here's why: https://youtu.be/4l7Is6vOAOA (less than 9 minutes and very clearly explained). Dismissing even a 10% chance of possible catastrophic risks is not what we practice in any other domains. Would you dismiss a concern over airplanes that…

Point is, nobody knows whether AGI really exists or not, based on our current approach. As I have answered in another comment section, natural language understanding haven't been cracked, at all. I mean at all. So the reality is, it is like ancient human knows birds can fly with wings, but people can only fly in their dreams. Now, we know people can think intelligently with their brain, but no one knows how to make c…

When you know AGI is possible with existing techniques, it is already too late to discuss the risks. As soon as it is possible with existing techniques, it will be implemented by somebody, and the risks will be realized.

Re: Thoughts on OpenAI, reinforcement learning, and killer robots

#172

Post by Jach »

visarga wrote: post #106
philipkglass wrote: post #58

investments into addressing mass unemployment and wealth inequality (both of which are well-documented to cause political instability) I don't really expect to see this solved in the USA before it's solved in poorer countries. In the USA there's a tension between pleasing the millions of surplus workers and pleasing the top billionaires that'll be minted when robots can replace millions of workers. It's cheaper for t…

The day we will invent a self-replicating factory that only uses cheap local raw materials, the current economic system will end. We could say the whole of the economy is a self replicating system, but we need to shrink that to a small size and make it not dependent on rare or contested materials. Humans, genes, and the ecosystem are self replicators as well. Self replication might be a different kind of singularity…

I would rather have successful AGI before universal replicators that are available to many merely human intelligences. Super especially hope for that if these replicators operate at the nano scale, which I think you're implying with the suggestion everything can be made from common raw material like dirt. But I'm not too optimistic. Governments couldn't even stop proliferation of 3D printed guns, how will they cope with other things in the future that have an economic incentive too?

Don't forget that even if certain materials no longer become rare or contested, location and raw energy will still be. Without a supreme AGI, or an upheaval of our most basic knowledge of physics, expect wars to be fought over rotating black hole real estate.

Re: Thoughts on OpenAI, reinforcement learning, and killer robots

#173

Post by kgwgk »

uptownfunk wrote: post #165
jacquesm wrote: post #67

Earlier quoted context omitted.

That's called 'data leakage'. http://machinelearningmastery.com/data-leakage-machine-learn...

Actually this is a clear case of overfitting leading to a high false positive rate resulting in the misidentification of a US ship for chinese.

I don't think that's a clear case of overfitting. You could have used a subset of the original data for training and the rest for validation and it would have generalised pretty well.

It doesn't generalise when the US ship is in Chinese waters, but that's because the system was never "learning" to recognize ships in the first place.

Re: Thoughts on OpenAI, reinforcement learning, and killer robots

#174

Post by pure21r »

j2kun wrote: post #127
_qbjt wrote: post #113

Earlier quoted context omitted.

> That is also not relevant to the future implications of AI, because as AI evolves so will humanity. So why do black hat hackers exist? Why do you have so much faith in humanity's ability and/or willingness to "evolve" alongside AI? My prediction is that AI will evolve and humans will still be the same murderous creatures they are today, only with more tools at their disposal. > You're also ignoring basic ideas that…

Why do you have so much faith that autonomous superhuman AI will be such a Dalek-eqsue threat? If it's faith against faith, and you have no evidence to back up your side (while humanity adapting has millennia of evidence), there's no point in discussing it. > Humans are pretty easy to manipulate...My fear is that with greater capabilities,... Nobody is disagreeing with you. But this is hardly an observation of the ap…

So what would be your equivalent position in the 1940s about the nuclear solution? "Don't worry about it, humanity has always prevented slow moving disasters."? Because the question is not about whether one should develop AI or not, but if we should be worried and work actively to minimize risk, as has been done on the nuclear case...

So suppose you simulate 100 Earths from today t=0? What are the best strategies to prevent this problem? Are we doing all we can? What proportion of the 100 Earths have a good outcome?

In other words, what are the risks and benefits of adopting a "worried" versus a "don't worry about it" position? For example, what's the worst that could happen if we all worry today for nothing, and in 50 years we all live happily in the cloud? What if we don't worry now and it turns out that AI somehow wiped half the population (or all of it)?

If there's a possible huge meteor (say 30% chance) of hitting the Earth in about 50 years, would you be "worried"?

Re: Thoughts on OpenAI, reinforcement learning, and killer robots

#175

Post by pure21r »

jacquesm wrote: post #56
_qbjt wrote: post #47

Earlier quoted context omitted.

Accelerated particle colliders are not exactly equivalent to AI. It's not as if I can just build one in my garage. Anyone in the world, regardless of intent, can train a neural network to do what it wants. Extrapolate this decades down the road at the point that AI and its capabilities are much more sophisticated. Do you want to put money on the likelihood that it's all just going to go down smoothly? I don't even kn…

> The idea of replacing human life with AI isn't exactly controversial in the realm of futurism. It isn't in Science Fiction. But this is reality, and in this reality we do not have AGI and we have no idea of how far we are away from it. And even if and when it happens there is absolutely no guarantee that that will lead to the extinction of the human race and/or us ending up as slaves to the machine.

So what would be your equivalent position in the 1940s about the nuclear solution? "Don't worry about it, humanity has always prevented slow moving disasters."?

Because the question is not about whether one should develop AI or not (this is going to happen in either case), but if we should be worried and work actively to minimize risk, as has been done on the nuclear case...

So suppose you simulate 100 Earths from today t=0? What are the best strategies to prevent this problem? Are we doing all we can? What proportion of the 100 Earths have a good outcome?

In other words, what are the risks and benefits of adopting a "worried" versus a "don't worry about it" position? For example, what's the worst that could happen if we all worry today for nothing, and in 50 years we all live happily in the cloud? What if we don't worry now and it turns out that AI somehow wiped half the population (or all of it)?

If there's a possible huge meteor (say 30% chance) of hitting the Earth in about 50 years, would you be "worried"?

Re: Thoughts on OpenAI, reinforcement learning, and killer robots

#176

Post by pure21r »

tanilama wrote: post #170
nopinsight wrote: post #140

Earlier quoted context omitted.

Unlike airplanes, AGI possesses agency and the ability to cause widespread harm and damage in today's computer-penetrated world. So if we wait until it is invented, there is no guarantee that there won't be danger or even catastrophes. Wouldn't you even agree that there is a 1% chance that AGI is possible and that it could harm us once invented? Also, the goalpost for identifying something as a challenging cognitive…

For NLU, to convince me the potential, one important evidence I would be particularly interested, is the demonstration of reasoning. Unlike the current black-box model, I would love to see the model give explanation, along with the answer, what it takes to lead such conclusion, be it a rule listed in the manual or case that happened before. This itself presents several big challenges: how do we represent the concept…

So what would be your equivalent position in the 1940s about the nuclear solution? "Don't worry about it, humanity has always prevented slow moving disasters."? Because the question is not about whether one should develop AI or not, but if we should be worried and work actively to minimize risk, as has been done on the nuclear case...

So suppose you simulate 100 Earths from today t=0? What are the best strategies to prevent this problem? Are we doing all we can? What proportion of the 100 Earths have a good outcome?

In other words, what are the risks and benefits of adopting a "worried" versus a "don't worry about it" position? For example, what's the worst that could happen if we all worry today for nothing, and in 50 years we all live happily in the cloud? What if we don't worry now and it turns out that AI somehow wiped half the population (or all of it)?

If there's a possible huge meteor (say 30% chance) of hitting the Earth in about 50 years, would you be "worried"?

Re: Thoughts on OpenAI, reinforcement learning, and killer robots

#177

Post by [deleted] »

nopinsight wrote: post #169
unityByFreedom wrote: post #166

Earlier quoted context omitted.

> Which value(s) should be prioritized over others? How about differing values between cultures and human groups within each culture? Don't kill humans sounds like a good start. Honestly, this problem seems a lot simpler to me than creating AGI itself. I agree it is challenging. The most imminent question is probably, "should a self driving car be allowed to kill 10 people on the sidewalk to avoid a head-on collision…

It is often not practical for something dumber to regulate something far more intelligent. (Humans win against lions despite our physical weakness.) So the best solution I have heard of is to create a Provably Safe AGI (or Friendly AI in Yudkowsky's term) and have them help us regulate other AI efforts. A moral core that aligns with human values need to be part of this Safe AGI. It is definitely very challenging to c…

[deleted]

Re: Thoughts on OpenAI, reinforcement learning, and killer robots

#178

Post by pure21r »

unityByFreedom wrote: post #146
nopinsight wrote: post #143

Earlier quoted context omitted.

We also don't know how long it will take to solve Provably Safe AI problem. Wouldn't it be wiser to start preparing now? Nuclear weapons were deemed impossible by some experts at the time. Throughout history, whenever there are a number of experts who believe something to be impossible and a substantial number who believe it to be possible, the latter are almost always right. See also my other answer that begins with…

> We also don't know how long it will take to solve Provably Safe AI problem. Wouldn't it be wiser to start preparing now? Sure, go for it. I think this has been in the news a lot recently because Musk told government leaders to expect AGI as early as 2030-2040. It's a bit of a ridiculous claim. One of the biggest AI competitions, ImageNet, expects to run for the next 12 years, and that's just for making sense of ima…

So what would be your equivalent position in the 1940s about the nuclear solution? "Don't worry about it, humanity has always prevented slow moving disasters."? Because the question is not about whether one should develop AI or not, but if we should be worried and work actively to minimize risk, as has been done on the nuclear case...

So suppose you simulate 100 Earths from today t=0? What are the best strategies to prevent this problem? Are we doing all we can? What proportion of the 100 Earths have a good outcome?

In other words, what are the risks and benefits of adopting a "worried" versus a "don't worry about it" position? For example, what's the worst that could happen if we all worry today for nothing, and in 50 years we all live happily in the cloud? What if we don't worry now and it turns out that AI somehow wiped half the population (or all of it)?

If there's a possible huge meteor (say 30% chance) of hitting the Earth in about 50 years, would you be "worried"?

Re: Thoughts on OpenAI, reinforcement learning, and killer robots

#179

Post by k_lander »

daly wrote: post #60

I've worked in a lot of AI-related projects and was around when the AI winter arrived. These various techniques that currently work by training, either supervised or self-training, can have fatal flaws. Take, for example, some high-tech camera technology. Use it on a drone to take pictures of warships from thousands of angles. You take pictures of U.S. warships, Russian warships, and Chinese warships. You achieve 100…

in your example, was the neural net trained with examples of US ships in Chinese waters?

Re: Thoughts on OpenAI, reinforcement learning, and killer robots

#180

Post by mostafab »

Reinforcement Learning is fundamental research, it needs sponsors like Musk to survive. Agriculture and medicine are applied research, they can find their own independent business models for monetization. That's what we are doing at Startcrowd: http://www.startcrowd.club
Post reply on HN