Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Thoughts on OpenAI, reinforcement learning, and killer robots

fast.ai

Post reply on HN
Print view View original

101–110 of 205 posts

Re: Thoughts on OpenAI, reinforcement learning, and killer robots

#101

Post by visarga »

daly wrote: post #60

I've worked in a lot of AI-related projects and was around when the AI winter arrived. These various techniques that currently work by training, either supervised or self-training, can have fatal flaws. Take, for example, some high-tech camera technology. Use it on a drone to take pictures of warships from thousands of angles. You take pictures of U.S. warships, Russian warships, and Chinese warships. You achieve 100…

This problem is well studied - there are ways to make a neural net explain what parts of the input most influenced the decision.

Another solution would be to use autoencoders or GANs to create a latent code from the input image. By construction, these codes need to carry the most important features about the input, because otherwise they couldn't reconstruct it.

And regarding analysis - a lot of groups are attempting the leap from mapping "X -> y" to reasoning based on typed entities and relations. Reasoning would be more like a simulator coupled with a MCMC system that tries out various scenarios in its 'imagination' before acting out.

There are many formulations: relational neural nets, graph based convolutional networks, physical simulators based on neural nets, text reasoning tasks based on multiple attention heads and/or memory. It's very exciting, we're closing in on reasoning.

Re: Thoughts on OpenAI, reinforcement learning, and killer robots

#102

Post by backpropaganda »

cshenton wrote: post #92
jph00 wrote: post #15

Earlier quoted context omitted.

Perhaps, but the jury is still very much out. The vast majority of RL applications are game playing. Very few examples of valuable applications to society or the economy. There's also plenty of evidence already that RL isn't really the right way to tackle the credit problem. E.g random search is only 10x slower.

In what circumstances is it only 10x slower? Random search is totally useless when your environment is stochastic. These algorithms aren't learning sequences of actions, in fact most use a 30 'no op' random start to avoid just that.

By 'random', he means evolutionary search. It's not really random, and is just a slower method for policy gradient. Here's the OpenAI blog post: https://blog.openai.com/evolution-strategies/

Re: Thoughts on OpenAI, reinforcement learning, and killer robots

#103

Post by jph00 »

backpropaganda wrote: post #95

> Reinforcement learning: the study of teaching computers how to beat Atari. This statement says more about the author and her inability to understand RL than about RL itself. RL doesn't fit fast.ai's "AI is easy" narrative therefore it's not worth doing.

There's nothing hard to understand about RL. I'm not sure where you get that idea - if you find it hard, perhaps you just need to look at some different way. Karpathy summarizes the differences with regular supervised learning in his policy gradient post:

> Policy gradients is exactly the same as supervised learning with two minor differences: 1) We don’t have the correct labels yi so as a “fake label” we substitute the action we happened to sample from the policy when it saw xi, and 2) We modulate the loss for each example multiplicatively based on the eventual outcome, since we want to increase the log probability for actions that worked and decrease it for those that didn’t.

(from http://karpathy.github.io/2016/05/31/rl/ )

Re: Thoughts on OpenAI, reinforcement learning, and killer robots

#104

Post by j2kun »

NortySpock wrote: post #88
peferron wrote: post #61

Earlier quoted context omitted.

Ok, I'll bite. Can you list several things that are vastly more likely to lead to the complete extinction of the human race (I'll assume that's what you meant by "destroy humanity") than malicious AGI?

Climate Change, leading to frequent crop failures, leading to the collapse of most nations and most industrial capacity. That's what keeps me up at night these days.

That would be at the top of my list. And then there's nuclear attacks/disaster, asteroid hitting the earth, superbugs and biological weapons gone awry, physics experiment gone awry creating a black hole... I put superintelligent AI down towards alien invasion on my personal list of humanity-ending risks.

Re: Thoughts on OpenAI, reinforcement learning, and killer robots

#105

Post by ridgeguy »

miguelrochefort wrote: post #37

Regarding "killer robots": If we can't make "paperclip maximization" the main goal of a some human, why do we expect to be able to make it the main goal of some AGI?

Try substituting in "wealth accumulation regardless of adverse effects" for paperclips.

Some humans have adopted that goal. If I understand your view, it should be a goal that an AGI could run with.

I think 'killer robots' might present more subtle dangers than a universe of paperclips.

Re: Thoughts on OpenAI, reinforcement learning, and killer robots

#106

Post by visarga »

philipkglass wrote: post #58

investments into addressing mass unemployment and wealth inequality (both of which are well-documented to cause political instability) I don't really expect to see this solved in the USA before it's solved in poorer countries. In the USA there's a tension between pleasing the millions of surplus workers and pleasing the top billionaires that'll be minted when robots can replace millions of workers. It's cheaper for t…

The day we will invent a self-replicating factory that only uses cheap local raw materials, the current economic system will end. We could say the whole of the economy is a self replicating system, but we need to shrink that to a small size and make it not dependent on rare or contested materials. Humans, genes, and the ecosystem are self replicators as well. Self replication might be a different kind of singularity that we reach even before AGI.

Basically, in order to make a self replicating factory we need advanced 3d-printing, robotics and a large library of schematics. Then, a "physical compiler" could assemble the desired object by orchestrating the various tools and the movement of parts inside the assembly line. If this automated factory can create its own parts, then we have a self replicator. If you make it all open source and ship seed factories around the world, soon everyone will have their own stack to rely on.

Re: Thoughts on OpenAI, reinforcement learning, and killer robots

#107

Post by randcraw »

jorgemf wrote: post #52
jorblumesea wrote: post #4

> Cracking AGI is a very long-term goal. Is this fair to say? I feel like advancements in the field happened far quicker than anyone expected, and every few years we are reevaluating timelines. Especially given the research happening in tandem that will almost certainly speed up AGI, like graphene, quantum computing, advanced GPU design... If you asked someone 10 years ago about the possibilities of ML/Deep learning…

My opinion is that it is a very long-term goal. Compare it to the autonomous car, getting the first 90% of a complete autonomous car is much easier than the last 10%. With AGI probably it is the same. Deep learning has got traction the last years, but we are almost using the same algorithms than 20 years ago, just in better hardware (plus some tweaks to make them work with bigger models). But I think we know almost n…

Agreed. To date, Deep Learning has accomplished outstanding levels of discrimination and classification by combining supervised learning with probabilistic pattern matching. But it's widely acknowledged that most human learning is unsupervised and relies heavily on building upon large hierarchies of facts, their interdependencies, and a deep reliance on causation, none of which DL has yet shown any real facility toward realizing. One-shot examples of one-shot learning doth not a facile brain make.

Until DL shows significant progress toward these -- building knowledgebases and enabling their reuse -- and does so at the superlinear rate of development it's achieved for pattern recognition, I think it's likely that further development of the missing components of AGI will remain gradual and decades away.

Re: Thoughts on OpenAI, reinforcement learning, and killer robots

#108

Post by tlb »

jph00 wrote: post #15
phreeza wrote: post #8

Earlier quoted context omitted.

Calling it an "Atari problem" sounds quite disparaging and misses the point. It's like calling a convolutional network doing the ImageNet task a "Doggy-detection" problem. That may be the original development problem, but the final product still helps detect cancer in CT scan images... Same goes for advances in reinforcement learning made on atari games.

Perhaps, but the jury is still very much out. The vast majority of RL applications are game playing. Very few examples of valuable applications to society or the economy. There's also plenty of evidence already that RL isn't really the right way to tackle the credit problem. E.g random search is only 10x slower.

The virtue of the Atari games is that they can run at 500 FPS per core, so you can train an RL agent in an hour on a small cluster. When working on fundamentals of the algorithms, being able to iterate quickly makes a huge difference in overall progress. Of course, the idea is that once the algorithms work well we can apply them to real problems, where they'll take months or years to learn something valuable.

Re: Thoughts on OpenAI, reinforcement learning, and killer robots

#109

Post by visarga »

jorblumesea wrote: post #4

> Cracking AGI is a very long-term goal. Is this fair to say? I feel like advancements in the field happened far quicker than anyone expected, and every few years we are reevaluating timelines. Especially given the research happening in tandem that will almost certainly speed up AGI, like graphene, quantum computing, advanced GPU design... If you asked someone 10 years ago about the possibilities of ML/Deep learning…

You are right to say

> If you asked someone 10 years ago about the possibilities of ML/Deep learning they'd say it was far off too.

But when you read a little more about recent advances, you gain a healthy respect for the difficulty of the problems that are still left to solve. That tends to make researchers have lower expectations for an easy emergence of AGI than the general public.

We can't even generate a whole page of text that doesn't sound silly, with any neural network or AI algorithm to date. We're a long way off.

Re: Thoughts on OpenAI, reinforcement learning, and killer robots

#110

Post by ridgeguy »

stale2002 wrote: post #27
jacquesm wrote: post #18

Earlier quoted context omitted.

> when we're (say) half way there, the amount of societal upheaval will be enormous. I had a conversation with a C level exec of a large company last week around this theme. My suggestion that limited AI such as self driving cars has the potential to create a vast number of extremely frustrated individuals making a second round of 'Sabotage' and Luddites a definite possibility was waved away as if those people don't…

If self driving cars/trucks become a thing, and cause millions of people to be unemployed, there is ALSO a correspondingly massive economy increase. IE, the world is now massively richer because of all this awesome new technology. Yes, there could be some short term disruption, but honestly I think things will end up fine, just because of the massive amount of extra money and wealth that the world will have that coul…

But it seems that increased economic benefits aren't shared. Thomas Piketty's Capital in the Twenty-First Century, average adjusted worker wages in the US from ~1975 to present, etc.

I'm concerned that 'some short term disruption' will actually be quite widespread and long-lasting, given increasing interdependencies.

Post reply on HN