Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Human-level control through deep reinforcement learning

nature.com

Post reply on HN
Print view View original

51–60 of 63 posts

Re: Human-level control through deep reinforcement learning

#51

Post by meric »

dwaltrip wrote: post #38
ratsimihah wrote: post #33

Earlier quoted context omitted.

How about you read the article and get a few books on the relevant topics? It would probably be much cheaper than going back to school.

Very good point. I taught myself web dev (now working at a pretty awesome startup) so I'm definitely familiar with that route. I have a few cool AI ideas I'm hoping to start spending more time in the coming months, and I have heard of some great online courses to check out. I was just curious as to how important institutional credentials are for this kind of thing, seeing as it much more academic than building CRUD w…

I think there are lots of AI competitions where you can join and make your name.

Re: Human-level control through deep reinforcement learning

#54

Post by meric »

bsdetector wrote: post #27
bmh100 wrote: post #7

> ...the authors used the same algorithm, network architecture, and hyperparameters on each game... This is huge. It shows that the algorithm was able to generalize across multiple problem sets within the same domain of "playing Atari 2600 games", and not simply a "lucky" choice of algorithm, network architecture, or hyperparameters that a random search for each game might choose. This is also not a violation of the…

> the algorithm was able to generalize across multiple problem sets Did it really? I think they reset it and retrained it for each game. I'd like to know how much more is needed to make one instance of the AI that can successfully play any of the games. To play all 49 games that it could learn, does it need to be an extra level deep? Or 49 times larger? Or 2^49 times more?

Can one human successfully play all of the games without prior practice in each game? As far as I know a human has to practice almost every game to be able to play all of them without losses. I think for an AI, achieving this standard is a good result - first practicing in each game and then play through all of them without losses.

Re: Human-level control through deep reinforcement learning

#55

Post by ya3r »

craftit wrote: post #53

It is an amazingly powerful technique. We've been working on a service which lets you do this kind of learning with any JSON stream. You can see a demo here: https://aiseedo.com/demos/cookiemonster/

The amazing part of what DeepMind has achieved is its capability to learn from raw pixel input with deep convolutional neural networks, which as I understand it quite different from what you do.

Still the reinforcement learning part is the same, but reinforcement learning was not the main contribution of this nature paper.

Re: Human-level control through deep reinforcement learning

#56

Post by morenoh149 »

j_m_b wrote: post #13

It is interesting how they are using various biological models to develop their own model. They gave their model a reward system and a memory. It will be interesting to see how far deep Q-networks can be extended and at what point they hit the wall of diminishing returns. |Nevertheless, games demanding more temporally extended planning strategies still constitute a major challenge for all existing agents including DQ…

I expect it could go far. Mind you I only did parts of Artificial Intellegence: A modern approach but the Q-learning algorthim seems very flexible. https://en.wikipedia.org/wiki/Q-learning It basically keeps doing good stuff, while exploring to get out of local minimas.

Re: Human-level control through deep reinforcement learning

#57

Post by arvinjoar »

albertzeyer wrote: post #50

An interesting critic by Schmidhuber about this publication: https://plus.google.com/100849856540000067209/posts/eLQf4KC9...

Seems to be critiquing the claim that this is "groundbreaking" and not much else. Nice to get some further context though. :)

Re: Human-level control through deep reinforcement learning

#58

Post by craftit »

ya3r wrote: post #55
craftit wrote: post #53

It is an amazingly powerful technique. We've been working on a service which lets you do this kind of learning with any JSON stream. You can see a demo here: https://aiseedo.com/demos/cookiemonster/

The amazing part of what DeepMind has achieved is its capability to learn from raw pixel input with deep convolutional neural networks, which as I understand it quite different from what you do. Still the reinforcement learning part is the same, but reinforcement learning was not the main contribution of this nature paper.

Its not all that different, we take multiple asynchronous streams of messages integrate them into a coherent predictive model, and use that to feed the reinforcement learning. The messages can contain images, a simple case can be seen in the demo with a 1d vision sensor.

Re: Human-level control through deep reinforcement learning

#59

Post by dabeeeenster »

dwaltrip wrote: post #30
sjtrny wrote: post #16

Watch it play: http://www.nature.com/nature/journal/v518/n7540/extref/natur... http://www.nature.com/nature/journal/v518/n7540/extref/natur...

This is so cool. I'd love to work on this stuff... Anyone know how hard it would be for someone who is fairly good at programming (works as a full stack developer and feels quite comfortable learning new things) and has strong math skills (undergrad degree) to break into this field? Is going back to school for a masters/phd the best way?

I did an M.SC in CogSci a long way back (1998) and funnily enough my thesis area was very very close to this (reinforcement learning with different network topologies).

The core area is pretty simple stuff to be honest. Obviously DeepMind are completely next-level but you can get pretty good results with basic understanding...

Re: Human-level control through deep reinforcement learning

#60

Post by bsdetector »

meric wrote: post #54
bsdetector wrote: post #27

Earlier quoted context omitted.

> the algorithm was able to generalize across multiple problem sets Did it really? I think they reset it and retrained it for each game. I'd like to know how much more is needed to make one instance of the AI that can successfully play any of the games. To play all 49 games that it could learn, does it need to be an extra level deep? Or 49 times larger? Or 2^49 times more?

Can one human successfully play all of the games without prior practice in each game? As far as I know a human has to practice almost every game to be able to play all of them without losses. I think for an AI, achieving this standard is a good result - first practicing in each game and then play through all of them without losses.

A person that learns 1 game will learn the next game much faster, because they have learned a concept such as a bullet or switch or reflection or wrapping. We take this for granted, but there was a time when Breakout was actually marginally fun because it was new.

A person that's played all the other games in the list can win Montezuma's Revenge on the first try; this AI can't play Montezuma's Revenge at all.

Post reply on HN