21–30 of 46 posts
What did the Wired editor mean by "solve" and "four tries"? (Or, for that matter, "complete"?)
That said, game theory has always been an excellent way to analyze AI systems. And using "modern" games (which generally provide attractive skins over a classic mechanic) certainly makes it easier to watch/sit through. When DeepMind starts beating people playing Diplomacy then we'll know we're in a whole new game.
I'm confused by the subtitle of the linked article, which says "The AI system was able to solve the complex game in just four tries". But the video shows the AI dying many more times than that, and not ultimately winning the game, just learning to explore a portion of it successfully. What did the Wired editor mean by "solve" and "four tries"? (Or, for that matter, "complete"?)
within a fraction of the training time, our agent explores a significant portion of the first level and obtains significantly higher scores than previously published agents
... it's probably just the usual hype.
If you end up blocked by a popup blocker, or just don't feel like reading fluff, or like me you've got the browser locked down too tightly for their integrated video player to work, the paper is at https://arxiv.org/pdf/1606.01868v1.pdf and the video mentioned is (probably, since I didn't see the original) at https://www.youtube.com/watch?v=0yI2wJ6F8r0 .
I have a feeling DeepMind AI will specialize in playing video games
I look forward to the day when you are taunted on the Starcraft boards for playing "like an AI" :-) That said, game theory has always been an excellent way to analyze AI systems. And using "modern" games (which generally provide attractive skins over a classic mechanic) certainly makes it easier to watch/sit through. When DeepMind starts beating people playing Diplomacy then we'll know we're in a whole new game.
So lets say I take the trained network, and flip the color of the pixels, and make some other cosmetic changes (keeping the game intact). Will the network then solve it in the first try?
Is it really true that Montezuma's Revenge is more challenging for DeepMind than Go, as they mention in the article?
Montezuma's Revenge, Castle Wolfenstein (not the shooter), and puzzle games in general have a problem of long term credit assignment and sparse reward. This "intrinsic reward" approach form the paper, based on pseudo counts seems to be one way to get an intermediate reward which helps the model learn toward an overall goal (winning/progress) which happens rarely. The previous best work had to pre-define the intrinsic…
On a somewhat related note, it seems clear that AI research and breakthroughs are occurring at breakneck speed. I wish there was a place where you could see expert commentary like your in layman terms on interesting or important papers that stand out.
Earlier quoted context omitted.
Montezuma's Revenge, Castle Wolfenstein (not the shooter), and puzzle games in general have a problem of long term credit assignment and sparse reward. This "intrinsic reward" approach form the paper, based on pseudo counts seems to be one way to get an intermediate reward which helps the model learn toward an overall goal (winning/progress) which happens rarely. The previous best work had to pre-define the intrinsic…
Thank you, that was an excellent summary! On a somewhat related note, it seems clear that AI research and breakthroughs are occurring at breakneck speed. I wish there was a place where you could see expert commentary like your in layman terms on interesting or important papers that stand out.