31–40 of 46 posts
Is it really true that Montezuma's Revenge is more challenging for DeepMind than Go, as they mention in the article?
This is interesting but it's still far from how a human would learn how to play the game. Humans don't have inbuilt rewards for Montezuma's Revenge, they acquire them culturally. How much of what was learned (by the machine, not the researchers) in playing Montezuma's Revenge could be applied to a game like Zelda? A human would instantly notice many of the connections between the two games: enemies that follow simple…
This is interesting but it's still far from how a human would learn how to play the game. Humans don't have inbuilt rewards for Montezuma's Revenge, they acquire them culturally. How much of what was learned (by the machine, not the researchers) in playing Montezuma's Revenge could be applied to a game like Zelda? A human would instantly notice many of the connections between the two games: enemies that follow simple…
A human would instantly notice many ...
[1] J. Schmidhuber. Formal theory of creativity, fun, and intrinsic motivation.
Earlier quoted context omitted.
Isn't DeepMind just an umbrella term like Watson? I would guess that AlphaGo and the software that learns games like this are distinct.
DeepMind is the name of an AI company that was acquired by Google. https://deepmind.com/ AlphaGo is the program that beat Go. These game playing algorithms are variations of Deep Q Learning Reinforcement Learning algos.
This is interesting but it's still far from how a human would learn how to play the game. Humans don't have inbuilt rewards for Montezuma's Revenge, they acquire them culturally. How much of what was learned (by the machine, not the researchers) in playing Montezuma's Revenge could be applied to a game like Zelda? A human would instantly notice many of the connections between the two games: enemies that follow simple…
I think if you watch a child play anything, even a video game, you'll see that when it's not immediately obvious what to do, a human will just 'mess around' and 'try stuff out' to 'just to see what happens'. As the paper says, the idea of novelty bonuses in reinforcement learning draws directly from discussions of curiosity and play and intrinsic motivation in animals and humans.
And this still doesn't address my original question. A human who was able to master Montezuma's Revenge would have a dramatic advantage in learning and mastering the game Zelda compared to somebody who had played neither game before. What experience, if any, could this machine be expected to bring to Zelda, assuming no modification by the researchers?
This is interesting but it's still far from how a human would learn how to play the game. Humans don't have inbuilt rewards for Montezuma's Revenge, they acquire them culturally. How much of what was learned (by the machine, not the researchers) in playing Montezuma's Revenge could be applied to a game like Zelda? A human would instantly notice many of the connections between the two games: enemies that follow simple…
I'm confused by the subtitle of the linked article, which says "The AI system was able to solve the complex game in just four tries". But the video shows the AI dying many more times than that, and not ultimately winning the game, just learning to explore a portion of it successfully. What did the Wired editor mean by "solve" and "four tries"? (Or, for that matter, "complete"?)
Earlier quoted context omitted.
DeepMind is the name of an AI company that was acquired by Google. https://deepmind.com/ AlphaGo is the program that beat Go. These game playing algorithms are variations of Deep Q Learning Reinforcement Learning algos.
DeepMind's main intellectual property, the reason why it was acquired, was DeepMind's reinforcement algorithm. The present paper, like the work on AlphaGo, is about improving this reinforcement algorithm.
So, DeepMind's greatest asset is they employ some really excellent people and have a substantial head start in terms of actually implementing AIs and getting them to work.
This is interesting but it's still far from how a human would learn how to play the game. Humans don't have inbuilt rewards for Montezuma's Revenge, they acquire them culturally. How much of what was learned (by the machine, not the researchers) in playing Montezuma's Revenge could be applied to a game like Zelda? A human would instantly notice many of the connections between the two games: enemies that follow simple…
So you use existing tools to encourage the player to survive, and then you add additional motivation to encourage the player to learn what it needs to do [ie explore]. And exploration is a common theme in games like Zelda.
I actually think that this technique will generalize to learning lots of games [eg the same engine might play Zelda & Montezuma's Revenge], but I don't think techniques in this vein alone will allow it to learn how to play one game and then immediately understand how to play the other.
[but "transfer learning" is a separate, challenging problem in AI]
This is interesting but it's still far from how a human would learn how to play the game. Humans don't have inbuilt rewards for Montezuma's Revenge, they acquire them culturally. How much of what was learned (by the machine, not the researchers) in playing Montezuma's Revenge could be applied to a game like Zelda? A human would instantly notice many of the connections between the two games: enemies that follow simple…
I think this technique -- in the title of the paper -- would transfer somewhat to other games, because it essentially encourages the player to go places it hasn't been [without simultaneously having it explore everything thoroughly]. So you use existing tools to encourage the player to survive, and then you add additional motivation to encourage the player to learn what it needs to do [ie explore]. And exploration is…
Moreover, we already have machines that are extremely proficient at solving very complex games when given enough context: expert systems. One very notable example is the bot which successfully completed the game NetHack [0]. Would DeepMind's novelty-based reward technique work for NetHack?