We all know that Mew2King is first reinforcement learning AI capable of beating Super Smash Bros pro players.
https://www.youtube.com/watch?v=z-1YfhUFtbY&feature=youtu.be...
Beating the World’s Best at Super Smash Bros. with Deep Reinforcement Learning
21–30 of 59 posts
Re: Beating the World’s Best at Super Smash Bros. with Deep Reinforcement Learning
#22Note: it doesn't learn from pixels but features directly from RAM; and superhuman reaction time, with performance badly degrading when human-like delays added. Good discussions on Reddit: https://www.reddit.com/r/MachineLearning/comments/5vh4ae/r_a... https://www.reddit.com/r/smashbros/comments/5vin8x/beating_t...
I could see this technology used for the bootstrapping of highly emergent MMO game worlds. It could be used to populate a world with fake "player" NPCs that are actually part of a simulated online ecosystem. Give the NPCs a large enough population, such that players cannot exert significant selection pressure, but give the NPCs real selection pressure through interaction with artificial life evolved with Genetic Algo…
League of legends for example has bots appear in PvP games.
While these bots are not produced by the game's developers not a lot was done then to get rid of these things. I guess they were tolerated since it just make the queue times smaller for human players.
( http://boards.na.leagueoflegends.com/en/c/gameplay-balance/b... )
Re: Beating the World’s Best at Super Smash Bros. with Deep Reinforcement Learning
#23We all know that Mew2King is first reinforcement learning AI capable of beating Super Smash Bros pro players. https://www.youtube.com/watch?v=z-1YfhUFtbY&feature=youtu.be...
and he still can't beat Armada
Re: Beating the World’s Best at Super Smash Bros. with Deep Reinforcement Learning
#24Earlier quoted context omitted.
This reminds me of Starcraft AI experiments. They can't actually make the computer smart, so they just jam 2000 button presses per second down the tube, giving every single unit its own simultaneous AI, and it out micromanages anyone. With Marines usually.
I heard that the DeepMind Starcraft project intends to limit their AI's APM (actions per minute) down to something human-like.
I read that too but I hope they know the difference between APM and EPM. Pros spam APM that they could never do real actions on but their EPM is considerably lower (if the bots make actions based on pros APM they will have an insurmountable advantage).
Re: Beating the World’s Best at Super Smash Bros. with Deep Reinforcement Learning
#25Earlier quoted context omitted.
Part of the skill in competitive play is to be able to predict what move your opponent is going to do next. Most mid-level players already have a good grasp of prediction, which is arguably along the sames lines of being able to know with certainty what action your opponent is taking a few frames before he does it. Coupling that with pretty obscene frame-lag for Smash, it's not really that much of an advantage. As we…
You're wildly swinging between advantageous and not. No, this is just playing games. The ground rules must be clear: you get the screenshots and keyboard input in every frame, as a normal player. If the resulting AI sucks, who cares? Failure is part of doing science.
> You're wildly swinging between advantageous and not.
What?
> The ground rules must be clear: you get the screenshots and keyboard input in every frame, as a normal player.
Perhaps if you want to start from flawed assumptions/ want to create an AI that's tweak-able to appear as human. Which would be pretty useful and practical for other applications, but not competitive play.
We could go on and on about digital vs. analog, but digital is good enough for your argument and doesn't require you to spend enormous resources on a trivial pursuit.
This going in the direction of nonsensical handicaps. You don't give AlphaGo stamina parameters that artificially slow down processing speed. You give it all the tools it needs to beat a human player.
Re: Beating the World’s Best at Super Smash Bros. with Deep Reinforcement Learning
#26Note: it doesn't learn from pixels but features directly from RAM; and superhuman reaction time, with performance badly degrading when human-like delays added. Good discussions on Reddit: https://www.reddit.com/r/MachineLearning/comments/5vh4ae/r_a... https://www.reddit.com/r/smashbros/comments/5vin8x/beating_t...
This reminds me of Starcraft AI experiments. They can't actually make the computer smart, so they just jam 2000 button presses per second down the tube, giving every single unit its own simultaneous AI, and it out micromanages anyone. With Marines usually.
Or individual muta micro. That was the winning "strategy" in the first BWAI cup many years ago.
Re: Beating the World’s Best at Super Smash Bros. with Deep Reinforcement Learning
#27Earlier quoted context omitted.
I could see this technology used for the bootstrapping of highly emergent MMO game worlds. It could be used to populate a world with fake "player" NPCs that are actually part of a simulated online ecosystem. Give the NPCs a large enough population, such that players cannot exert significant selection pressure, but give the NPCs real selection pressure through interaction with artificial life evolved with Genetic Algo…
League of legends for example has bots appear in PvP games. While these bots are not produced by the game's developers not a lot was done then to get rid of these things. I guess they were tolerated since it just make the queue times smaller for human players. ( http://boards.na.leagueoflegends.com/en/c/gameplay-balance/b... )
Bots has come a long way through Guild Wars where they were basically fancy scripts to inducing AI in the start to becoming chatbots and having bots capable to do elite areas with teams "undetected" (knew the person within the guild, slipped up on guildrun using multiple instances of ghosts in ts). There was also pvp (different bots) bots that can/could do alot more than predetermined patterns.
In pve they became so common that the game economy became completely based off them (without the majority's knowledge).
The intelligence of other bots/programs to give a player an unfair advantage of some sort has also come an awfully long way. One example is the leaps the aimbot took in Halo PC, from being easy to detect even when used by top players to being nearly indetectible (except when priority issues or other bugs/glitches appear).
Re: Beating the World’s Best at Super Smash Bros. with Deep Reinforcement Learning
#28Earlier quoted context omitted.
I heard that the DeepMind Starcraft project intends to limit their AI's APM (actions per minute) down to something human-like.
I read that too but I hope they know the difference between APM and EPM. Pros spam APM that they could never do real actions on but their EPM is considerably lower (if the bots make actions based on pros APM they will have an insurmountable advantage).
Pros spam APM to keep warm, during battles or production macros they will frequently have a high EPM as well.
Re: Beating the World’s Best at Super Smash Bros. with Deep Reinforcement Learning
#29Earlier quoted context omitted.
I was similarly disappointed when I read this, but upon further reflection I still like this paper. It is very plausible that both of these problems could be fixed, it would just take a lot more time/power to train, and the resulting system would likely not run in real time making it impossible to test against real humans. Further advancement in this area will require huge leaps in hardware performance. Luckily in th…
I'm not nearly that pessimistic. Beating SSBM is well within the capability of a well-tuned A3C, and definitely within the capabilities of a group like DeepMind. More neuromorphic hardware is unnecessary and with current RL methods, they are more CPU-bound than GPU-bound (take a look at the NN they use, it's trivially small; most of the computation goes towards running many SSB games in parallel in order to generate…
The main blocker on using pixels has been getting them from the emulator. I doubt pixels would give a big advantage over RAM features, especially after projectile and stage info is added (there's a PR pending). Captain Falcon (the main character used) doesn't have projectiles anyways.
In fact, RAM features are likely to be much more useful for model-based approaches, which may be important for solving the action-delay problems.
As for multiple characters, the character ID is available to the network. I doubt pixels will be help there either.
Re: Beating the World’s Best at Super Smash Bros. with Deep Reinforcement Learning
#30Note: it doesn't learn from pixels but features directly from RAM; and superhuman reaction time, with performance badly degrading when human-like delays added. Good discussions on Reddit: https://www.reddit.com/r/MachineLearning/comments/5vh4ae/r_a... https://www.reddit.com/r/smashbros/comments/5vin8x/beating_t...
we instead use features read from the game’s memory on each frame, consisting of each player’s position, velocity, and action state, along with several other values So it's cheating, presumably knowing the opponents action before the animation even starts to play.
It does not see the opponent's actions before they take effect on screen, and the actual controller states are not part of the feature representation we used (though they actually are somewhere in the RAM).