Beating the World’s Best at Super Smash Bros. with Deep Reinforcement Learning

Post by erik » Thu, Feb 23, 2017, 3:54 AM UTC

SerLava wrote: post #6

gwern wrote: post #3

Note: it doesn't learn from pixels but features directly from RAM; and superhuman reaction time, with performance badly degrading when human-like delays added. Good discussions on Reddit: https://www.reddit.com/r/MachineLearning/comments/5vh4ae/r_a... https://www.reddit.com/r/smashbros/comments/5vin8x/beating_t...

This reminds me of Starcraft AI experiments. They can't actually make the computer smart, so they just jam 2000 button presses per second down the tube, giving every single unit its own simultaneous AI, and it out micromanages anyone. With Marines usually.

Post by nickpsecurity » Thu, Feb 23, 2017, 4:02 AM UTC

gwern wrote: post #18

modeless wrote: post #5

Earlier quoted context omitted.

I was similarly disappointed when I read this, but upon further reflection I still like this paper. It is very plausible that both of these problems could be fixed, it would just take a lot more time/power to train, and the resulting system would likely not run in real time making it impossible to test against real humans. Further advancement in this area will require huge leaps in hardware performance. Luckily in th…

I'm not nearly that pessimistic. Beating SSBM is well within the capability of a well-tuned A3C, and definitely within the capabilities of a group like DeepMind. More neuromorphic hardware is unnecessary and with current RL methods, they are more CPU-bound than GPU-bound (take a look at the NN they use, it's trivially small; most of the computation goes towards running many SSB games in parallel in order to generate…

Post by dyselon » Thu, Feb 23, 2017, 6:22 AM UTC

willwhitney wrote: post #11

jwtadvice wrote: post #10

While the AI might be cheating by taking salient features from RAM rather than from pixel values, this is still an incredible feat. Just a few years ago we did not have generic algorithms that could take even salient features and self-learn policies to near this level this quickly.

Yup, it's definitely an advantage to get all the correct values from the game state. But not as much as you might think; the vision portion of a DQN or similar trains quite quickly. Plus, our bot doesn't have any clue about projectiles. We don't know where they live in memory, so the network doesn't get to know about them at all.

Post by Natsu » Thu, Feb 23, 2017, 6:33 AM UTC

avbor wrote: post #38

malikNF wrote: post #22

Earlier quoted context omitted.

League of legends for example has bots appear in PvP games. While these bots are not produced by the game's developers not a lot was done then to get rid of these things. I guess they were tolerated since it just make the queue times smaller for human players. ( http://boards.na.leagueoflegends.com/en/c/gameplay-balance/b... )

Well, Riot solved that by getting rid of Dominion. ;) But seriously, they were mainly present in the game mode mentioned, and now in Co-Op vs AI, just because they want to level the account and sell it. Doesn't really affect the majority of the population until selling point, and once it's been sold, it's easy to tell a botted account.

Post by stupidcar » Thu, Feb 23, 2017, 10:53 AM UTC

nickpsecurity wrote: post #42

SerLava wrote: post #6

Earlier quoted context omitted.

This reminds me of Starcraft AI experiments. They can't actually make the computer smart, so they just jam 2000 button presses per second down the tube, giving every single unit its own simultaneous AI, and it out micromanages anyone. With Marines usually.

That's not what Starcraft AI field is about. They actually started with a combo of people doing planner-oriented systems and micro-oriented systems. Hybrids followed that. There's many methods at play. Here's a survey: https://www.cs.mun.ca/~dchurchill/pdf/starcraft_survey.pdf The competitions that involved humans showed humans destroyed them by spotting their patterns and beating those patterns. Also with bluffing o…