Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Beating the World’s Best at Super Smash Bros. with Deep Reinforcement Learning

arxiv.org

Post reply on HN
Print view View original

51–59 of 59 posts

Re: Beating the World’s Best at Super Smash Bros. with Deep Reinforcement Learning

#51

Post by Blackthorn »

willwhitney wrote: post #11
jwtadvice wrote: post #10

While the AI might be cheating by taking salient features from RAM rather than from pixel values, this is still an incredible feat. Just a few years ago we did not have generic algorithms that could take even salient features and self-learn policies to near this level this quickly.

Yup, it's definitely an advantage to get all the correct values from the game state. But not as much as you might think; the vision portion of a DQN or similar trains quite quickly. Plus, our bot doesn't have any clue about projectiles. We don't know where they live in memory, so the network doesn't get to know about them at all.

Getting them from RAM instead of the screen doesn't give you an advantage on (for example) DI or ledge teching?

Re: Beating the World’s Best at Super Smash Bros. with Deep Reinforcement Learning

#52

Post by Humdeee »

hkmurakami wrote: post #26
SerLava wrote: post #6

Earlier quoted context omitted.

This reminds me of Starcraft AI experiments. They can't actually make the computer smart, so they just jam 2000 button presses per second down the tube, giving every single unit its own simultaneous AI, and it out micromanages anyone. With Marines usually.

Or individual muta micro. That was the winning "strategy" in the first BWAI cup many years ago.

I remember this. And even the first bot vs bot competition was won by a clever bot scripter simply denying the gas early on from his opponent. Took all his competition down to a standstill.

Re: Beating the World’s Best at Super Smash Bros. with Deep Reinforcement Learning

#53

Post by WhitneyLand »

What's the key insight here compared to previous systems?. As far as I can tell, still no one can beat simple non-deterministic games that require some planning.

My favorite example is Ms. Pac Man because it seems so old and simplistic. Been tried by a dozen teams and no one can beat a decent human.

Re: Beating the World’s Best at Super Smash Bros. with Deep Reinforcement Learning

#54

Post by jbf1001 »

stcredzero wrote: post #19
gwern wrote: post #3

Note: it doesn't learn from pixels but features directly from RAM; and superhuman reaction time, with performance badly degrading when human-like delays added. Good discussions on Reddit: https://www.reddit.com/r/MachineLearning/comments/5vh4ae/r_a... https://www.reddit.com/r/smashbros/comments/5vin8x/beating_t...

I could see this technology used for the bootstrapping of highly emergent MMO game worlds. It could be used to populate a world with fake "player" NPCs that are actually part of a simulated online ecosystem. Give the NPCs a large enough population, such that players cannot exert significant selection pressure, but give the NPCs real selection pressure through interaction with artificial life evolved with Genetic Algo…

Or even some kind of "Always online" MMO like chronicles of elyria. Being able to just tell your character to play while you are away without 'scripting' them would be nice.

https://chroniclesofelyria.com/

Re: Beating the World’s Best at Super Smash Bros. with Deep Reinforcement Learning

#55

Post by jensv »

smaili wrote: post #8

As someone who's played for quite a while I can tell you SSBM is one of the most complex games I've ever come across.

Why do you think the game is complex? Fairly simple game with low barrier to entry which is great when you invite guests over for games. Super Simple Button Mash!

Re: Beating the World’s Best at Super Smash Bros. with Deep Reinforcement Learning

#56

Post by chrisdbaldwin »

jensv wrote: post #55
smaili wrote: post #8

As someone who's played for quite a while I can tell you SSBM is one of the most complex games I've ever come across.

Why do you think the game is complex? Fairly simple game with low barrier to entry which is great when you invite guests over for games. Super Simple Button Mash!

Likely due to the advanced, non-intuitive mechanics that have been discovered over the years. The entry barrier may be low, but the skill cap is high.

Re: Beating the World’s Best at Super Smash Bros. with Deep Reinforcement Learning

#57

Post by crystalPalace »

guest wrote: post #27
malikNF wrote: post #22

Earlier quoted context omitted.

League of legends for example has bots appear in PvP games. While these bots are not produced by the game's developers not a lot was done then to get rid of these things. I guess they were tolerated since it just make the queue times smaller for human players. ( http://boards.na.leagueoflegends.com/en/c/gameplay-balance/b... )

Bots has come a long way through Guild Wars where they were basically fancy scripts to inducing AI in the start to becoming chatbots and having bots capable to do elite areas with teams "undetected" (knew the person within the guild, slipped up on guildrun using multiple instances of ghosts in ts). There was also pvp (different bots) bots that can/could do alot more than predetermined patterns. In pve they became so…

Runescape's economy was largely based on bots and the economy crashed when Jagex finally purged them all.

Re: Beating the World’s Best at Super Smash Bros. with Deep Reinforcement Learning

#58

Post by forgotmysn »

Sniffnoy wrote: post #44
forgotmysn wrote: post #23

Earlier quoted context omitted.

and he still can't beat Armada

I am possibly being here the person who accidentally takes the joke literally, but Mew2King has in fact beaten Armada on three occasions: Once at SKTAR 3, once at Smash Summit 2, and most recently at UGC Smash Open.

haha i know he has, but M2K isn't performing the way he used to and the record is like 12-3 in favor of Armada, if I'm not mistaken

Re: Beating the World’s Best at Super Smash Bros. with Deep Reinforcement Learning

#59

Post by vladfi1 »

dyselon wrote: post #45
willwhitney wrote: post #11

Earlier quoted context omitted.

Yup, it's definitely an advantage to get all the correct values from the game state. But not as much as you might think; the vision portion of a DQN or similar trains quite quickly. Plus, our bot doesn't have any clue about projectiles. We don't know where they live in memory, so the network doesn't get to know about them at all.

Can I ask what the feature set looked like? I always kind of wanted to do this with the Skullgirls AI, but never had the time while we were developing it. As a developer, I obviously had full access to the game state, but I'm still not really sure what the best way to represent that state to a neural network is.

It was just basic stuff like player positions, velocities, and animation states.
Post reply on HN