Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Grandmaster level in StarCraft II using multi-agent reinforcement learning

deepmind.com

Post reply on HN
Print view View original

281–290 of 321 posts

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#281

Post by baddox »

tryptophan wrote: post #149
300bps wrote: post #136

Earlier quoted context omitted.

Starcraft is an incredibly complex game. Even basic strategies will win if they’re done faster. APM (actions per minute) is a very significant factor into who is winning. Apparently they limited their AI player to 264 APM but that’s still incredibly high and done with machine level consistency. That’s almost 4.5 actions per second!! I know there are human level players at and probably above that level but that really…

And the bot also can parse the entire screen in .03 seconds and then jump to a new area of the map. No human can monitor the entire map like the bot can.

If they want to model the constraints of humans, they probably need to create some “attention” system where the AI needs to choose where to invest its finite attention resources. For example, the AI could choose to focus more on the minimap, which would give them higher reaction times to move around the map, but would reduce their reaction times for things on the main screen.

A great example would be seeing the faint image of cloaked units. How does that work with the AI? Can the just phrase the current screen and instantly see any cloaked unit? I can imagine an attention system where more attention dedicated to part of the screen would increase the probability of noticing a cloaked unit there.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#282

Post by chillacy »

linuxftw wrote: post #100
_f9cu wrote: post #46

Why they wouldn't solve more impactful problems rather than playing games?

Because that's all they're capable of doing with it. Games can be easily repeated, have clear outcomes. There's no need for nuanced thinking or actual problem-solving. Just approximate the current scenario against previously successful strategies and pursue that strategy. A real challenge would be to invent a new game, have both a human and an "AI" read the rules for the first time, and then compete in their first ga…

> Just approximate the current scenario against previously successful strategies and pursue that strategy.

Just about sums up business school.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#283

Post by thrower123 »

freddie_mercury wrote: post #279
the_af wrote: post #241

Earlier quoted context omitted.

I think part of the problem is that every videogame is a parlor trick in a sense. Unless you are a tournament player or someone who enjoys a really hard challenge, the bulk of the customers who buy the game just want to enjoy the illusion of a challenge. You don't really need a complex AI directing the hordes of enemies you want to defeat; you just want the illusion of battling enemies and emerging the victor, and ho…

To extend your point, it is also clear from analytics in Steam & mobile gaming that games have massive amounts of player dropout. It is pretty rare, even in relatively short single player campaigns, to see 30% of people who bought the game finish it. So effort making the AI even better is always going to only be for an extremely small niche of players that play a game long enough to exhaust the limits of simple AI. J…

Steam numbers are massively inflated by the tons of game purchases that come through sales, bundles, and giveaways and are never played. I have something like 600 games in my Steam library, and I would bet that the median playtime would be 0.0 hours...

Just today I saw something that looked interesting, and it was 85% off because of a Halloween sale. So I grabbed it for a couple bucks. Free time being such a rare commodity, there's a solid chance I never even install it... and then there's always Stellaris or Mount & Blade or Medieval Total War that I could happily play until the heat death of the universe, competing for my attention.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#284

Post by skybrian »

the_af wrote: post #241
Mirioron wrote: post #214

Earlier quoted context omitted.

> There is a genuine advance here, but keep in mind that when an AI is developed by the game developers, they're not necessarily playing to win, but to make the AI fun to beat, and without using too much computer power, which would make the game slower. This point is being brought up a lot, but I don't really buy it. Yes, there have been instances where the AI being too good discouraged players from playing the game…

I think part of the problem is that every videogame is a parlor trick in a sense. Unless you are a tournament player or someone who enjoys a really hard challenge, the bulk of the customers who buy the game just want to enjoy the illusion of a challenge. You don't really need a complex AI directing the hordes of enemies you want to defeat; you just want the illusion of battling enemies and emerging the victor, and ho…

While there are certainly plenty of parlor tricks and other illusions, I think in the best case, a game can contain a finely tuned learning experience, where by training you actually do get better at something. (Often something unimportant, but still fun.)

Throwing people into the deep end of the pool usually doesn't work so well. Scaling down the difficulty helps, but really what you want is challenges that teach you something. And you don't really need AI for that.

Last year I started learning to play accordion out of a method book. It has a bunch of songs ordered from easy to hard, and they're also chosen to teach certain skills. It reminded me quite a bit of good level design.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#285

Post by freddie_mercury »

thrower123 wrote: post #283
freddie_mercury wrote: post #279

Earlier quoted context omitted.

To extend your point, it is also clear from analytics in Steam & mobile gaming that games have massive amounts of player dropout. It is pretty rare, even in relatively short single player campaigns, to see 30% of people who bought the game finish it. So effort making the AI even better is always going to only be for an extremely small niche of players that play a game long enough to exhaust the limits of simple AI. J…

Steam numbers are massively inflated by the tons of game purchases that come through sales, bundles, and giveaways and are never played. I have something like 600 games in my Steam library, and I would bet that the median playtime would be 0.0 hours... Just today I saw something that looked interesting, and it was 85% off because of a Halloween sale. So I grabbed it for a couple bucks. Free time being such a rare com…

Isn't that just proving my point?

Most people who buy your game aren't going to invest even 10 hours in it or come close to maxing out the AI because they only picked it up because it was 85% off during a Steam sale.

Maybe you'll be able to figure out from Early Access numbers how popular your game is and whether you need to invest in super-awesome AI. But it seems like for most games it would be a waste of effort. More likely it would be retrofitted into an already popular game in a DLC or future patch or something.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#286

Post by eru »

YeGoblynQueenne wrote: post #258
emerongi wrote: post #201

Earlier quoted context omitted.

Most of the approaches used are re-usable, which is a big part of why we can develop new AIs for games faster than ever. You can take the algorithm(s) that was used in one game and use it in another. Yes, a human is still needed to decide which approach to use, but we are slowly approaching a world where building an AI becomes easier and faster. It will become absolutely irrelevant that a single AI can not play all g…

>> The brains of an AI are also transferrable. Built a superhuman AI? Send it to a friend! So far I haven't seen Google sending their AIs to a friend.

They are putting ML inference on the Pixel phones.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#287

Post by sanxiyn »

narrator wrote: post #162

We need to create an open source organization to build these brains at home because: "Recomputing the AlphaGo Zero weights will take about 1700 years on commodity hardware."[1] I see these huge AI brains creating a new class divide between people who have access to these new AI brains and those who don't. The mission of open source has always been to break down these barriers to empowerment with technology. Thus, thi…

AlphaGo Zero was extremely computation inefficient because in (IMO misguided) attempt to be "general", DeepMind avoided even slightly specializing training to Go. KataGo optimized training to Go, slightly, and obtained 100x speedup, and that was low hanging fruit.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#288

Post by sanxiyn »

jdpigeon wrote: post #276

I'm being lazy and don't want to dig, but does anyone know which of the AlphaStar networks (Protoss, Terran, Zerg) is best at beating the others?

They didn't publish cross-race self-play statistics, but AlphaStar's MMR is 6275 for Protoss, 6048 for Terran, and 5835 for Zerg. So it is best at Protoss, and then Terran and Zerg, in that order.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#289

Post by halflings »

Strom wrote: post #212
Erlich_Bachman wrote: post #206

Earlier quoted context omitted.

Yes, those might have some impact, but it is clear that the progress is there, with this new APM cap and camera movement etc. You can also see in replays that the AI often makes mechanical mistakes, missing spells, missing units, even ordering wrong units from outside the screen - so it surely seems that if it's win rate was conditioned in any strong way on its sheer mechanical ability, it would have learned to not m…

The mistakes it makes are due to bad decisions. There have been no claims by DeepMind that they have some sort of chaos engineering [1] going on where the AI decides one thing and then the output system actually does another thing. Also I think you overestimate the AI/IT knowledge of these top players that they're consulting. I have great respect towards them, but they're not renaissance men [2] who both play 10 hour…

>[2] The problem with both parties (the pros & deepmind) is that they're so overspecialized. I'm nowhere near as good as them at their respective fields, but I am a professional programmer and diamond in StarCraft II. In addition I've built StarCraft II AI myself, although with different goals related to finding optimal strategies.

So many things wrong with this comment.

You're nowhere near a professional StarCraft player if you're in Diamond league (I play casually and I'm on high plat, bordering Diamond), and Oriol Vinyals, the lead research scientist behind this project, is one of the most renowned scientists in the field and used to play StarCraft at a professional level. They also said that other employees at DeepMind are at Masters level, and helped test the AlphaStar.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#290

Post by sanxiyn »

hervature wrote: post #253

Having a hard time to parse what is the action space here. The paper claims: AlphaStar’s action space is defined as a set of functions with typed arguments Looking at citation 7, it seems like they are structuring the action space as (First pick high level action)->(Pick argument 1 for action)->...->(Pick argument n for action). If this is the case, this seems to be "cheating" calling this AI as humans have completel…

It is extremely different from AlphaZero... In fact, they heavily rely on human knowledge, which is like opposite of AlphaZero. To quote the paper, "We found our use of human data to be critical in achieving good performance with reinforcement learning".
Post reply on HN