Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Grandmaster level in StarCraft II using multi-agent reinforcement learning

deepmind.com

Post reply on HN
Print view View original

81–90 of 321 posts

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#81

Post by minblaster »

baddox wrote: post #70
interblag wrote: post #59

This is a really interesting one to digest. As with previous announcements about AlphaStar, much of the feedback (here and elsewhere) is about the fundamental challenge of assessing human vs. machine in an RTS. These points are very valid - stepping back however, this still feels like a pretty incredible accomplishment. I'm a gold league SC2 player, so maybe in the 30th-50th percentile. Three years ago, when DeepMind…

> The speed at which AlphaStar overtook previous AIs seems to me to be nearly unprecedented in AI research. Is it not simply the case that, before AlphaStar, very little money and effort was being put into developing AIs for Starcraft 2?

Starcraft is an incredibly complex game. 10^26 possible moves at any point (you can click/drag anywhere on the screen, pressing a keyboard button as you do so), imperfect information, real-time constraints, etc.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#83

Post by Nimitz14 »

diggan wrote: post #47
Nimitz14 wrote: post #35

The title seems to contradict the subtitle: > Google AI beats top human players at strategy game StarCraft II vs > DeepMind’s AlphaStar beat all but the very best humans at the fast-paced sci-fi video game.

Yeah, subtitle seems much better. My impression before reading was that it did beat the top players but reading the articles makes it clear it beats everyone BUT the top players... Also, this part seems a bit weird from the article: > The AI wasn’t able to beat the best player in the world, as AIs have in chess and Go, but DeepMind considers its benchmark met, and says it has completed the StarCraft II challenge. So…

They're framing it positively to distract from the fact they can't do it, the reality is they've spent millions of $ on compute and yet their agent is still terrible at strategy (and tactics sometimes too), they probably decided it's best to stop now before sinking even more money into it.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#84

Post by gwern »

baddox wrote: post #70
interblag wrote: post #59

This is a really interesting one to digest. As with previous announcements about AlphaStar, much of the feedback (here and elsewhere) is about the fundamental challenge of assessing human vs. machine in an RTS. These points are very valid - stepping back however, this still feels like a pretty incredible accomplishment. I'm a gold league SC2 player, so maybe in the 30th-50th percentile. Three years ago, when DeepMind…

> The speed at which AlphaStar overtook previous AIs seems to me to be nearly unprecedented in AI research. Is it not simply the case that, before AlphaStar, very little money and effort was being put into developing AIs for Starcraft 2?

That depends on what you consider 'very little'. SC has had annual computer tournaments and research was published on it routinely. Few computer games see nearly as much money and effort put into developing AIs for them, yet the curve of progress was not exactly impressive.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#85

Post by goalieca »

jamra wrote: post #71
username90 wrote: post #31

Earlier quoted context omitted.

They are testing the AI's ability to do tactics and strategy, not motor and visual tasks. It has built in delay and needs to move around the map to gather information just like a human, so it doesn't have any significant unfair advantages. Edit: Note that the version they sent out to the ladder had significant larger delay, significant lower APM, and didn't get any information not visible on the screen unlike the fir…

However the computer can macro while in a fight which is something people can’t dI. We would miss the fight and lose the game. It’s not just a tactics game. Attention span and where your eyes are matter.

It’s so bad that below master people shouldn’t really be doing too much micro during a fight because they’ll lose out on macro. Even in pro matches you see attention span issues and sometimes avoidance of too much micro to win. Really good Micro is going to win you any battle.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#86

Post by Nimitz14 »

jplayer01 wrote: post #69
king-rat wrote: post #18

Not being robust to strategies it hasn't seen before is a serious shortcoming in a real time strategy game. That also indicates an interesting flaw in how this model is trained - in the millions of games it plays against itself, how do you ensure that it tries every viable (and some inviable) strategies? Sure, it couldn't best Serral but I wonder how it would fare against Has, a player known for some pretty off the w…

> Not being robust to strategies it hasn't seen before is a serious shortcoming in a real time strategy game. I'm not sure if you've actually played or followed competitive SC2. This is absolutely normal . Players will pull something completely unexpected out of a hat and win. The losing player will learn from it in future games. That's just how it goes. Unexpected strategies are really hard to counter when you've ne…

I don't think you understood his point. A player doesn't need to have seen a strategy before to react correctly to it. Sometimes a player will pull something completely unexpected out of a hat and lose, because the other guy reacted correctly thanks to his game experience. If you watch any high GM player stream half the time his reaction to what his (worse) opponent is doing is "WTF is this?" as he then proceeds to crush it. Alphastar cannot do that.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#87

Post by [deleted] »

Nimitz14 wrote: post #39
nightcracker wrote: post #5

> After 50 games, however, DeepMind hit a snag. Some players had noticed that three user accounts on the Battle.net gaming platform had played the exact same number of StarCraft II games over a similar time frame — the three accounts that AlphaStar was secretly using. When watching replays of these matches, players noticed that the account owner was performing actions that would be extremely difficult, if not impossi…

Funny how they put it as: > When watching replays of these matches, players noticed that the account owner was performing actions that would be extremely difficult, if not impossible, for a human. framing it like the agent was doing some sort insane play, when the reality is the main evidence of a player being alphastar (other than its garbage decision making) was the fact that it wasn't using hotkeys!

[deleted]

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#88

Post by justicezyx »

floitsch wrote: post #16
nightcracker wrote: post #5

> After 50 games, however, DeepMind hit a snag. Some players had noticed that three user accounts on the Battle.net gaming platform had played the exact same number of StarCraft II games over a similar time frame — the three accounts that AlphaStar was secretly using. When watching replays of these matches, players noticed that the account owner was performing actions that would be extremely difficult, if not impossi…

These extremely difficult/impossible things didn't really give an advantage. For example, AlphaStar would sometimes click on an object at the border of the screen. For humans that would be almost impossible, because the screen would scroll when the mouse approaches the border. Similarly, AlphaStar would not play with group hotkeys, but use a different technique. However, in none of the analyses, people noticed things…

What you said is a meaningful advtange.

Scrolling screen is a major action to gather info and control units.

If one can control near edge units without scrolling it gives more stable view and lower chance of making mistakes.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#89

Post by kingkawn »

bsdetector wrote: post #72
username90 wrote: post #31

Earlier quoted context omitted.

They are testing the AI's ability to do tactics and strategy, not motor and visual tasks. It has built in delay and needs to move around the map to gather information just like a human, so it doesn't have any significant unfair advantages. Edit: Note that the version they sent out to the ladder had significant larger delay, significant lower APM, and didn't get any information not visible on the screen unlike the fir…

When Watson won Jeopardy, its lead designer thought a 5 ms reaction time was fair because one time a human anticipating the buzzer light got 2 ms. Of course to any reasonable person a consistent 5 ms is obviously vastly superior to once in a blue moon lottery-winning luck. Sounds like it's the same deal here, where the designers made some effort to level the interface playing field, but also left in some advantages l…

Yes, the point of the entire project was for the system to see if it can win.

If we have reached the point where to even risk a loss requires implementing artificial restraints on the AI...well...

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#90

Post by interblag »

baddox wrote: post #70
interblag wrote: post #59

This is a really interesting one to digest. As with previous announcements about AlphaStar, much of the feedback (here and elsewhere) is about the fundamental challenge of assessing human vs. machine in an RTS. These points are very valid - stepping back however, this still feels like a pretty incredible accomplishment. I'm a gold league SC2 player, so maybe in the 30th-50th percentile. Three years ago, when DeepMind…

> The speed at which AlphaStar overtook previous AIs seems to me to be nearly unprecedented in AI research. Is it not simply the case that, before AlphaStar, very little money and effort was being put into developing AIs for Starcraft 2?

There's definitely some truth to that. But looking back on arXiv there are papers going back years, including from Harvard/UofT [1], Tencent [2], Facebook [3], etc. There has also been an SC1 tournament running since at least 2011 [4]

I'm sure that the AlphaStar effort absolutely dwarfed everything that came before in terms of investment (as I'm sure AlphaGo did for Go as well) but based on my reading I'd also bet that Starcraft has likely seen the most continued AI research effort of any imperfect information, real-time game over the last decade - I think that's also part of what made it appealing for DeepMind over Dota 2 or any of the other options in this class of game.

[1] https://arxiv.org/pdf/1909.02682.pdf [2] https://arxiv.org/pdf/1907.09467.pdf [3] https://arxiv.org/pdf/1906.12266.pdf [4] https://liquipedia.net/starcraft/SSCAIT

Post reply on HN