Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AlphaStar: Mastering the Real-Time Strategy Game StarCraft II

deepmind.com

Post reply on HN
Print view View original

181–190 of 459 posts

Re: AlphaStar: Mastering the Real-Time Strategy Game StarCraft II

#181

Post by dx87 »

jgon wrote: post #150
wnevets wrote: post #115

Earlier quoted context omitted.

>I think a far more interesting limitation would be to cap APM at 150 or so, or to artificially limit action precision with some sort of virtual mouse that reduced accuracy as APM increased. IIRC OpenAI limits the reaction time to ~200ms when playing DoTA2. AI employing better strategies than humans will always be more interesting than AI that can out click humans.

Even the 200ms reaction time seemed overly slanted towards the AI. I don't think that is the actual reaction time of top pros, in the matches the AI played the human player would teleport in from complete invisibility and try to use an instant cast spell and the AI would have already teleported out. Yes the theoretically may have been constrained to a 200ms reaction time, but in practice the AI was playing at a super…

Another advantage was that the AI is just reading the game state through an API, it doesn't have to look on the screen. The game can be difficult to watch from a pro's perspective since they have to constantly click around the map to see what's happening, but the AI has perfect knowledge of everything it is capable of seeing, all without having to physically move a mouse to click on the screen.

Re: AlphaStar: Mastering the Real-Time Strategy Game StarCraft II

#182

Post by derefr »

dcl wrote: post #164
derefr wrote: post #146

Earlier quoted context omitted.

Another way to think about it is that a human brain is mostly doing transfer-learning, on top of a 99%-baked deep net that was wired up during foetal development from our DNA, where that DNA-persisted model has "seen" hundreds of millions of years of training data. Humans don't have to learn to process, recognize, and classify objects in visual sense-data, for example. We can do that from the moment we're born, becau…

This is a widely underappreciated fact when it comes to comes to comparing the 'training experience' of humans versus bots. And it extends far beyond processing 'sense data' - A human likely has some level of understanding of how the game works based on experience from other games it has played and from 'real life' - we know almost instinctively that 'high ground' is likely to give a combat advantage without having t…

Not only that, humans (and many other eusocial species) have an instinctual intuitional understanding of many aspects of game theory.

For example, humans, even from infancy, prefer games where it is possible to punish cheating (i.e. take revenge upon cheaters) to games where it is not. This isn't just "we're animals that have evolved to enact tit-for-tat strategies [by e.g. injustice triggering rage] because they lead to cooperation which leads to egalitarian utility"; this is actual analysis—instantaneous, intuitive analysis—of a system of rules, to notice, in advance of ever being slighted, whether you'll be likely to end up in an "unjust" social situation if you agree to the given ruleset. There is an "accelerated co-processor" of high-level abstract game-theoretic information—and layers to extract that information from sense-data—that ship as part-and-parcel of the human brain model. We never need to learn how to judge unfairness, any more than we need to learn how to see.

Re: AlphaStar: Mastering the Real-Time Strategy Game StarCraft II

#183

Post by gwern »

jaredtn wrote: post #145

I'd be interested to see its hierarchical strategy and planning, especially across such a long timespan. Does anyone have any good references for similar hierarchical planning work (Feudal Networks, etc.) to look at?

It doesn't use hierarchical approaches at all, apparently. Just flat Impala with LSTMs and CNNs every tick.

Re: AlphaStar: Mastering the Real-Time Strategy Game StarCraft II

#184

Post by b_tterc_p »

freeflight wrote: post #162
sl1ck731 wrote: post #137

Earlier quoted context omitted.

The purpose of limiting speed isn't to make an interesting contest, it is to accurately compare the "math" instead of the speed the math is done at. It isn't surprising that its fast, the surprising part is that it can make human-like decisions. The only way to compare whether its thinking is human-like is to restrain it from "brute forcing" the contest through speed. The model has likely learned that the faster it d…

But isn't the competency of a Starcraft player is also measured on his/her speed? In that context, you can't really measure strategy without accounting for timing/speed because a lot of tactics and strategies only become viable once the player has the required speed to actually realize them aka "micro".

Exactly, and due to superhuman micro, the AI has cornered itself into learning a small subset of the strategy space. It’s not good at strategy because it’s optimized itself for just getting into micro-handled situations.

It’s not good at strategizing with all the options available to it given it’s micro ability, it has “one” strategy that leveraged the micro as much as it could, and when given a strategic challenge by mana, it didn’t know what to do.

Re: AlphaStar: Mastering the Real-Time Strategy Game StarCraft II

#185

Post by CydeWeys »

celeritascelery wrote: post #174
taneq wrote: post #158

Earlier quoted context omitted.

"Yes but X has a tiny problem space compared to something like Y. People want to see an AI win because it's smart, not because it crunches numbers." 1980: X = Tic-tac-toe, Y = Chequers 1990: X = Chequers, Y = Chess 2000: X = Chess, Y = Go 2019: X = Go, Y = StarCraft 2030: X = Any video game, Y = ???

Is a AI that wins at Starcraft only because it has crazy high APM really going to help get to the next X? We could have built that 10 years ago. All it proves is that computers have faster reflexes then humans. That won’t help them become problem solvers for the future.

Speed of play is a fundamentally important gameplay mechanic of any real-time game. One of the main reasons the pros are better than amateurs at these types of game is because they play and react faster.

And yes, of course computers are much better at doing things more quickly than humans. It's not even remotely close for us. The AIs are clearly better. It's not cheating either; they are legitimately better at it than us.

It sounds like you're simply objecting to pitting people up against computers in real-time games entirely.

Re: AlphaStar: Mastering the Real-Time Strategy Game StarCraft II

#186

Post by arcticfox »

ehsankia wrote: post #165
rcheu wrote: post #46

This is really impressive, I didn't expect starcraft to be played this well by a machine learning based AI. I'm excited to read the paper when it comes out! That said, I'm not sure I agree that it was winning mainly due to better decision making. For context, I've been ranked in the top 0.1% of players and beaten pros in Starcraft 2, and also work as a machine learning engineer. The stalker micro in particular looked…

> While they have similar APM to SC2 pros Wasn't the APM closer to half that of the pros? https://storage.googleapis.com/deepmind-live-cms/images/SCII...

This is super deceiving and I'm kind of upset they posted this image, knowing it would mislead people not familiar with the game. The AI sits around during lulls at During the fights, the critical moments in when MaNa would top out at ~600 humanly inaccurate APM (this is 10 inputs per second), the AI would jump up to over 1000 - we don't know exactly what it was doing, but it was presumably pixel-precise. Meanwhile the physical inertia of the mouse is a challenge for humans at that speed - imagine trying to click five totally different places with perfect precision in a single second.

Re: AlphaStar: Mastering the Real-Time Strategy Game StarCraft II

#187

Post by MereInterest »

freeflight wrote: post #172
andreyk wrote: post #74

Earlier quoted context omitted.

Also worth noting that it starts by imitation learning from pros. I'd be curious to see if the macro can be learned without imitation; a much harder challenge. Also, playing with full visibility as was mostly the case in the demonstration is quite lame...

If you learn, why not learn from the best, the pros? These people already have spent years figuring out what works and what doesn't. Why not draw from that pool of knowledge and instead spend extra time going through the same motions?

Because then you don't know whether the AI learned by experimentation or by mimicking. To draw an analogy, imagine the difference between somebody reading and following an algorithm to solve a Rubik's cube, as opposed to somebody being handed a Rubik's cube and experimenting. If expert-level strategies can be reproduced without being explicitly shown to the person/AI, then it means something is going right in your methodology.

Re: AlphaStar: Mastering the Real-Time Strategy Game StarCraft II

#188

Post by b_tterc_p »

cjbprime wrote: post #151
celeritascelery wrote: post #88

Earlier quoted context omitted.

Mana got himself in the same situation where he was surrounded by stalkers on multiple sides, but this time the micro wasn’t so crazy that he couldn’t manage it, and he was able to take on one group at a time. The immortal drop, while unanswered, was not really that effectual.

But it was answered: AlphaStar pulled a huge stalker army that was about to hit MaNa's base all the way back home to (attempt to) answer the drop, repeatedly. If you have more complexity to your army but fewer army units, as MaNa did, a delay like that is how you win the game.

It’s funny because this works against the standard Ai too.

Re: AlphaStar: Mastering the Real-Time Strategy Game StarCraft II

#189

Post by asdfasgasdgasdg »

jgon wrote: post #150
wnevets wrote: post #115

Earlier quoted context omitted.

>I think a far more interesting limitation would be to cap APM at 150 or so, or to artificially limit action precision with some sort of virtual mouse that reduced accuracy as APM increased. IIRC OpenAI limits the reaction time to ~200ms when playing DoTA2. AI employing better strategies than humans will always be more interesting than AI that can out click humans.

Even the 200ms reaction time seemed overly slanted towards the AI. I don't think that is the actual reaction time of top pros, in the matches the AI played the human player would teleport in from complete invisibility and try to use an instant cast spell and the AI would have already teleported out. Yes the theoretically may have been constrained to a 200ms reaction time, but in practice the AI was playing at a super…

The game is difficult to watch, but does anyone honestly believe that an AI is going to have a difficult time parsing the scene if it is trained to do so? That to me just seems like a question of resources. We're pretty good at image recognition and segmentation now, and that's without the unlimited amounts of training data one could generate when using a controlled game environment with a limited range of possible animations and effects. This is why I find the prospect of the AI agent having to parse the screen entirely uninteresting.

Re: AlphaStar: Mastering the Real-Time Strategy Game StarCraft II

#190

Post by thomasfortes »

dx87 wrote: post #181
jgon wrote: post #150

Earlier quoted context omitted.

Even the 200ms reaction time seemed overly slanted towards the AI. I don't think that is the actual reaction time of top pros, in the matches the AI played the human player would teleport in from complete invisibility and try to use an instant cast spell and the AI would have already teleported out. Yes the theoretically may have been constrained to a 200ms reaction time, but in practice the AI was playing at a super…

Another advantage was that the AI is just reading the game state through an API, it doesn't have to look on the screen. The game can be difficult to watch from a pro's perspective since they have to constantly click around the map to see what's happening, but the AI has perfect knowledge of everything it is capable of seeing, all without having to physically move a mouse to click on the screen.

Yeah, put an eye tracker in a pro and you'll see that the eyes are constantly changing the focus point, if you can watch the entire scene with the same precision without the need to focus on it you're already at a nice advantage.

As an aside, a few pro gamers prefer to play on windowed mode for exactly this reason.

Post reply on HN