DeepMind and Blizzard Open StarCraft II as an AI Research Environment • Hacker News Board

Re: DeepMind and Blizzard Open StarCraft II as an AI Research Environment

#251

Post by FrozenVoid » Thu, Aug 10, 2017, 12:18 PM UTC

gradys wrote: post #198

qub1t wrote: post #71

A lot of people here seem to be underestimating the difficulty of this problem. There are several incorrect comments saying that in SC1 AIs have already been able to beat professionals - right now they are nowhere near that level. Go is a discrete game where the game state is 100% known at all times. Starcraft is a continuous game and the game state is not 100% known at any given time. This alone makes it a much hard…

I think a big component is not really machine learning but more related to how to represent state at any given time, which will necessarily involve a lot of human-tweaking of distilling down what really are the important things that influence winning. I agreed with everything you said until here. Developing good representations of state is precisely what today's machine learning is so good at. This is the key contrib…

>a ConvNet to process individual frames combined with a recurrent layer to track state over time. > are harder to apply Thats an understatement: Starcraft is immune to Monte-Carlo approach or anything based on analyzing pixel data: The tree state of actual battle has thousands of choices pet unit per second with minor variations in location, there is no discrete state of chessboard(at best millions of cells): viewing the game at low-level(pixels) creates gigantic amount of data. units constantly move/attack/die and get blocked by other units/terrain.

Predicting an enemy move(MC simulation) will be impossible and you can make several moves per second(even at 120-140 APM) easily. That means 1.you need real-time response, unlike Go there isn't a time buffer to decide 2.you always need to react at the current time(or allowing enemy advances) 3.there are very few "good moves" in starcraft(moving randomly on the "board" will just waste time) , so MC simulation will miss them more than 99% of time due randomness.

MC approach is vastly inferior in this case, i think they'll be forced to operate on higher level strategy rather than just microing every unit optimally(i.e. treating it like chess in real-time). Brute-forcing billions of potential moves simply won't work.

Re: DeepMind and Blizzard Open StarCraft II as an AI Research Environment

#252

Post by _vufv » Thu, Aug 10, 2017, 12:33 PM UTC

yflu wrote: post #19

Havoc wrote: post #10

That's surprising. I thought Bliz didn't want anyone near sc2 but approved of sc1 being used for this purpose.

SC1 really doesn't make sense for this, 80% of the skill is just keeping on top of the mindless but mechanically intensive stuff, which is trivial beyond trivial for an AI. SC2's automated away most of this (pretty much everything but production cycles), which makes it a better measure for AI vs human.

> ... 80% of the skill is just keeping on top of the mindless but mechanically intensive stuff, which is trivial beyond trivial for an AI.

This statement is wrong. SC1 has admittedly less tech-tree depth & strategic approaches than SC2 does (purely because of lower number of different units/upgrades), but there are innumerable variations that are imperceivable to the lay observer.

I'll go on a limb to say that SC1 has more refined rock-paper-scissors system than SC2 ever had (taboo to speak of on reddit).

Re: DeepMind and Blizzard Open StarCraft II as an AI Research Environment

#253

Post by avar » Thu, Aug 10, 2017, 12:56 PM UTC

uw_rob wrote: post #200

avar wrote: post #191

Earlier quoted context omitted.

You say you understand the point but you seem to have entirely missed it, the same problem remains. The point of the APM limit is presumably to fairly emulate a human player, but the APMs for humans are obviously averaged over a minute. However, for certain things humans can initiate actions in SC2 in quick succession that, if sustained over a greater time period, would yield a ridiculous APM rate. Think a Terran pla…

> the APMs for humans are obviously averaged over a minute This is wrong. I don't know the exact number, but APM is averaged over around a second. I suspect this is done because APM is a more meaningful compared to APS, for humans at least. Here is a graph from Scelight that high lights this: https://goo.gl/photos/9cjNxDwWoB1pmWkg9

You still end up with a computer that can perform 3 actions all within the first millisecond of a second and still end up with human-like 180 APM (3 APS), even though no human could replicate what it just did.

Re: DeepMind and Blizzard Open StarCraft II as an AI Research Environment

#254

Post by moh_maya » Thu, Aug 10, 2017, 12:59 PM UTC

littlestymaar wrote: post #172

littlestymaar wrote: post #158

Earlier quoted context omitted.

As a long time StarCraft fan I don't share your point of view : People usually refer to StarCraft as a strategy game but there's actually really little strategy involved : during the first weeks after a new map pool is released, the pro players explore different build orders that are strong on it. And after this period, when the meta-game has settled, the winner of a match (best of 3 or 5) is almost always the one wh…

Actually I realize that my cyclist vs motorcycle analogy is better that I first thought : You might think that a race between the winner of the last Tour de France and an automated motorcycle is a good challenge because automated driving is hard (especially with crowd running around on the road and temporary road signs for the race circuit). But in fact it wouldn't be funny because all the motorcycle have to do is fo…

That's only assuming they are competing at the same time. An average of timed trials across a pre-defined route (like TDF) would allow you to present a similar challenge to both the human & the automated motor cycle.

Couldn't an analogous structure be used to assess the AI for the SC II as well?

Disclaimer: I only know about SC. Not really a player.

Re: DeepMind and Blizzard Open StarCraft II as an AI Research Environment

#255

Post by Double_Cast » Thu, Aug 10, 2017, 1:26 PM UTC

bluetwo wrote: post #162

littlestymaar wrote: post #158

Earlier quoted context omitted.

As a long time StarCraft fan I don't share your point of view : People usually refer to StarCraft as a strategy game but there's actually really little strategy involved : during the first weeks after a new map pool is released, the pro players explore different build orders that are strong on it. And after this period, when the meta-game has settled, the winner of a match (best of 3 or 5) is almost always the one wh…

Interesting. So would you say that there are two parts here, figuring out a general strategy for a new map and then maximizing execution?

Skill is often divided into 3 components: macro-management; micro-management; and mechanics.

Macro refers to decisions regarding economy. It includes finances, build order, counters, etc. Macro is mostly strategic.

Micro refers to decisions regarding battle. It includes troop positioning, focus fire, kiting, etc. Micro is mostly tactical.

Mechanics refers to execution. I.e. do your fingers have the dexterity and APM to accomplish your goals effectively? If not, practice makes perfect.

Re: DeepMind and Blizzard Open StarCraft II as an AI Research Environment

#256

Post by Double_Cast » Thu, Aug 10, 2017, 1:33 PM UTC

bluetwo wrote: post #162

littlestymaar wrote: post #158

Earlier quoted context omitted.

As a long time StarCraft fan I don't share your point of view : People usually refer to StarCraft as a strategy game but there's actually really little strategy involved : during the first weeks after a new map pool is released, the pro players explore different build orders that are strong on it. And after this period, when the meta-game has settled, the winner of a match (best of 3 or 5) is almost always the one wh…

Interesting. So would you say that there are two parts here, figuring out a general strategy for a new map and then maximizing execution?

It's possible you're referring to the tradeoff between exploration vs exploitation.

Re: DeepMind and Blizzard Open StarCraft II as an AI Research Environment

#257

Post by empath75 » Thu, Aug 10, 2017, 1:45 PM UTC

bluetwo wrote: post #162

littlestymaar wrote: post #158

Earlier quoted context omitted.

As a long time StarCraft fan I don't share your point of view : People usually refer to StarCraft as a strategy game but there's actually really little strategy involved : during the first weeks after a new map pool is released, the pro players explore different build orders that are strong on it. And after this period, when the meta-game has settled, the winner of a match (best of 3 or 5) is almost always the one wh…

Interesting. So would you say that there are two parts here, figuring out a general strategy for a new map and then maximizing execution?

Except you can only execute your plan until you have contact with your opponent, and then it's improvisation. The rest of the game is only 'easy' if you've executed your opening far better than your opponent, or you have a rock-paper-scissors situation where your build dominates theirs. Both situations are not very common at the pro level.

Re: DeepMind and Blizzard Open StarCraft II as an AI Research Environment

#258

Post by Double_Cast » Thu, Aug 10, 2017, 1:54 PM UTC

dyarosla wrote: post #183

TulliusCicero wrote: post #166

Earlier quoted context omitted.

It's not literally continuous, but it is real-time rather than turn-based, and positions of units are essentially floats rather than (a small range of) ints. That makes it effectively continuous (too large to just generate a tree of all possible actions and then prune).

Are you sure positions of units are essentially floats? Given how the units seem to arrange themselves (from what I see), I would guess that it's not close to the full range of floats, and instead there are just a few fractional pixel locations that units snap to. This is just a guess however. -- If this is the case though, the space could be represented by taking larger integer values (say, a magnitude of 1 or 2 hig…

Buildings snap to a grid. Units take up space according to a hitbox. Hitbox size varies according to each type of unit (E.g. Thors are huge). This becomes important when dealing with AoE.

Consider a group of mutalisks. If you select-all and issue an attack-command or move-command, the mutalisks will bunch up tight and then disperse. Cf a video on the "magic-box technique".

So I wouldn't be surprised if position-values were floats.

Re: DeepMind and Blizzard Open StarCraft II as an AI Research Environment

#259

Post by kaden » Thu, Aug 10, 2017, 2:00 PM UTC

FrozenVoid wrote: post #251

gradys wrote: post #198

Earlier quoted context omitted.

I think a big component is not really machine learning but more related to how to represent state at any given time, which will necessarily involve a lot of human-tweaking of distilling down what really are the important things that influence winning. I agreed with everything you said until here. Developing good representations of state is precisely what today's machine learning is so good at. This is the key contrib…

>a ConvNet to process individual frames combined with a recurrent layer to track state over time. > are harder to apply Thats an understatement: Starcraft is immune to Monte-Carlo approach or anything based on analyzing pixel data: The tree state of actual battle has thousands of choices pet unit per second with minor variations in location, there is no discrete state of chessboard(at best millions of cells): viewing…

>Brute-forcing billions of potential moves simply won't work.

The problem is all AI/ML is essentially recorded, recursive, constrained brute forcing.

Re: DeepMind and Blizzard Open StarCraft II as an AI Research Environment

#260

Post by FrozenVoid » Thu, Aug 10, 2017, 2:17 PM UTC

kaden wrote: post #259

FrozenVoid wrote: post #251

Earlier quoted context omitted.

>a ConvNet to process individual frames combined with a recurrent layer to track state over time. > are harder to apply Thats an understatement: Starcraft is immune to Monte-Carlo approach or anything based on analyzing pixel data: The tree state of actual battle has thousands of choices pet unit per second with minor variations in location, there is no discrete state of chessboard(at best millions of cells): viewing…

>Brute-forcing billions of potential moves simply won't work. The problem is all AI/ML is essentially recorded, recursive, constrained brute forcing.

You can apply it on higher level like that guy who bruteforced the 7roach rush for Zerg in SC2. http://lbrandy.com/blog/2010/11/using-genetic-algorithms-to-... Problem is that build orders are just optimizing the opening economy and these unbalanced openings will be just patched out in the future.