Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

StarCraft Micromanagement with Reinforcement Learning and Curriculum Transfer

arxiv.org

Post reply on HN
Print view View original

41–50 of 54 posts

Re: StarCraft Micromanagement with Reinforcement Learning and Curriculum Transfer

#41

Post by MikkoFinell »

TulliusCicero wrote: post #36
WhompingWindows wrote: post #33

Creating a micromanagement AI is only the first step. This must be piped into a macro AI, which produces the units with with micromanagement can be done. Then, another critical layer beyond micro or macro is scouting the opponent and predicting what their plan is. Next, you need to adapt your own strategy, which must also be planned at the start of the match since you can't scout instantaneously or perfectly. Finally…

I don't think anyone doubts computers will eventually beat pros in Starcraft. I do think it will take quite a bit longer though, even with recent advancements seen in the likes of AlphaGo. From a human perspective, Starcraft doesn't seem "harder" than Chess or Go, but it has more fluidity and ambiguity, both things computers struggle with. And the ruleset, as it were, is VASTLY more complicated.

I disagree that Starcraft doesn't seem harder. The strategic depth is arguably comparable, but the difference is you need the dexterity of a master pianist to execute.

Re: StarCraft Micromanagement with Reinforcement Learning and Curriculum Transfer

#42

Post by xg15 »

jon_richards wrote: post #40
WhompingWindows wrote: post #33

Creating a micromanagement AI is only the first step. This must be piped into a macro AI, which produces the units with with micromanagement can be done. Then, another critical layer beyond micro or macro is scouting the opponent and predicting what their plan is. Next, you need to adapt your own strategy, which must also be planned at the start of the match since you can't scout instantaneously or perfectly. Finally…

SC2 has a lot of things that help the player over an AI, like imperfect information, a very large ruleset, and continuous movement and time. The main advantage AI can exploit is "infinite micro" units. There are certain units in SC2 that are primarily limited by someone's ability to pay attention to them. Terran Reapers, for instance, in early patches could dance back and forth just within range Zerg Roaches, but out…

It depends what your goals are, but those development nevertheless sound useful.

If you want to stay "human-comparable", it sounds as if it wouldn't be hard to artificially impose those restriction on an AI - e.g. you could simulate a "viewport" that the AI can move around and restrict the micro engine to only take inputs from within that viewport, etc.

(Assuming the AI could make use of that advantage anyway. If I remember correctly, there is a lot of research going on that uses "attention" itself as a primitive in AI architecture. So depending on the architecture, AI might not always be able to make use of that advantage.)

On the other hand, an AI that does use infinite-micro strategy might be useful for UI research. E.g., you could pit it against a human player and use the human player's performance as a benchmark for UI improvements.

Re: StarCraft Micromanagement with Reinforcement Learning and Curriculum Transfer

#43

Post by littlestymaar »

jon_richards wrote: post #40
WhompingWindows wrote: post #33

Creating a micromanagement AI is only the first step. This must be piped into a macro AI, which produces the units with with micromanagement can be done. Then, another critical layer beyond micro or macro is scouting the opponent and predicting what their plan is. Next, you need to adapt your own strategy, which must also be planned at the start of the match since you can't scout instantaneously or perfectly. Finally…

SC2 has a lot of things that help the player over an AI, like imperfect information, a very large ruleset, and continuous movement and time. The main advantage AI can exploit is "infinite micro" units. There are certain units in SC2 that are primarily limited by someone's ability to pay attention to them. Terran Reapers, for instance, in early patches could dance back and forth just within range Zerg Roaches, but out…

> like imperfect information

The information is only imperfect for a human but an IA can have a way better access to information: - if you scout you opponent mineral line and click on the minerals, you know exactly how much minerals he's harvested since the beginning of the game. If you scout your opponent army and building, you know exactly how much he spent. Then you can deduce if he has something hidden somewhere or in production. Of course this is impossible to do for a human, but really easy for a computer. - scouting is easy but requires a lot of actions, an IA could have one mutalisk wandering over the map the whole game without being killed, which is impossible for a human unless you dedicate full time on this.

Imho, beating humans to StarCraft isn't a big deal in itself, but it would need a lot of work to build the AI to deal with the game's complexity. But it's merely domain specific work that has little to no interest on AI in general. Creating an AI that trains itself how to be good at StarCraft will take much more times and effort, but at least it will provide value outside of the specific problem.

Re: StarCraft Micromanagement with Reinforcement Learning and Curriculum Transfer

#44

Post by llarsson »

daemonk wrote: post #4

I want an AI assisted mode with API hooks where people can code their own sophisticated "macros" that can be activated with a command/button. Limit the code to X number of characters. It would be awesome to see a tournament with this.

https://github.com/Blizzard/s2client-api is at least a thing

Re: StarCraft Micromanagement with Reinforcement Learning and Curriculum Transfer

#45

Post by animal531 »

fosco wrote: post #25
daemonk wrote: post #4

I want an AI assisted mode with API hooks where people can code their own sophisticated "macros" that can be activated with a command/button. Limit the code to X number of characters. It would be awesome to see a tournament with this.

if this were available in true form, would that reduce the game to the fastest unit no matter what the damage. as an example see the zerg vs tank below. I understand the idea of code manipulation but what might be better is just having human programmed AI's fight each other (which I am fairly sure is going and can be viewed live but I do not have the link handy.

When playing humans though the AI's can be limited in certain fashions, such as by lowering the available actions per minute (overall), or per unit; which lowers their non-human skills.

Re: StarCraft Micromanagement with Reinforcement Learning and Curriculum Transfer

#46

Post by ajuc »

s_m_t wrote: post #9
calebh wrote: post #6

For those who don't know Starcraft: microing refers to the small scale/precise maneuvering of units during a battle, as well as activating special unit abilities at certain times. In the lower level leagues microing is not as important as having a good overall large-scale "macro" strategy. The microing really starts coming into play at the higher levels where it can give the edge needed to win a battle. I am not very…

How can this be so? In my experience playing Starcraft the decision space for macro is very, very, very limited. You are essentially locked into a handful of viable build orders for the early game, a handful of timing attack opportunities in the early and mid game, and after that things veer so off track that very few players do more than wing it.

> and after that things veer so off track that very few players do more than wing it

If you ignore the part where you have lots of choice, then there's little choice, indeed :)

Re: StarCraft Micromanagement with Reinforcement Learning and Curriculum Transfer

#47

Post by vk23 »

littlestymaar wrote: post #43
jon_richards wrote: post #40

Earlier quoted context omitted.

SC2 has a lot of things that help the player over an AI, like imperfect information, a very large ruleset, and continuous movement and time. The main advantage AI can exploit is "infinite micro" units. There are certain units in SC2 that are primarily limited by someone's ability to pay attention to them. Terran Reapers, for instance, in early patches could dance back and forth just within range Zerg Roaches, but out…

> like imperfect information The information is only imperfect for a human but an IA can have a way better access to information: - if you scout you opponent mineral line and click on the minerals, you know exactly how much minerals he's harvested since the beginning of the game. If you scout your opponent army and building, you know exactly how much he spent. Then you can deduce if he has something hidden somewhere…

> Then you can deduce if he has something hidden somewhere or in production. Of course this is impossible to do for a human, but really easy for a computer.

This is definitly not impossible for humans. High level SC players are able to keep track of the mineral and gas spendings of their opponent in the early game (After the early game there is usually no complete information to calculate this). Just seeing the timing of taking the gas geysers gives you information about the earliest possible time a certain unit/building can be produced.

Of course a AI can scout more and take more variables into account but humans are surprisingly good at this too.

> Imho, beating humans to StarCraft isn't a big deal in itself, but it would need a lot of work to build the AI to deal with the game's complexity. But it's merely domain specific work that has little to no interest on AI in general.

The same thing was said about early chess AIs and the Alpha GO project. In the end both lead to major advancements in AI. If you solve the domain specific problem starcraft you could easily apply the gained knowledge to similiar problems.

Re: StarCraft Micromanagement with Reinforcement Learning and Curriculum Transfer

#48

Post by ajuc »

WhompingWindows wrote: post #31
s_m_t wrote: post #9

Earlier quoted context omitted.

How can this be so? In my experience playing Starcraft the decision space for macro is very, very, very limited. You are essentially locked into a handful of viable build orders for the early game, a handful of timing attack opportunities in the early and mid game, and after that things veer so off track that very few players do more than wing it.

The problem here is the conflation of "macro" with "strategy". If the parent comment had defined the terms properly, there'd be less disagreement here. They rightly define micromanagement as the control and use of small numbers of units, whereas macromanagement refers to the management of the non-fighting, production related activities like producing workers constantly until saturated, producing supply depots on time…

Marines in TvZ would be so OP with perfect splits...

Re: StarCraft Micromanagement with Reinforcement Learning and Curriculum Transfer

#50

Post by jon_richards »

vk23 wrote: post #47
littlestymaar wrote: post #43

Earlier quoted context omitted.

> like imperfect information The information is only imperfect for a human but an IA can have a way better access to information: - if you scout you opponent mineral line and click on the minerals, you know exactly how much minerals he's harvested since the beginning of the game. If you scout your opponent army and building, you know exactly how much he spent. Then you can deduce if he has something hidden somewhere…

> Then you can deduce if he has something hidden somewhere or in production. Of course this is impossible to do for a human, but really easy for a computer. This is definitly not impossible for humans. High level SC players are able to keep track of the mineral and gas spendings of their opponent in the early game (After the early game there is usually no complete information to calculate this). Just seeing the timin…

At the top levels, it's a lot more about where those units/buildings are than whether they exist. You know they're going to be mining and spending efficiently, so just seeing the number of workers gives you a very good idea of how much other stuff they have. Pros also check to see what technologies are being researched (buildings have a different animation while researching).

You can't get much more scouting than that because there is a cost associated with scouting, either because you lose a worker's time or because you spread out your army.

Post reply on HN