Live data from Hacker News

DeepMind and Blizzard Open StarCraft II as an AI Research Environment

deepmind.com

231–240 of 283 posts

Re: DeepMind and Blizzard Open StarCraft II as an AI Research Environment

#231

Earlier quoted context omitted.

>The big advancement in alphago is that by using deep learning it is able to evaluate different board-states without doing any search, using a neural net. It still uses a Monte-Carlo Tree Search to get to the level where it can beat human pro players. >Starcraft is real-time, there are tons of different actions you can take, the actions are not independent (pressing attack does something different if you have a unit…

What are their other options besides Starcraft2? This doesn’t seem like a PR stunt (not that the PR isn’t a bonus), but there’s already a history of AI competitions for Brood War, the game is more balanced than arguably any other RTS, and even though it is “primitive” as a strategy game in your estimation, AI isn’t ready to tackle a more advanced strategy game.

>What are their other options besides Starcraft2?

Uh, real strategy games? Something like this:

https://www.gog.com/game/m_a_x_m_a_x_2

Very simple ruleset, huge strategic depth.

Re: DeepMind and Blizzard Open StarCraft II as an AI Research Environment

#232
post #71

A lot of people here seem to be underestimating the difficulty of this problem. There are several incorrect comments saying that in SC1 AIs have already been able to beat professionals - right now they are nowhere near that level. Go is a discrete game where the game state is 100% known at all times. Starcraft is a continuous game and the game state is not 100% known at any given time. This alone makes it a much hard…

I known nothing about what they are trying to solve, but it would be interesting if their goal was not just to beat humans but to make a game AI that was actually fun to play.

Re: DeepMind and Blizzard Open StarCraft II as an AI Research Environment

#233

Earlier quoted context omitted.

Why not first allow the AI to have unlimited APM and beat humans, then restrict it later? Because I don't think we're even close to the easier problem.

In what universe is taking 10,000 actions per minute an easier problem for a neural net than 100 such actions?

"In what universe is taking 10,000 actions per minute an easier problem for a neural net than 100 such actions?"

StarCraft is precisely such universe. If you could micromanage units perfectly, you can do some amazing tricks. Here's an example of what I'm talking about: https://www.youtube.com/watch?v=IKVFZ28ybQs

Re: DeepMind and Blizzard Open StarCraft II as an AI Research Environment

#235
post #198

Earlier quoted context omitted.

I think a big component is not really machine learning but more related to how to represent state at any given time, which will necessarily involve a lot of human-tweaking of distilling down what really are the important things that influence winning. I agreed with everything you said until here. Developing good representations of state is precisely what today's machine learning is so good at. This is the key contrib…

i feel like you misunderstood that part of the argument. he is saying representing the state is very hard, and you are saying: given a well represented state, ML is very good at finding the important features, reducing the dementionality, and finding mathematical transformations, etc. deep learning has been so successful with images because representing them is trivial - flattened pixel vector. with your last paragra…

People should just read the article, I think. It answers all the things you are debating (limit on APM, what features are used, what models they already tried and how well they perform).

Re: DeepMind and Blizzard Open StarCraft II as an AI Research Environment

#236
post #71

A lot of people here seem to be underestimating the difficulty of this problem. There are several incorrect comments saying that in SC1 AIs have already been able to beat professionals - right now they are nowhere near that level. Go is a discrete game where the game state is 100% known at all times. Starcraft is a continuous game and the game state is not 100% known at any given time. This alone makes it a much hard…

As a long-time high level SC2 player, one additional thing that makes SC2 so difficult is that the game has multiple layers of tactics and strategy that require specialized logic, but those layers also interact and synergize in a deep way.

- There is the overall strategic game of 'Who is ahead economically? Given that, should I be expanding, attacking, or defending?', with the implicit understanding that the player with the current economic advantage puts pressure on its opponent to attack - There is a resource management and build-order system where you need to plan and optimize building as big and as effective a unit composition as quickly as possible, except there are a lot of tradeoffs: you can build for a stronger army sooner, as opposed to a weaker army alter - There is a tactical micromanagement battle where small groups of units are pitted against one another, and where small tactical movements can gain very large materiel advantages. Units are relatively short ranged, so to damage or defend effectively requires effective positioning. Most armies fight better as a cohesive group ('ball'), except there are units that specifically punish and do splash damage that need individual micromanagement. Battles can take place over a short period and be over quickly, or can be long-running positional skirmishes that last for half the game, where each player is constantly probing for weakness before one finally goes for the throat. - The economy fundamentally depends on worker units that are vulnerable to harassment, so the tactical battle requires a choice between putting everything into one large army and pushing, or splitting units into smaller groups and harassing in multiple places, or various mixes (small group to harass, bulk of army to defend, etc.) - If keyboard and mouse action rates are capped, then at every moment in time, the player must decide whether it is more profitable to devote actions to managing the army (micro) or managing the overall economy (macro). Choosing wrongly usually results in a loss - There is an implicit rock-paper-scissor tradeoff at the highest levels of the game: a 'greedy' strategy that cuts corners and favors economy over military will generally beat a 'safe' balanced strategy. Very aggressive strategies win against greed and generally lose against safe - There is the ability to scout your opponent to see whether they are going greedy, safe, or aggressive, but scouting requires an early investment in units and making subtle inferences about the opponent's build order, so the choice of whether to scout and how is not a trivial one - There can be bluffs where your opponent purposefully allows a scout of a key building, kills your scout, then cancels that building and chooses an entirely different technology instead

And all these layers interact:

- For example, if you go for an aggressive strategy, then you must commit blindly at the beginning of the game and often try to deny enemy attempts to scout you - If you scout that your opponent's army consists of units that are faster than yours, then they generally have much higher harassment potential, which pushes you towards a defensive posture. On the flip side, your opponent can use this threat to improve their economic position instead of attacking.

There is long-term planning at the strategic, informational, and also tactical levels. Effective high-level play requires an accurate model of what your opponent is doing in an environment where it's easy for your opponent to deny acquiring that information.

I'd wager that if you took two evenly matched professional level players, and then revealed the entire map to one player but not the other, you would go from a 50% to a 95%+ win rate.

Re: DeepMind and Blizzard Open StarCraft II as an AI Research Environment

#238
post #137

Earlier quoted context omitted.

I know this isn't the exact same as the article, but when genetic algorithms were introduced to solve for build orders, the "seven roach rush" was in vogue, something that was unexpected at the time and "discovered" using GA. I think there is a space for finding strategies that have more leeway in execution and thus are more suitable for humans to pilot rather than have machine level micro.

I love the story of the Seven Roach Rush. To quote the linked article, "The most interesting part of this build, however, is how counter-intuitive it is. It violates several well-known (and well-adhered-to) heuristics used by Starcraft players when creating builds." I'm fairly certain that this application of machine learning will present some surprising strategies. http://lbrandy.com/blog/2010/11/using-genetic-algor…

> I'm fairly certain that this application of machine learning will present some surprising strategies.

And so it begins...

Re: DeepMind and Blizzard Open StarCraft II as an AI Research Environment

#239

Earlier quoted context omitted.

What are their other options besides Starcraft2? This doesn’t seem like a PR stunt (not that the PR isn’t a bonus), but there’s already a history of AI competitions for Brood War, the game is more balanced than arguably any other RTS, and even though it is “primitive” as a strategy game in your estimation, AI isn’t ready to tackle a more advanced strategy game.

>What are their other options besides Starcraft2? Uh, real strategy games? Something like this: https://www.gog.com/game/m_a_x_m_a_x_2 Very simple ruleset, huge strategic depth.

Why do you not think StarCraft is a real strategy game?

Re: DeepMind and Blizzard Open StarCraft II as an AI Research Environment

#240

Earlier quoted context omitted.

It's not literally continuous, but it is real-time rather than turn-based, and positions of units are essentially floats rather than (a small range of) ints. That makes it effectively continuous (too large to just generate a tree of all possible actions and then prune).

Are you sure positions of units are essentially floats? Given how the units seem to arrange themselves (from what I see), I would guess that it's not close to the full range of floats, and instead there are just a few fractional pixel locations that units snap to. This is just a guess however. -- If this is the case though, the space could be represented by taking larger integer values (say, a magnitude of 1 or 2 hig…

It's most certainly integers on a very small grid.
Post reply on HN