Live data from Hacker News

DeepMind and Blizzard Open StarCraft II as an AI Research Environment

deepmind.com

131–140 of 283 posts

Re: DeepMind and Blizzard Open StarCraft II as an AI Research Environment

#131
post #71

A lot of people here seem to be underestimating the difficulty of this problem. There are several incorrect comments saying that in SC1 AIs have already been able to beat professionals - right now they are nowhere near that level. Go is a discrete game where the game state is 100% known at all times. Starcraft is a continuous game and the game state is not 100% known at any given time. This alone makes it a much hard…

> Starcraft is a continuous game and the game state is not 100% known at any given time. It seems to me that multiplayer games may feel continuous to a human player but are still designed around a series of discrete states called ticks where each tick is determined from the previous state plus inputs. Why is this distinction made in the context of how difficult it is to develop an AI?

Technically you're right, but there's a real qualitative difference. Each "tick" in a game like StarCraft is on the order of tens of milliseconds. When you send out an army to attack your opponent, it's quite possible that the actual confrontation won't happen until 10,000 ticks in the future.

Also, the dimensionality of the state space in a "continuous" game is orders of magnitude larger. In a game like chess or Go, you may have dozens or hundreds of moves available at each turn, but only a few of them will be "locally optimal". In StarCraft, there are many more degrees of freedom -- attack timing, positioning, formation, banking versus spending resources, and so on. A good AI will need to be able to abstract that huge state space down to something more tractable.

Re: DeepMind and Blizzard Open StarCraft II as an AI Research Environment

#132

> even strong baseline agents, such as A3C, cannot win a single game against even the easiest built-in AI. Then, why not release code for the built in ai, and improve on it ? Or is the built in ai cheating ?

The goal is to "grow" an AI through reinforcement learning and other techniques that are broadly applicable outside starcraft. The existing rules based systems in the game A) are extremely Starcraft specific and the methods have comparatively limited utility outside video game AI, B) unlikely to be scalable to fairly beat a professional human (which introduces too much variability to capture reliably into rules)

Re: DeepMind and Blizzard Open StarCraft II as an AI Research Environment

#134
post #126
post #40

Earlier quoted context omitted.

Until today's release of the headless Linux client, you still had to run the full StarCraft program, which gets expensive fast. And it massively complicates the workflow to have to play through every game serially to recreate the state rather than simply reading random rows of data from a 300GB dataframe on disk.

FYI, there are two things being discussed here. There dataset linked in the comment above is for Brood War. The headless client released today is for SC2.

I am aware of that. The point remains the same: both Brood War and SC2 are expensive to run, so you really don't want to and it's worth spending disk space to cache the results of playing out a replay files. This will probably also be true of the replay files DM/Blizzard will be releasing even with the lite client.

Re: DeepMind and Blizzard Open StarCraft II as an AI Research Environment

#135
post #91

Earlier quoted context omitted.

Maybe what usaphp is getting at is that the AI could still gain an advantage by doing a set of actions much faster than humanly possible in just a fraction of a second as long as it kept its total number of actions that minute below the cap.

Exactly. Most actions in idle game mode (i.e. when there's no active battle, micro or macro to be done) are null, the gamers simply repeat random meaningless keystrokes; this helps with being always alert and ready for anything that requires a rapid response. An AI doesn't need to do this. Source: Ex pro Starcraft (Brood War) gamer.

There no were BW pro-gamers outside of Korea.

Re: DeepMind and Blizzard Open StarCraft II as an AI Research Environment

#136
post #115

Earlier quoted context omitted.

For SC1 BW, you can already watch https://www.youtube.com/user/certicky for a weekly highlights broadcast / commentary of AIs. Right now there is an AI arms race where the previously dominant AI (Iron Bot) is being beaten by challengers. It used to happen when Iron Bot did something silly. Now it is happening because the competitors have really stepped up their game. The current board leader (krasi0) has a strategy s…

This surprises me - I would have expected the computer mechanics to be much better than what a human can achieve. Do you have insight into why that isn't true?

AIs can click faster, but humans still have better overall strategy and planning, which matters a lot in this game.

Re: DeepMind and Blizzard Open StarCraft II as an AI Research Environment

#137
post #103

Earlier quoted context omitted.

I wonder if we will see any advanced cheese strats come out of this. I'm assuming some implementations will eventually develop micro control that is far beyond any human player's capabilities, which would make things like all-in probe rushing much more viable. Instead of playing the normal meta in a computer-vs-human, I imagine an advanced AI would simply send all of its workers off the mineral line as soon as the ga…

I know this isn't the exact same as the article, but when genetic algorithms were introduced to solve for build orders, the "seven roach rush" was in vogue, something that was unexpected at the time and "discovered" using GA. I think there is a space for finding strategies that have more leeway in execution and thus are more suitable for humans to pilot rather than have machine level micro.

I love the story of the Seven Roach Rush. To quote the linked article, "The most interesting part of this build, however, is how counter-intuitive it is. It violates several well-known (and well-adhered-to) heuristics used by Starcraft players when creating builds."

I'm fairly certain that this application of machine learning will present some surprising strategies.

http://lbrandy.com/blog/2010/11/using-genetic-algorithms-to-...

Re: DeepMind and Blizzard Open StarCraft II as an AI Research Environment

#138

It's a bit too bad they're having to move towards supervised learning and imitation learning. I totally understand why they need to do that given the insane decision trees, but I was really hoping to see what the AI would learn to do without any human example, simply because it would be inhuman and interesting. I'm really interested in particular if an unsupervised AI would use very strange building placements and pe…

They can always finetune using RL later. Superversied training was the first step at making AlphaGo work.

Re: DeepMind and Blizzard Open StarCraft II as an AI Research Environment

#139
post #115

Earlier quoted context omitted.

For SC1 BW, you can already watch https://www.youtube.com/user/certicky for a weekly highlights broadcast / commentary of AIs. Right now there is an AI arms race where the previously dominant AI (Iron Bot) is being beaten by challengers. It used to happen when Iron Bot did something silly. Now it is happening because the competitors have really stepped up their game. The current board leader (krasi0) has a strategy s…

This surprises me - I would have expected the computer mechanics to be much better than what a human can achieve. Do you have insight into why that isn't true?

I'm trying to come up with a simple and satisfying explanation, but the best I can do is "Starcraft is a complex game of balancing your ability to attack and defend. AIs have difficulty with situational awareness."

But I think that case analysis of what I've observed may be more telling.

Case 1, The AIs tend to over-react, or under-react. For example, when a Zerg player sees flying units they may start to go scourge heavy. If a human player notices this, they may build a Wraith or two, causing the Zerg player to waste a lot of money. This can happen naturally between AIs as well. Terrans depleting the Command Center's invisibility sweep too soon is another one... Something is getting hurt, but you may want to wait until after the Tanks are in siege mode before attacking (not while they are converting).

Case 2, Lack of Memory. This one happens a lot. You'll see an AI do something bad. Then, in a situation where circumstances obviously wouldn't have improved, try again a few seconds later. I'd want to blame the fog of war, but futile attacks on static towers is a common example.

Case 3, Fight or Flight. Sometimes you'll see units fleeing from a battle they cannot win. Sounds good. But sometimes they are being pursued by units that can pick them off during retreat. And they aren't fleeing for more support, they are just avoiding a bad situation. When in reality, those units are dead no matter what, might as well try to take down an enemy unit or two. The inverse can happen too, where units stand and fight in a situation they could run away from and get reinforcements. I've heard predicting combat outcomes is really difficult in SC.

Case 4, Under utilization. Vultures are the unit that stands out in my mind as one that an AI can handle better than a human. They are fast, have a great punch, and can deploy mines really effectively (some bots mine much of the map). What was a harassment unit becomes an offensive unit that can hold its own in a "fire/flee/repeat" pattern (imagine having 5 of them do that in the same area). High Templar are the opposite. I tend to expect AIs to do poorly when storming, it is currently a highlight/joy to see them utilize the ability effectively. But the ability is meant to discourage enemies from grouping units too closely together. Which allows for Carriers and Mutas to do a lot more coupling / damage than they would be able to otherwise do. Under current circumstances, it would be possible to see a dozen or more Mutas being crippled in a single storm. I rarely see nukes/Ghosts or Defilers used. Queens also seem to be under utilized.

Case 5, Target Prioritization. When you see a Carrier, kill it rather than the Interceptors. When you see a Medic or two, kill them before the Marines. Same with SCVs repairing in some cases (with sufficient firepower, the unit will be dead faster than it can be healed). One AI loves Carriers, and part of the reason their strategy works is that a lot of units go after Interceptors rather than Carriers, allowing the Carriers to retreat (the AI judges when to do this well) and rebuild.

It isn't that AIs can't play at a professional level, but this represents the current level of the bots. Looking over it... The AIs can pull off things we can't, but professional level situational awareness / judgement is tough.

Re: DeepMind and Blizzard Open StarCraft II as an AI Research Environment

#140

Earlier quoted context omitted.

Previous AI StarCraft tournaments have put an actions-per-minute (APM) cap on the AIs, which prevents them from micro-managing individual units.

I think this definitely needs to be the case if this is going to be an interesting research project at all. Edit: Nevermind, my intuition in this seems to be wrong - someone more knowledgeable about this claims below that computer mechanics are still not as good as a good human's in the SC1 version of this.

In SC1 it's trivially easy to make a bot that spams so many actions it prevents your units from functioning properly. In fact it's easy to do accidentally. A lot of my early efforts into making a bot have been trying to find ways to reduce its APM without making it harder to code.

"More actions = better" makes sense because we're used to human players who are using all their actions for something relatively effective and because (I'll assert) they're well below the optimal APM. But the optimal APM is probably something like 1000, not the 10k a bot can easily reach.

Post reply on HN