I think I know what my final year project will be.
DeepMind and Blizzard Open StarCraft II as an AI Research Environment
241–250 of 283 posts
Re: DeepMind and Blizzard Open StarCraft II as an AI Research Environment
#242I'd be really interested in how differently tiered data sets (ladder rank) would work as sources for teaching.
Is it possible that training on diamond players is less effective than training on, say, silver? Is that actually even an interesting thing to look at?
Re: DeepMind and Blizzard Open StarCraft II as an AI Research Environment
#243Earlier quoted context omitted.
It's not literally continuous, but it is real-time rather than turn-based, and positions of units are essentially floats rather than (a small range of) ints. That makes it effectively continuous (too large to just generate a tree of all possible actions and then prune).
Are you sure positions of units are essentially floats? Given how the units seem to arrange themselves (from what I see), I would guess that it's not close to the full range of floats, and instead there are just a few fractional pixel locations that units snap to. This is just a guess however. -- If this is the case though, the space could be represented by taking larger integer values (say, a magnitude of 1 or 2 hig…
Even if you just bucketized things at the pixel level, that leaves you with a range in the thousands for each dimension.
Re: DeepMind and Blizzard Open StarCraft II as an AI Research Environment
#244Earlier quoted context omitted.
I think this definitely needs to be the case if this is going to be an interesting research project at all. Edit: Nevermind, my intuition in this seems to be wrong - someone more knowledgeable about this claims below that computer mechanics are still not as good as a good human's in the SC1 version of this.
In SC1 it's trivially easy to make a bot that spams so many actions it prevents your units from functioning properly. In fact it's easy to do accidentally. A lot of my early efforts into making a bot have been trying to find ways to reduce its APM without making it harder to code. "More actions = better" makes sense because we're used to human players who are using all their actions for something relatively effective…
So the challenge for the AI would be to figure out which action to do with the limited supply of actions and time.
Re: DeepMind and Blizzard Open StarCraft II as an AI Research Environment
#245Earlier quoted context omitted.
In what universe is taking 10,000 actions per minute an easier problem for a neural net than 100 such actions?
"In what universe is taking 10,000 actions per minute an easier problem for a neural net than 100 such actions?" StarCraft is precisely such universe. If you could micromanage units perfectly, you can do some amazing tricks. Here's an example of what I'm talking about: https://www.youtube.com/watch?v=IKVFZ28ybQs
I'm imagining an AI war where the next advancement is the micro of the siege engines to optimise targets and timing of shots to hit large groups after the initial splash avoidance. AI on both sides keep trying to maintain a one-step-ahead strategy which minimises/maximises casualties based on predicting the exact shooting/dodging strategy of the opponents. Will be interesting to follow developments in this area!
Re: DeepMind and Blizzard Open StarCraft II as an AI Research Environment
#246Earlier quoted context omitted.
i feel like you misunderstood that part of the argument. he is saying representing the state is very hard, and you are saying: given a well represented state, ML is very good at finding the important features, reducing the dementionality, and finding mathematical transformations, etc. deep learning has been so successful with images because representing them is trivial - flattened pixel vector. with your last paragra…
I was actually assuming the input representation would just be a video stream, which (combined with audio) is enough for human players, but looking more into it, it's a lot more than a video feed[1]. It feels a little like cheating, but I guess processing the game UI video feed isn't the interesting part of the problem. Plus, it makes the problem much more accessible to hobbyists who can't afford the GPU cluster requ…
Using just the video feed, the AI would be required to reconstruct an overview of the strategic situation, and then develop a forward strategy on top of that involving individual units. Even for a much simpler game like doom, video-only input is enough for strategies like "see an enemy, target and shoot it as fast as possible".
For an AI to be able to effectively compete in a complex game like SC2, preparing high-level inputs is important. Look at these like shortcuts, heuristic approximations of task that would be hard to represent and train with deep learning. I would guess an implementation would need multiple independent nets for various tasks, combined with heuristics. Then each could be separately trained to do the given task.
Re: DeepMind and Blizzard Open StarCraft II as an AI Research Environment
#247A lot of people here seem to be underestimating the difficulty of this problem. There are several incorrect comments saying that in SC1 AIs have already been able to beat professionals - right now they are nowhere near that level. Go is a discrete game where the game state is 100% known at all times. Starcraft is a continuous game and the game state is not 100% known at any given time. This alone makes it a much hard…
From what I saw in the API, the AI will potentially have some key advantages like more accurate micromanagement, and that can make a significant difference in a combat setting. They can try to compensate for this by throttling the number of actions per minute, but that won't compensate for extremely well-planned pixel-perfect clicks. This is a very powerful tactical advantage that can offset strategic deficiencies, if any.
Now, I would not compare SC1 bots to whatever DeepMind is going to create. SC1 bots were in their majority just rule-based bots with hand-coded strategies. DeepMind will create machine learning based bot, train it with data based on thousands if not millions of replays, and test it privately, maybe hiring a professional in the process (same they did with Fan Hui 5p), and make it play itself millions of times. It's a matter of time until they get it right and they get to pick when that time is. They will not organize a match until they feel their probability of winning is significant.
Re: DeepMind and Blizzard Open StarCraft II as an AI Research Environment
#248Earlier quoted context omitted.
I wonder if we will see any advanced cheese strats come out of this. I'm assuming some implementations will eventually develop micro control that is far beyond any human player's capabilities, which would make things like all-in probe rushing much more viable. Instead of playing the normal meta in a computer-vs-human, I imagine an advanced AI would simply send all of its workers off the mineral line as soon as the ga…
I know this isn't the exact same as the article, but when genetic algorithms were introduced to solve for build orders, the "seven roach rush" was in vogue, something that was unexpected at the time and "discovered" using GA. I think there is a space for finding strategies that have more leeway in execution and thus are more suitable for humans to pilot rather than have machine level micro.
Part of the problem SC2 units are not very balanced and each patch tries to make them "more balanced". SC2 design settled on making unique units at cost of balance.
Roaches in fact are quite overpowered, with very fast regen and quick ranged attack(they move as fast as hydralisks). A versatile and low-cost unit(cheaper than a Hydralisk). And the reason they're so powerful, SCII units of other races in general are more powerful than broodwar units and have less weaknesses. Broodwar instead has weak, easy dying units that force micro to extend their lifespan. SC2 units always have easy regen/heal/repair and the player just masses them in huge attack groups with minimal micro(their blocking boxes are tiny and pathing is good enough). The rock-paper-scissors from broodwar(which ephasized soft-counter) morphed into hard counters to everything, which lowered the strategic depth to "make whatever kill the majority of enemy unit type"(since its the best cost-effect decision at any point). SC2 "pro matches" are never decided in micro battles, they're most a competition on who can more effectively spend resources. SC2 micro is laughably unoriginal and tactically irrelevant(resource competition is far more important). ..And the reason SC2 can't have good micro in principle is not the 3D engine overhead, its server latency and lag. Perfect LAN games in broodwar with sub 10ms latency and millisecond reflexes can't exist within central servers hosting hundreds of players.
Re: DeepMind and Blizzard Open StarCraft II as an AI Research Environment
#249Earlier quoted context omitted.
For the same reason the APM are limited: to ensure that what we are doing is really focusing on advancing strategy rather than brute mechanical skill. If I played against an AI using nothing but the rendered frames and sound of a game as input, I might not even make the stipulation on reflexes. I'd be humbled if I lost. As it stands now, most of the games I like have bad AI. Sure, it can be fun to play a hack and sla…
This is not the same AI you normally face in a game. Most (all?) of those AI opponents use rules written by the game developers to make decisions and some of them simply cheat to be competitive ( cough Mario Kart 64 cough ). This blog is about creating AIs that interact with the game the same way humans do, the computer plays by the same rules and has no special access to the game state beyond what the player would h…
Okay, thanks. I appreciate the counterpoint. I guess I'd like to see it both ways: bots limited to human speed and bots not.
Re: DeepMind and Blizzard Open StarCraft II as an AI Research Environment
#250Any predictions for how long it will take for an AI to win against the world's best player?
Awhile. This just isn't like Go or Chess. The gap from perfect information to imperfect information is quite a chasm, and from turn-based to real-time is even more vast. I play Age of Empires 2 semi-competitively, and I just can't imagine the research progress that would have to be made for a pro to lose to an APM-limited AI agent. So much of the game comes down to intuiting what your opponent is planning without bei…
At pro level, much of the game is about what information you can gain, and about choosing what to show and, more importantly, what you don't show (hide) and acting on non-triggers.
An example of a non-trigger is knowing that if I haven't seen a certain unit at time X, I know I'm safe to do Y. It is acting upon the information that something didn't happen.
To expand: I saw my opponent starting two gases at my 21 supply scout. When I scouted again at 47 supply, I saw no gas heavy units, so I can deduce the gas was used for better technology. This will allow me the opportunity to increase my worker count by Z before building army, or I could try and kill my opponent right there for his technological greed.