DeepMind and Blizzard to release StarCraft II as an AI research environment • Hacker News Board

Re: DeepMind and Blizzard to release StarCraft II as an AI research environment

#331

Post by zacharycohn » Sun, Nov 06, 2016, 12:54 AM UTC

burke wrote: post #132

theptip wrote: post #122

This is pretty interesting. DeepMind’s last triumph (beating the best human Go players with AlphaGo) is impressive, but Go is a great fit for neural networks as they stand today; it’s stateless, so you can fully evaluate your position based on the state of the board at a given turn. That’s not a good fit for most real-world problems, where you have to remember something that happened in the past. E.g. the ‘fog of war…

> When Google’s AI can beat a human at StarCraft, it’s time to be very afraid. I don't actually agree with this. Unlike Go, StarCraft is not only a game of strategy; "micro" (micro-managed tactics, basically) also plays a big role. An AI is going to be able to issue a LOT more commands per second than even the most skilled humans, giving them a natural tactical advantage. Strategy is more difficult for an AI, but not…

Micro is important, but good macro way more so.

Re: DeepMind and Blizzard to release StarCraft II as an AI research environment

#332

Post by taneq » Sun, Nov 06, 2016, 11:34 AM UTC

sillysaurus3 wrote: post #119

cosarara97 wrote: post #111

Earlier quoted context omitted.

What about APM? Doesn't the speed at which the AI can input commands give it a big advantage?

It's like saying that a Go AI would have a huge advantage because it can move instantly, giving the opponent no time to think. Perhaps it contributes, but strategy is far more important. See "The Marginal Advantage," an essay from the famous Starcraft player Day9: http://www.teamliquid.net/forum/brood-war/64514-competitive-...

This essay reminds me very much of the "Playing to Win" series by David Sirlin: http://www.sirlin.net/ptw/

It's also worth noting that AlphaGo optimizes its perceived chance of winning the game, rather than its score. It "gets complacent" and mostly plays very conservatively the moment it's ahead on points, because it doesn't care about margins, only about win/loss.

Re: DeepMind and Blizzard to release StarCraft II as an AI research environment

#333

Post by skybrian » Sun, Nov 06, 2016, 4:34 PM UTC

seanwilson wrote: post #267

skybrian wrote: post #254

Earlier quoted context omitted.

Deceit boils down to assuming your opponent will misjudge the situation and make a mistake. In Chess or Go, computer players choose the best move assuming that the opponent won't make any mistakes (will never be fooled). There's no attempt to model a weaker opponent. This is why computer players in Go tend to fall apart when they start losing. They don't see a path to victory against a perfect opponent, so all moves…

> In Chess or Go, computer players choose the best move assuming that the opponent won't make any mistakes (will never be fooled). There's no attempt to model a weaker opponent. What I meant was there's no reason an AI couldn't try to fool a human in a perfect information context. For example, you could bait a (probably not world class though) opponent into capturing a piece which puts them in a bad position, where c…

It's more of a testing limitation. These programs are only occasionally tested against humans. You need expert players to test, they're not available all the time, and it takes a long time to play a single game (unlike poker). You can use databases of online games but it won't tell you how the player would have responded to a different move 37.

If you're writing a program to beat anyone, there isn't much point in optimizing the algorithm for a specific opponent's weaknesses when there are improvements to make that would work against anyone and the improvement in strength can be evaluated using computer testing alone.

Work on exploiting specific opponents' weaknesses is typically done using much simpler games (check out what's been done for rock paper scissors).

Re: DeepMind and Blizzard to release StarCraft II as an AI research environment

#334

Post by HeavyStorm » Sun, Nov 06, 2016, 5:18 PM UTC

rfrank wrote: post #209

theptip wrote: post #198

Earlier quoted context omitted.

Agreed that micro is a big advantage that AIs have held in previous AI vs. Human StarCraft contests, but the OP stated that the SC2 research environment will limit the AI to (elite) human APM levels, so that edge will be somewhat blunted. It's probably fair to question whether all actions are counted equally; I could believe that an AI could allocate their action budget more widely than a human can, even while hittin…

When I was playing SC2 frequently a few years ago, someone made a program that used genetic algorithms to construct build orders [1]. It would produce a build that technically worked, but in a lot of cases they weren't very viable. I tried to make a blink stalker rush that'd be the rough equivalent of the build shown in the post liked above, just with the Protoss race in the game. The program would tell me to do thin…

Hey, thanks, I was looking for that this week!

Re: DeepMind and Blizzard to release StarCraft II as an AI research environment

#335

Post by theptip » Sun, Nov 06, 2016, 7:39 PM UTC

shannonpig wrote: post #285

theptip wrote: post #122

This is pretty interesting. DeepMind’s last triumph (beating the best human Go players with AlphaGo) is impressive, but Go is a great fit for neural networks as they stand today; it’s stateless, so you can fully evaluate your position based on the state of the board at a given turn. That’s not a good fit for most real-world problems, where you have to remember something that happened in the past. E.g. the ‘fog of war…

I think you're thinking about the difference between full information games vs partial information games, rather than stateless games (the state of a starcraft game is stored on a central server, the computer just has no access to it). There's already been excellent progress in poker AI based on traditional game theory, see http://poker.cs.ualberta.ca/ .

You're right, that's the technical term for it, thanks. I was referring to the agent's evaluation function being stateless (i.e. it does not _need_ to persist state from previous turns in order to play optimally).

I think that there are two distinct aspects of the incomplete information that are significant; first, ignoring temporality, you have the fog of war, so you can't see the whole board. This is probably easier to address in a RNN, since you can play quite well as an amnesiac that just reacts to things that are currently visible. But you need to scout less if you have a memory of what's out there, so the amnesiac won't be able to play optimally.

Then there's the temporal aspect. The set of previous states of the game is not stored in the game and made available to the player, and so to play optimally you have to have a memory. This is where new techniques will be necessary.

These are separate problems, I think, so it will be interesting to see if DeepMind can make progress without reaching human performance on the second part.

Re: DeepMind and Blizzard to release StarCraft II as an AI research environment

#336

Post by Houshalter » Sun, Nov 06, 2016, 9:07 PM UTC

ygra wrote: post #309

Houshalter wrote: post #227

Earlier quoted context omitted.

One game experimented with smart enemy units that could sneak up on a player and attack them from behind, or outflank them. The players hated it and assumed the computer was cheating and just spawning enemies behind them. The same is sort of true in RTS games. The new rerelease of age of empires 2 has vastly improved AI that was developed by modders. Lots of players thought it was just cheating. But in general it has…

For the vast majority of players it's probably irrelevant whether the AI cheats or not. The goal is not to play fair and beat the human usually, but to put up a good fight and make things challenging and, most importantly, fun for the human. Of course, competitive and professional players have different expectations or requirements in that regard. Still, AI in games is mostly a trade-off. It usually can't take too mu…

I think the majority of players would prefer AIs that are challenging through better strategy and tactics, rather than just being bullet sponges or having better aim. People also get frustrated when the computer wins through cheating, rather than actually being better.

Re: DeepMind and Blizzard to release StarCraft II as an AI research environment

#337

Post by TulliusCicero » Sun, Nov 06, 2016, 9:32 PM UTC

nogbit wrote: post #183

TulliusCicero wrote: post #77

Earlier quoted context omitted.

In practice right now the AI still loses even with unlimited APM. It's just too dumb to really take advantage of its execution speed.

Buts it's intentionally dumbed down to keep the game fun, loosing all the time would get frustrating real quick. Theres no ROI for AI devs to kick the humans ass (except for maybe now with our AI/Cloud/VR/ craze).

It is not intentionally dumbed down. Current AIs are just too stupid at strategy and tactics to beat competent human players.

Re: DeepMind and Blizzard to release StarCraft II as an AI research environment

#338

Post by tempestn » Mon, Nov 07, 2016, 2:27 AM UTC

halflings wrote: post #305

tempestn wrote: post #297

Earlier quoted context omitted.

I agree with you. As I said, > Your parent post appears to suggest mitigating these other advantages in various ways (although I don't personally agree with that approach, since the computer isn't doing something that's out of reach for humans, just things that aren't practical for humans to achieve consistently). As far as your last paragraph, I suspect the issue is that people building AIs generally try to follow t…

I don't think it's as easy as that. Even with perfect macro + micro, your opponent could simply scout you, see what build you're going for, and counter your build. Pro players do have (quasi-)perfect macro: They build units and buildings exactly when they should be built (especially at the early stages of the game). They also have an excellent micro. What makes one player win against the other is often the choices th…

The problem with that is, most counters to first tier units rely on splash damage. As zerg for instance, if your opponent is rushing with lings or marines, you need banelings. But it would be trivial for an ai to neutralize the splash damage, so that wouldn't work.

Generally speaking, the counters are balanced for human players. Biased toward high level players, but still human.

Re: DeepMind and Blizzard to release StarCraft II as an AI research environment

#339

Post by spdionis » Tue, Nov 08, 2016, 2:53 PM UTC

otempomores wrote: post #296

pavel_lishin wrote: post #221

Earlier quoted context omitted.

A nice illustration of the potential battlefield of tomorrow.

Privat third in first row you ve been ai selected to draw splash damage for king and cuntry. Everybody else give him some privacy

"cuntry"... the irony...

Re: DeepMind and Blizzard to release StarCraft II as an AI research environment

#340

Post by pdimitar » Wed, Nov 09, 2016, 3:40 PM UTC

blazespin wrote: post #275

burke wrote: post #132

Earlier quoted context omitted.

> When Google’s AI can beat a human at StarCraft, it’s time to be very afraid. I don't actually agree with this. Unlike Go, StarCraft is not only a game of strategy; "micro" (micro-managed tactics, basically) also plays a big role. An AI is going to be able to issue a LOT more commands per second than even the most skilled humans, giving them a natural tactical advantage. Strategy is more difficult for an AI, but not…

Is it just me, or is anyone else a little disturbed that some of the greatest minds in AI are optimizing it for winning at a highly strategical/tactical game of warfare..

As a StarCraft 2 amateur, it is my opinion that it has a lot more to do with being able to manage a lot of actions at the same time -- production, defense, expansion, scouting and attacking your opponent. SC2 is definitely not only about war. The pro players are periodically uploading videos explaining SC2 economy and the huge importance of proper timing and type of production in order to achieve maximum growth -- and consequently, aggression towards the opponent.

Don't get me wrong; I'd prefer DeepMind's efforts be focused on a huge and realistic economics simulator but there are simply no such games (not ones that are good and/or realistic anyway).

IMO StarCraft 2 is the best economy and war simulator these days so that's why DeepMind chose it.