Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

DeepMind and Blizzard Open StarCraft II as an AI Research Environment

deepmind.com

Post reply on HN
Print view View original

191–200 of 283 posts

Re: DeepMind and Blizzard Open StarCraft II as an AI Research Environment

#191

Post by avar »

yannyu wrote: post #176
mulmen wrote: post #151

Earlier quoted context omitted.

usaphp's point seems to be that the period should be less than one minute. When the police measure your speed with radar they are not waiting an hour to average your speed. The same idea applies here.

Yep, I entirely understand the point. Consider that when the police measure your speed, they're still measuring in MPH. Just because the units contains a specific time-frame (hours), doesn't mean the measurement is made over an hour. Similarly, just because the units for APM contains a specific time-frame (minutes) doesn't mean the measurement must be made over a minute. A 150APM limit doesn't necessarily mean that t…

You say you understand the point but you seem to have entirely missed it, the same problem remains.

The point of the APM limit is presumably to fairly emulate a human player, but the APMs for humans are obviously averaged over a minute.

However, for certain things humans can initiate actions in SC2 in quick succession that, if sustained over a greater time period, would yield a ridiculous APM rate. Think a Terran player highlighting the barracks and tapping "A" really fast, say with a 10ms delay, that would result in 600 APM.

Let's say a human can build 5 marines like that in quick succession. You're going to have to allow for temporary spikes in APM to not unfairly give the human player an advantage.

But if you do that the computer is able to really rapidly execute more complex actions that the human can't because he's limited by the SC2 UI, whereas every action via the API is equally difficult for the computer. E.g. moving 3 different subgroups of marines out of the way of a High Templar Storm. I doubt any human player could select 3 different subgroups of marines from one big group and move them out of the way in 3 different directions within the span of 60ms (10ms for each action of select/move for 3x groups).

So any "fair" APM limit really needs to be dynamic in some way, or at least take into account the complexity of actions (say highlighting a group v.s. tapping "A"). It's not at all obvious how to do this while retaining fairness and not giving either the human or the computer an unfair advantage.

Re: DeepMind and Blizzard Open StarCraft II as an AI Research Environment

#192

Post by chii »

hacker_9 wrote: post #121

There's something funny about a company that is actively developing bleeding edge AI technology, but who can't design a webpage that works on mobile without crashing.

Just goes to show how complicated web tech is, even ai researchers can't get it right!

Re: DeepMind and Blizzard Open StarCraft II as an AI Research Environment

#193

Post by branja »

mulmen wrote: post #186
branja wrote: post #170

Earlier quoted context omitted.

to emulate human handicaps at the interface layer. I didn't say it would be free

But why is that desirable? Why would we want to emulate the human physical handicaps in our quest to advance AI at a strategy level?

For the same reason the APM are limited: to ensure that what we are doing is really focusing on advancing strategy rather than brute mechanical skill. If I played against an AI using nothing but the rendered frames and sound of a game as input, I might not even make the stipulation on reflexes. I'd be humbled if I lost.

As it stands now, most of the games I like have bad AI. Sure, it can be fun to play a hack and slash against lots of little, dumb minions, but FPS, RTS AI these days still don't cut it as savvy opponents. Often they have inhuman perception, direct knowledge of game state, or higher starting resources, but they make abysmal decisions.

Yes, I realize these are unlikely, expensive goals and incremental progress is how things are done. I just want to know if it's possible or desirable to emulate actual human reaction time.

Do you disagree this would in principle help separate strategy from godlike reflexes?

Re: DeepMind and Blizzard Open StarCraft II as an AI Research Environment

#194

Post by gambler »

teraflop wrote: post #131
greedy_buffer wrote: post #112

Earlier quoted context omitted.

> Starcraft is a continuous game and the game state is not 100% known at any given time. It seems to me that multiplayer games may feel continuous to a human player but are still designed around a series of discrete states called ticks where each tick is determined from the previous state plus inputs. Why is this distinction made in the context of how difficult it is to develop an AI?

Technically you're right, but there's a real qualitative difference. Each "tick" in a game like StarCraft is on the order of tens of milliseconds. When you send out an army to attack your opponent, it's quite possible that the actual confrontation won't happen until 10,000 ticks in the future. Also, the dimensionality of the state space in a "continuous" game is orders of magnitude larger. In a game like chess or Go,…

To the best of my knowledge, the only thing in SC2 that requires pixel-level precision is selecting units. Everything else can just as easily be represented as a fairly coarse grid with no loss of expressiveness. Buildings are explicitly snapped to a grid, and moving your units several pixels to either side simply doesn't matter. So calling SC2 "continuous" in terms of space is misleading.

I don't think there is anything that requires super-fast response times either, so you could conceivably get ~1 frame per second and not lose much information.

Well, IIRC, there are some visual indicators that rely on blinking, but I don't think they are crucial.

Re: DeepMind and Blizzard Open StarCraft II as an AI Research Environment

#196

Post by oh_sigh »

strgrd wrote: post #103
qub1t wrote: post #71

A lot of people here seem to be underestimating the difficulty of this problem. There are several incorrect comments saying that in SC1 AIs have already been able to beat professionals - right now they are nowhere near that level. Go is a discrete game where the game state is 100% known at all times. Starcraft is a continuous game and the game state is not 100% known at any given time. This alone makes it a much hard…

I wonder if we will see any advanced cheese strats come out of this. I'm assuming some implementations will eventually develop micro control that is far beyond any human player's capabilities, which would make things like all-in probe rushing much more viable. Instead of playing the normal meta in a computer-vs-human, I imagine an advanced AI would simply send all of its workers off the mineral line as soon as the ga…

A sensible thing for human-AI matches is to enforce a maximum number of actions per second and/or actions over period of time, which would be in line with a standard human player.

Re: DeepMind and Blizzard Open StarCraft II as an AI Research Environment

#197

Post by uw_rob »

pavedwalden wrote: post #88
yannyu wrote: post #85

Earlier quoted context omitted.

Why? Speed is in MPH or KPH, would it be better to go by m/s? The convention in gaming is APM, so they're just using the nomenclature that is already understood.

Maybe what usaphp is getting at is that the AI could still gain an advantage by doing a set of actions much faster than humanly possible in just a fraction of a second as long as it kept its total number of actions that minute below the cap.

APM isn't averaged over a minute time frame. I don't know the exact calculations they use ( and it has been changed a couple of times) but roughly, if you go above 5 actions per second your APM will be shown as 300.

I believe the reason they use APM over APS is because 3.4 APS isn't as meaningful as 204 APM.

Re: DeepMind and Blizzard Open StarCraft II as an AI Research Environment

#198

Post by gradys »

qub1t wrote: post #71

A lot of people here seem to be underestimating the difficulty of this problem. There are several incorrect comments saying that in SC1 AIs have already been able to beat professionals - right now they are nowhere near that level. Go is a discrete game where the game state is 100% known at all times. Starcraft is a continuous game and the game state is not 100% known at any given time. This alone makes it a much hard…

I think a big component is not really machine learning but more related to how to represent state at any given time, which will necessarily involve a lot of human-tweaking of distilling down what really are the important things that influence winning.

I agreed with everything you said until here. Developing good representations of state is precisely what today's machine learning is so good at. This is the key contribution of deep learning.

You seem to be supposing that a human expert is going to be carefully designing a set of variables to track, and in doing so conveying what features of the input to pay attention to and what can be ignored. Presumably the ML can then handle figuring out the optimal action to take in response to those variables.

I think it's much more likely to be the other way around. ML is really good at taking high dimensional input with lots of noise and figuring out to map that to meaningful (to it, if not to us) high-level variables. In other words, modern AI is good at perception.

What it's significantly less good at compared to humans is what might formally be called the policy problem. Given high level variables that describe the situation, what's the best course of action? This involves planning. We think of it in terms of breaking the problem into sub-objectives, considering possible courses of action, decomposing a high level plan into a sequence of directly executable actions, etc. AIs might "think" of this problem in different terms than these, but it seems like it still has to do this kind of work if it is going to have a chance to succeed.

We don't have obvious ways to model this part of the problem. For the perception/representation building problem, I can almost guarantee the solution is going to be a ConvNet to process individual frames combined with a recurrent layer to track state over time. On the other hand, I'm seeing some plausible solutions to the policy problem emerging in the literature, but it's still very much an open question what will emerge as the go-to. In AlphaGo, this part of the problem is where they brought in non-ML algorithmic solutions like Monte Carlo tree search, and one of the reasons StarCraft is interesting compared to Go is that those algorithmic solutions are harder to apply.

Re: DeepMind and Blizzard Open StarCraft II as an AI Research Environment

#199

Post by jtolmar »

yulaow wrote: post #180
jtolmar wrote: post #173

Earlier quoted context omitted.

The ones built into the game can see past fog of war. Blizzard published a nice overview of how they work, though I don't think it actually mentions that cheat. http://classic.battle.net/scc/faq/aiscripts.shtml The ones created using BWAPI cannot (you can call a function that lets you, but it's banned in all tournaments). The one thing that BWAPI bots know that a human doesn't is a persistent unit ID - if a marine le…

Are you sure? I remember the video about the whole blizzard story and the Ai creator of starcraft2 said that the new AIs, compared to those of starcraft, have only the same information a player has and no way to "cheat"

I thought the question was a bout the original game?

I don't know much about the SC2 AIs. You could play some games and watch the replays to tell if consistently looks like it's responding to things it shouldn't know. I remember it being pretty blatant in BW.

Re: DeepMind and Blizzard Open StarCraft II as an AI Research Environment

#200

Post by uw_rob »

avar wrote: post #191
yannyu wrote: post #176

Earlier quoted context omitted.

Yep, I entirely understand the point. Consider that when the police measure your speed, they're still measuring in MPH. Just because the units contains a specific time-frame (hours), doesn't mean the measurement is made over an hour. Similarly, just because the units for APM contains a specific time-frame (minutes) doesn't mean the measurement must be made over a minute. A 150APM limit doesn't necessarily mean that t…

You say you understand the point but you seem to have entirely missed it, the same problem remains. The point of the APM limit is presumably to fairly emulate a human player, but the APMs for humans are obviously averaged over a minute. However, for certain things humans can initiate actions in SC2 in quick succession that, if sustained over a greater time period, would yield a ridiculous APM rate. Think a Terran pla…

> the APMs for humans are obviously averaged over a minute

This is wrong. I don't know the exact number, but APM is averaged over around a second. I suspect this is done because APM is a more meaningful compared to APS, for humans at least.

Here is a graph from Scelight that high lights this: https://goo.gl/photos/9cjNxDwWoB1pmWkg9

Post reply on HN