Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

DeepMind and Blizzard Open StarCraft II as an AI Research Environment

deepmind.com

Post reply on HN
Print view View original

141–150 of 283 posts

Re: DeepMind and Blizzard Open StarCraft II as an AI Research Environment

#141

Post by placeybordeaux »

j9461701 wrote: post #89
captainmuon wrote: post #43

Earlier quoted context omitted.

> SC2 got rid of a lot of that You think so? My impression is that SC2 had a lot more of repetitive tasks you had to do. E.g. wall off the ramp, send a worker scouting, ... and you have to perform certain actions every X seconds (like using chrono boost). A lot or mastering the game is rote learning, and polishing a build order. Another big part is constantly scouting and reacting to what the enemy is doing. Due to t…

>You think so? Brood War is the most mechanically demanding game I have ever played, and certainly the most demanding that has ever been an esport. [This clip]( https://www.youtube.com/watch?v=UXH8eCcvQMI ) of Flash playing SK Terran style is illustrative of what I mean. Let's go down the list of things Flash needs to do: He needs to click on every one of his production structures every 15-19 seconds, and click M or…

> Starcraft 2 is a game I could get to masters league in, despite only playing make a few hours a week for a month or two.

Was this in beta? I played from beta to end of WoL and got masters in NA. It was not easy.

Re: DeepMind and Blizzard Open StarCraft II as an AI Research Environment

#142

Post by cjbprime »

bytefactory wrote: post #65
gwern wrote: post #55

Earlier quoted context omitted.

I think it is doable in under 5 years, but this critically depends on the resources invested by DM and other DL orgs. Deep RL is hugely demanding of computational resources to iterate your designs - for example, the first AlphaGo took something like 3 GPU-years to train it once (2 or 3 months parallelized); however, with much more iteration, DM was able to get Master's from-scratch training down to under 1 month. Now…

> (It might not even be as complex as people think ... Yeah, I suspect you're right. Eliezer was alluding to this with the AlphaGo victory as well: > ... Human neural intelligence is not that complicated and current algorithms are touching on keystone, foundational aspects of it. https://www.facebook.com/yudkowsky/posts/10153914357214228?p... I can't decide if I would be bummed or excited if that turns out to be the…

Yeah! I mentioned the same sentiment in this 2012 post when it was becoming clear that computers were reaching human strength at Go via brute force: http://blog.printf.net/articles/2012/02/23/computers-are-ver...

Re: DeepMind and Blizzard Open StarCraft II as an AI Research Environment

#143

Post by giblaz »

haeffin wrote: post #127
jncraton wrote: post #123

Earlier quoted context omitted.

You're basically correct in terms of SC not technically being continuous. There are discrete steps under the hood. One of the significant challenges is figuring out how to use 42ms (the frame duration on fastest speed) of computing time to decide what actions, if any, to take next. You don't have the luxury of taking many minutes to decide one move as you would in a game like chess or go. You also don't alternate tak…

SC2 ticks faster than SC1 - you only have 22ms. You don't need to tie everything to tick rate though, a strategy module could update way slower.

Bingo. If this had the tasks split up among multiple threads/processes correctly and using a very fast performing language + good developers, the tickrate is less important. Some army control module could manage the unit micro within the bound of a tick with other modules updating other info the system draws from to perform actions.

Re: DeepMind and Blizzard Open StarCraft II as an AI Research Environment

#144

Post by jtolmar »

strgrd wrote: post #103
qub1t wrote: post #71

A lot of people here seem to be underestimating the difficulty of this problem. There are several incorrect comments saying that in SC1 AIs have already been able to beat professionals - right now they are nowhere near that level. Go is a discrete game where the game state is 100% known at all times. Starcraft is a continuous game and the game state is not 100% known at any given time. This alone makes it a much hard…

I wonder if we will see any advanced cheese strats come out of this. I'm assuming some implementations will eventually develop micro control that is far beyond any human player's capabilities, which would make things like all-in probe rushing much more viable. Instead of playing the normal meta in a computer-vs-human, I imagine an advanced AI would simply send all of its workers off the mineral line as soon as the ga…

Currently bot micro can perfectly time hit and run "dance" maneuvers on all their attacking units independently (several top bots are Terran and do this with Vultures). But solving this in a way that takes terrain into account is much, much harder, and a skilled human could chase the whole army into a wall and kill it.

Similarly there are worker rush bots that do some impressive things against other bots, but positioning is hard and a skilled human can beat the bot by clumping its workers up in the right shape.

Re: DeepMind and Blizzard Open StarCraft II as an AI Research Environment

#145

Post by aerovistae »

Synaesthesia wrote: post #82
cjbprime wrote: post #69

Earlier quoted context omitted.

On the other hand, bots are starting to beat professionals at (thousands of hands of repeated) Poker, so I think we can't say that imperfect information is something that's especially intractable for maching learning algorithms. http://spectrum.ieee.org/automaton/robotics/artificial-intel...

Yeah but compare the search space of Poker vs Starcraft.

Exactly. And poker is, again, turn-based with a single move to be made. And it may not be "perfect information," but it again is a game of static board state where there is probably a statistically optimal move for a given state provided you have memory of the other players' previous moves so far in a round.

Re: DeepMind and Blizzard Open StarCraft II as an AI Research Environment

#146

Post by jjn2009 »

greedy_buffer wrote: post #112
qub1t wrote: post #71

A lot of people here seem to be underestimating the difficulty of this problem. There are several incorrect comments saying that in SC1 AIs have already been able to beat professionals - right now they are nowhere near that level. Go is a discrete game where the game state is 100% known at all times. Starcraft is a continuous game and the game state is not 100% known at any given time. This alone makes it a much hard…

> Starcraft is a continuous game and the game state is not 100% known at any given time. It seems to me that multiplayer games may feel continuous to a human player but are still designed around a series of discrete states called ticks where each tick is determined from the previous state plus inputs. Why is this distinction made in the context of how difficult it is to develop an AI?

Its computationally difficult to calculate every possible move a character might do in the occluded region. This requires a long attention span and running simulations of possible opponent actions based on previous 'frames'. Alpha go solved some of this by reducing the amount of space searching and instead search the likely set of possible choices by the opponent but each evaluation is for a single frame. If for some reason a piece on a go board could disappear and reappear then during that time it was gone its impact on decision making would be either nil or skew heavily towards nil compared to the rest of the opponents pieces depending on how many frames in the past are used.

Re: DeepMind and Blizzard Open StarCraft II as an AI Research Environment

#147

Post by mulmen »

ktwo wrote: post #96
JefeChulo wrote: post #30

"so agents must interact with the game within limits of human dexterity in terms of “Actions Per Minute”." I am really glad they are limiting APM because otherwise things just get stupid.

IMO there should also be a precision limit. The timing of actions should include human-typical jitter and the wrong action should sometimes be activated to simulate misclicks/fat-finger keypresses — e.g., messing up a control group by assinging a unit to the wrong number key. The bot must also not be able to act faster than human reaction times (~250ms), this could be enforced by adding a fixed delay to the observati…

Since existing bots are far from being competitive with human players why further handicap them in ways that deal with an entirely different domain?

Re: DeepMind and Blizzard Open StarCraft II as an AI Research Environment

#148

Post by SilasX »

ktwo wrote: post #96
JefeChulo wrote: post #30

"so agents must interact with the game within limits of human dexterity in terms of “Actions Per Minute”." I am really glad they are limiting APM because otherwise things just get stupid.

IMO there should also be a precision limit. The timing of actions should include human-typical jitter and the wrong action should sometimes be activated to simulate misclicks/fat-finger keypresses — e.g., messing up a control group by assinging a unit to the wrong number key. The bot must also not be able to act faster than human reaction times (~250ms), this could be enforced by adding a fixed delay to the observati…

Right -- likewise it would be nice to have it mimic human cognition limits, like time lag for loading a new info source into memory.

Re: DeepMind and Blizzard Open StarCraft II as an AI Research Environment

#149

Post by mulmen »

branja wrote: post #122
ktwo wrote: post #96

Earlier quoted context omitted.

IMO there should also be a precision limit. The timing of actions should include human-typical jitter and the wrong action should sometimes be activated to simulate misclicks/fat-finger keypresses — e.g., messing up a control group by assinging a unit to the wrong number key. The bot must also not be able to act faster than human reaction times (~250ms), this could be enforced by adding a fixed delay to the observati…

ideally they'd train it on real keypresses rather than actions

Why would that be ideal? Wouldn't that just make ML at the strategy layer harder without doing anything to make the discoveries more valuable?

Re: DeepMind and Blizzard Open StarCraft II as an AI Research Environment

#150

Post by Impossible »

qub1t wrote: post #71

A lot of people here seem to be underestimating the difficulty of this problem. There are several incorrect comments saying that in SC1 AIs have already been able to beat professionals - right now they are nowhere near that level. Go is a discrete game where the game state is 100% known at all times. Starcraft is a continuous game and the game state is not 100% known at any given time. This alone makes it a much hard…

Minor nitpick, video games running on digital computers are by definition still discrete even if they feel continuous. Networked multiplayer wouldn't be possible in RTS games if that wasn't the case. The granularity of unit positions and turns in Starcraft obviously leads to a much larger state space, so I get what you're saying, for AI its effectively continuous.
Post reply on HN