Deep Reinforcement Learning to Play StarCraft • Hacker News Board

Re: Deep Reinforcement Learning to Play StarCraft

#11

Post by danielvf » Wed, Sep 14, 2016, 4:56 PM UTC

pinouchon wrote: post #8

I think Starcraft is a very interesting challenge for AI because it involves planning in an environment that is only partially observable: you must scout in order to see what your opponent is up to, and even then, you don't see everything. If DeepMind works on this, I really hope that they constrain the AI (APM-wise) so that its only chance of winning is by good planning and strategy, not super-fast micro.

According to this paper Facebook is only working on micro - attempting to win a few simple (one to two unit types) battles that humans can win 100% of the time against AI.

As you said, the glory of StarCraft is it's strategic level information game. Will be interesting to see what comes out of attempting to learn that.

Re: Deep Reinforcement Learning to Play StarCraft

#12

Post by vinchuco » Wed, Sep 14, 2016, 4:58 PM UTC

pkfrank wrote: post #9

danielvf wrote: post #7

Earlier quoted context omitted.

There are already incredibly good micro StarCraft AI, able to operate at thousands of APM - this isn't the limiting aspect of current StarCraft AI. The "Hard" part of StarCraft is that it is a huge Rock Paper Scissors game with only the information you fight for. You have to be able to piece together a picture of your opponents actions and forces from small cues.

Wow, I hadn't seen this before. Here is "Automaton 2000" controlling 20 marines vs 40 banelings, without losing a single unit. https://youtu.be/DXUOWXidcY0?t=52 Pretty cool.

That's incredible. The speed advantage here: https://youtu.be/IKVFZ28ybQs Brings to mind the power of high frequency trading

Re: Deep Reinforcement Learning to Play StarCraft

#13

Post by h4nkoslo » Wed, Sep 14, 2016, 5:00 PM UTC

It's a misconception that StarCraft is a strategy game. If you look at how it's actually played by human pros, it looks closer to a fighting game; very reflex-driven & heavy on micro-interactions. You would expect an un-gated AI with effectively infinite actions per second to do very well.

Re: Deep Reinforcement Learning to Play StarCraft

#14

Post by danielvf » Wed, Sep 14, 2016, 5:00 PM UTC

pkfrank wrote: post #9

danielvf wrote: post #7

Earlier quoted context omitted.

There are already incredibly good micro StarCraft AI, able to operate at thousands of APM - this isn't the limiting aspect of current StarCraft AI. The "Hard" part of StarCraft is that it is a huge Rock Paper Scissors game with only the information you fight for. You have to be able to piece together a picture of your opponents actions and forces from small cues.

Wow, I hadn't seen this before. Here is "Automaton 2000" controlling 20 marines vs 40 banelings, without losing a single unit. https://youtu.be/DXUOWXidcY0?t=52 Pretty cool.

And as cool as that is, this is even more terrifying, as a hundred zergslings dodge seige tank cannons and destroy them.

https://youtu.be/IKVFZ28ybQs

It's enough to make you scared for the future of humanity.

Re: Deep Reinforcement Learning to Play StarCraft

#15

Post by whorleater » Wed, Sep 14, 2016, 5:00 PM UTC

anirul wrote: post #4

After loosing on the Go territory seams like fb is trying to challenge Alphabet on StarCraft. DeepMind already declared they will go for StartCraft as a next challenge does it mean that they accept the challenge? I'm actually happy to see what could be the result! The only weak point for the StarCraft community is that it would be on SC1 and not SC2.

SC1 is not a weak point, it's generally considered to be better balanced, and is the "gold standard" for RTS's. SC1 also has far more training dataset than SC2.

Re: Deep Reinforcement Learning to Play StarCraft

#16

Post by zitterbewegung » Wed, Sep 14, 2016, 5:03 PM UTC

h4nkoslo wrote: post #13

It's a misconception that StarCraft is a strategy game. If you look at how it's actually played by human pros, it looks closer to a fighting game; very reflex-driven & heavy on micro-interactions. You would expect an un-gated AI with effectively infinite actions per second to do very well.

I think you are confusing the focus that is apparent in the micro of the game or micromanagement which requires high APM to the macro such as getting enough bases to keep on creating units to fight and troop composition.

Re: Deep Reinforcement Learning to Play StarCraft

#17

Post by devindotcom » Wed, Sep 14, 2016, 5:05 PM UTC

h4nkoslo wrote: post #13

It's a misconception that StarCraft is a strategy game. If you look at how it's actually played by human pros, it looks closer to a fighting game; very reflex-driven & heavy on micro-interactions. You would expect an un-gated AI with effectively infinite actions per second to do very well.

Well, it literally is a real time strategy game. It's just that once the meta stabilizes, macro advantages become fewer and fewer and micro skills become the deciders. Perhaps at this point it might be better to refer to it as a real time tactics game. The AI would presumably not fail at micro so it would dominate at this point in the game's lifecycle, but it might be able to be taken advantage of at a macro level.

Re: Deep Reinforcement Learning to Play StarCraft

#18

Post by empath75 » Wed, Sep 14, 2016, 5:11 PM UTC

h4nkoslo wrote: post #13

It's a misconception that StarCraft is a strategy game. If you look at how it's actually played by human pros, it looks closer to a fighting game; very reflex-driven & heavy on micro-interactions. You would expect an un-gated AI with effectively infinite actions per second to do very well.

It works on several different levels. You're talking about what competitive players call 'micro', which could be considered the implementation of the strategy.

I haven't played for a while, but usually there are a few basic 'builds', which are essentially memorized openings-- and there are 3 types of openings -- Macro builds where you focus on building an economy, while sacrificing military resources for a long-game, 'all-in' builds, which sacrifice your economy to build an early military advantage and win within the first few minutes, and various mid-range builds that try to do a little bit of both.

An all in is largely just down to micro and execution and it either wins or it doesn't, but the other two types of builds have a large strategic element-- for example, you need to scout to check if your opponent is all-in-ing, you can do harassment to distract your opponent from implementing his strategy by interrupting his economy, and then there's planning for the end game, building defenses, and the whole question of what you do if your original plan fails for one reason or another. There's a lot of thinking involved on multiple levels simultaneously, both spacial and temporal.

Re: Deep Reinforcement Learning to Play StarCraft

#19

Post by Analemma_ » Wed, Sep 14, 2016, 5:13 PM UTC

This is pretty cool, although I think MOBAs (Dota, LoL) would be an even better test of AI skills than StarCraft. They also have imperfect information, but place more importance on strategy and less on micro than StarCraft; require some game theory and bluffing in the draft, and would need multiple agents to cooperate (assuming you set it up so that you had 5 AIs play the game, with well-defined communication channels, rather than one controlling the five players, which I think is the right way to go).

Seems like there's more potential for useful AGI techniques in that direction.

Re: Deep Reinforcement Learning to Play StarCraft

#20

Post by PhearTheCeal » Wed, Sep 14, 2016, 5:17 PM UTC

h4nkoslo wrote: post #13

It's a misconception that StarCraft is a strategy game. If you look at how it's actually played by human pros, it looks closer to a fighting game; very reflex-driven & heavy on micro-interactions. You would expect an un-gated AI with effectively infinite actions per second to do very well.

It seems like most people in this thread are claiming that high APM and the ability to have perfect control over everything happening on the map will give the AI an advantage that will force a win.

Here is a video of one of the best current StarCraft bots losing to an D-rank (low skill) human player. The bot's APM is ~5500 while the human's is ~200. https://www.youtube.com/watch?v=ztNYOnx_YQo

The fact is, no AI has ever beaten even an amateur player in a tournament. Even with great micro, if your play is too predictable then the human will learn it and exploit it.

I, for one, am very excited to see the development of new StarCraft AIs. And especially SC2 AIs so that it can challenge the current world champions.