Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

DeepMind and Blizzard Open StarCraft II as an AI Research Environment

deepmind.com

Post reply on HN
Print view View original

211–220 of 283 posts

Re: DeepMind and Blizzard Open StarCraft II as an AI Research Environment

#211

Post by sidusknight »

jtolmar wrote: post #204
littlestymaar wrote: post #187

Earlier quoted context omitted.

This comment in insightful, thanks ! > Not to mention that the search space is massively massively larger That's what I'm not really convinced about. The build-order space is not that big (compared to Go's positions) and once you got a good micro-management engine I'm affraid this will lead to something like : if protos or zerg pick protoss then 8 gate -> 9 pylon -> scout : if no counter to 4-gates, then 4-gates and…

The preferred opening for Protoss in PvZ (on most maps) is the forge fast-expand. If the Zerg player doesn't want to play an economic game in response, they have a variety of all-in strategies available. There is a lengthy article on Team Liquid about how Protoss should respond to these. http://wiki.teamliquid.net/starcraft/Protoss_Counter_to_Zerg... What I'd note here is that: 1. It's a rather long list. 2. Good sco…

Protoss hasn't went FEE PvZ in a long time. It used to be good, though.

Re: DeepMind and Blizzard Open StarCraft II as an AI Research Environment

#212

Post by sidusknight »

aerovistae wrote: post #209
popcorncolonel wrote: post #101

Earlier quoted context omitted.

Why not first allow the AI to have unlimited APM and beat humans, then restrict it later? Because I don't think we're even close to the easier problem.

In what universe is taking 10,000 actions per minute an easier problem for a neural net than 100 such actions?

Have you played SC2 before? It's hard to explain if you haven't.

Re: DeepMind and Blizzard Open StarCraft II as an AI Research Environment

#213

Post by rjeli »

gradys wrote: post #201
Impossible wrote: post #150

Earlier quoted context omitted.

Minor nitpick, video games running on digital computers are by definition still discrete even if they feel continuous. Networked multiplayer wouldn't be possible in RTS games if that wasn't the case. The granularity of unit positions and turns in Starcraft obviously leads to a much larger state space, so I get what you're saying, for AI its effectively continuous.

They're discrete with such high cardinality that successful approaches will likely model them assuming they're basically continuous. Neural network layer activations are also discrete after all, but they're often 256+ dimensional vectors of float32s or float16s.

Well, WaveNet[0] outputs audio in the time (not freq.) domain using PixelCNN, so it's not unthinkable.

https://deepmind.com/blog/wavenet-generative-model-raw-audio...

Re: DeepMind and Blizzard Open StarCraft II as an AI Research Environment

#215

Post by mulmen »

branja wrote: post #193
mulmen wrote: post #186

Earlier quoted context omitted.

But why is that desirable? Why would we want to emulate the human physical handicaps in our quest to advance AI at a strategy level?

For the same reason the APM are limited: to ensure that what we are doing is really focusing on advancing strategy rather than brute mechanical skill. If I played against an AI using nothing but the rendered frames and sound of a game as input, I might not even make the stipulation on reflexes. I'd be humbled if I lost. As it stands now, most of the games I like have bad AI. Sure, it can be fun to play a hack and sla…

This is not the same AI you normally face in a game. Most (all?) of those AI opponents use rules written by the game developers to make decisions and some of them simply cheat to be competitive (cough Mario Kart 64 cough).

This blog is about creating AIs that interact with the game the same way humans do, the computer plays by the same rules and has no special access to the game state beyond what the player would have. With these constraints there are no existing bots for StarCraft or StarCraft 2 that can even beat the built-in rule-based AI. They aren't even close to beating professional players.

If the strategy abilities are so weak today that we can't even beat the tutorial AI then why introduce further arbitrary handicaps on the bots? How do those handicaps advance the state of the strategy layer? The AI has many potential advantages over the human player beyond just reaction time. Should we also limit the amount of data the bot considers to emulate the amount of inputs a human player can process? What about emulating human memory, can a human really learn from 60,000+ games? What about 1.5 million?

I do not think it is desirable to emulate human limitations in AI unless you are trying to create an artificial human. I think the advantage of creating an AI is to do something people can't already do so why should we impose our physical constraints on them?

I do not think it is important to separate reflex from strategy. Since every player has a different APM ability some strategies are more valid than others for each individual. If I do not have the reflexes of a professional player there are strategies I cannot employ. As long as StarCraft is not imposing APM limits on human players to maintain competitiveness the bots should also not have a limit.

Re: DeepMind and Blizzard Open StarCraft II as an AI Research Environment

#216

Post by jtraffic »

JefeChulo wrote: post #30

"so agents must interact with the game within limits of human dexterity in terms of “Actions Per Minute”." I am really glad they are limiting APM because otherwise things just get stupid.

Philosophically, I wonder if it's better to just acknowledge the differences between AI and humans, and let them play to their strengths. It seems common to think that we need to constrain AI in certain ways to be more like humans, but the constraints are always artificial. APM is one constraint, but what about working memory? What about multithreading? We already allow the AI access to computational resources humans don't have. Why draw that line at APM, exactly?

Re: DeepMind and Blizzard Open StarCraft II as an AI Research Environment

#217

Post by autokad »

gradys wrote: post #198
qub1t wrote: post #71

A lot of people here seem to be underestimating the difficulty of this problem. There are several incorrect comments saying that in SC1 AIs have already been able to beat professionals - right now they are nowhere near that level. Go is a discrete game where the game state is 100% known at all times. Starcraft is a continuous game and the game state is not 100% known at any given time. This alone makes it a much hard…

I think a big component is not really machine learning but more related to how to represent state at any given time, which will necessarily involve a lot of human-tweaking of distilling down what really are the important things that influence winning. I agreed with everything you said until here. Developing good representations of state is precisely what today's machine learning is so good at. This is the key contrib…

i feel like you misunderstood that part of the argument.

he is saying representing the state is very hard, and you are saying: given a well represented state, ML is very good at finding the important features, reducing the dementionality, and finding mathematical transformations, etc.

deep learning has been so successful with images because representing them is trivial - flattened pixel vector.

with your last paragraph is that in starcraft, that raises some questions on what rules is the AI going to adhere to.

in SC, you don't view the entire board. you view the minimap / hear noises and alerts and decide were to focus your attention on the map. in battle, being able to click and accurately place attacks quickly is important.

Do you give the computer full view of what they would be able to see? does the computer have 10 million clicks per second abilities, essentially every action is like hitting pause and then making the next action?

Re: DeepMind and Blizzard Open StarCraft II as an AI Research Environment

#218

Post by baddox »

bitxbitxbitcoin wrote: post #207
strgrd wrote: post #103

Earlier quoted context omitted.

I wonder if we will see any advanced cheese strats come out of this. I'm assuming some implementations will eventually develop micro control that is far beyond any human player's capabilities, which would make things like all-in probe rushing much more viable. Instead of playing the normal meta in a computer-vs-human, I imagine an advanced AI would simply send all of its workers off the mineral line as soon as the ga…

I'd say we're certainly going to see crazy advanced cheese strats - ones that humans wouldn't be able to hope to pull off. This could definitely be done with computer micro and wouldn't be defendable with human micro. An example would be moving probes around in such a way to maximize their shield regen - or switching the top clickable unit while stacked - who knows...

But if that's all the AI can do, and people know it, then defending should be pretty easy. Any worker or land-based rush can be fairly trivially defended by walling off.

Re: DeepMind and Blizzard Open StarCraft II as an AI Research Environment

#219

Post by joejev »

placeybordeaux wrote: post #159
j9461701 wrote: post #156

Earlier quoted context omitted.

Mid-way through WoL and again shortly after launch of HoTS. I played BW a lot as a kid, and the fundamental concepts carry over pretty decently. Always build workers, don't get supply blocked, expand as fast as your opponent lets you get away with, never let your resources bank, upgrades are always worth it, etc. etc. As I recall someone in WoL actually went from bronze to masters building literally no units except m…

To be fair marines and medivacs are amazing units in WoL. The same likely wouldn't apply with most other two unit combo. Roach ling maybe, maybe blink stalkers.

Destiny did this with just queens and drones. A lot of games were insane creep spread and spine crawler pushes.

Re: DeepMind and Blizzard Open StarCraft II as an AI Research Environment

#220

Post by gradys »

autokad wrote: post #217
gradys wrote: post #198

Earlier quoted context omitted.

I think a big component is not really machine learning but more related to how to represent state at any given time, which will necessarily involve a lot of human-tweaking of distilling down what really are the important things that influence winning. I agreed with everything you said until here. Developing good representations of state is precisely what today's machine learning is so good at. This is the key contrib…

i feel like you misunderstood that part of the argument. he is saying representing the state is very hard, and you are saying: given a well represented state, ML is very good at finding the important features, reducing the dementionality, and finding mathematical transformations, etc. deep learning has been so successful with images because representing them is trivial - flattened pixel vector. with your last paragra…

I was actually assuming the input representation would just be a video stream, which (combined with audio) is enough for human players, but looking more into it, it's a lot more than a video feed[1].

It feels a little like cheating, but I guess processing the game UI video feed isn't the interesting part of the problem. Plus, it makes the problem much more accessible to hobbyists who can't afford the GPU cluster required to productively experiment on models that process streams of 1080p video.

Still, in principle, I think modern ML modeling approaches could handle the problem of transforming the video feed into a useful high level state representation. I don't think I misunderstood the OP in that regard at least.

[1] - https://github.com/deepmind/pysc2/blob/master/docs/environme...

Post reply on HN