Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Grandmaster level in StarCraft II using multi-agent reinforcement learning

deepmind.com

Post reply on HN
Print view View original

231–240 of 321 posts

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#231

Post by ionforce »

gambler wrote: post #187

From the paper: >Humans play StarCraftthrough a screen that displays only part of the map along with a high-level view of the entire map, to e.g. avoid information overload. The agent interacts with the game through a similar camera-like interface What exactly does that mean? Does it or does it not play by operating purely on image data human players would see on the screen? How much of the system's interaction with…

Everything about AlphaStar is typed and discrete. It has perfect inputs because it uses an API (and does not read pixel data).

Human limitations that AlphaStar shares:

- Data that requires the camera to see (e.g. enemy location, enemy HP)

- Inability to examine/target cloaked units

Possibly unfair, super-human things AlphaStar has access to:

- Instantaneous awareness of cloaked units

- Knowledge of things humans need to infer/click (e.g. upgrades)

- Global map awareness of unit positions (taking into account fog of war)

Definitely unfair:

- Can select arbitrary collections of units, including outside of camera view

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#232

Post by tialaramex »

evandijk70 wrote: post #181
Mirioron wrote: post #132

Earlier quoted context omitted.

What makes this amazing isn't specific to StarCraft 2. AI in strategy games has been really lackluster. I can't think of a single example of a strategy game where an AI was competitive against experienced players due to strategy and tactics, rather than inhuman speed, accuracy or cheating. So it's not just about AI in StarCraft 2, but rather AI in essentially any (strategy) game. Now we have an example of an AI that…

I am not sure what you mean by 'strategy' here, don't Chess, Go, and No-Limit Hold'em fall under those categories?

The video game examples they give are what's called Real Time Strategy which is quite different from the games you described in which players explicitly take turns.

Chess and Go are already quite different from NLH, because they are purely about strategy. The best strategy in Chess either wins or leads to a draw if played by both sides every game. In NLH an optimal strategy just breaks even (ignoring the rake) against other optimal players and makes money on average against anybody else. But over even a few hundred hands you can't tell.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#233

Post by skybrian »

Mirioron wrote: post #214
skybrian wrote: post #168

Earlier quoted context omitted.

There is a genuine advance here, but keep in mind that when an AI is developed by the game developers, they're not necessarily playing to win, but to make the AI fun to beat, and without using too much computer power, which would make the game slower. Also, due to commercial pressures it's tough to put a lot of effort into the AI for a game that's still being changed to make it more fun. Even now, I wouldn't really e…

> There is a genuine advance here, but keep in mind that when an AI is developed by the game developers, they're not necessarily playing to win, but to make the AI fun to beat, and without using too much computer power, which would make the game slower. This point is being brought up a lot, but I don't really buy it. Yes, there have been instances where the AI being too good discouraged players from playing the game…

Well, yes, you need to ramp up the difficulty in a way that's fun.

The article describes "exploiter agents that focus on helping the main agent grow stronger" as key to their approach. That seems promising? Maybe it could be used to make bosses with specific strengths and weaknesses where part of the game is figuring out how to beat them.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#234

Post by dmreedy »

tialaramex wrote: post #227
dmreedy wrote: post #190

Earlier quoted context omitted.

> On the high level, there is no "chess AI", "go AI", "image classification AI" and "dexterous manipulation AI". These are all sides of the same coin, that gets significantly better every year. On a practical level, this is not true. There are different algorithms, different architectures, different hyperparameters required for each of these problems, and often for each subdomain within each of these problems, and of…

Are you writing this from last century? Deepmind's best-in-class chess and Go AIs are the same code (AlphaZero) just given respectively rules and game state input for either chess or Go and then allowed to train on the target game. One of the fun works in progress in this space is teaching AIs to play a suite of 80s video games. Getting quite good at several games where the idea is to go right and not die is pretty e…

I don't mean to imply AlphaZero is not impressive; it surely is. Nor do I mean to imply that any of these advances aren't impressive. I do mean to imply that "closed-world games with well-defined rules" is a relatively small subdomain of problems. And that BERT looks pretty different from AlphaZero.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#235

Post by nitwit005 »

papreclip wrote: post #157
bsaul wrote: post #58

Has anyone read the actual paper ? This summary really makes it look like "mission accomplished", but this was much much more interesting than that. We saw AI do "obviously stupid things", and we also saw them improve a lot in the middle of the trial, as many youtubers showed. AI was also much more interesting when playing the protoss race, and really felt like it was responding to the opponents actions, and the othe…

IMO it's a testament to how games like SC are collectively and thoroughly "solved" by the community, and how the games aren't that complex after all. I never followed SC or SC2, but my observation of the pro scene and competitive ladder for Warcraft 3 was that cookie cutter strats dominated. Pro players were typically those who executed best, not those who innovated best. Personally I felt disappointed by the fact th…

It's a bit more interesting when people play several matches against each other. They are forced to switch tactics.

Raw, spur of the moment, innovation probably isn't a good idea generally, as you'd be doing something with no practice. People do have a fairly big bag of tricks from older tactics that are less common these days.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#236

Post by scott_s »

Strom wrote: post #205

> Agents were capped at a max of 22 agent actions per 5 seconds, where one agent action corresponds to a selection, an ability and a target unit or point, which counts as up to 3 actions towards the in-game APM counter. Moving the camera also counts as an agent action, despite not being counted towards APM. I'm happy to see that they've greatly improved the APM cap. During the earlier showmatch they had an extremely…

How much do misclicks or accidental double clicks matter at the pro level?

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#237

Post by gpm »

fwip wrote: post #209
gpm wrote: post #163

Earlier quoted context omitted.

> Even basic strategies will win if they’re done faster. This isn't really true. Basic strategies done faster still lose miserably to humans, we can see this from the long history of SC Broodwar AI tournaments where they have a human play the best AI at the end (and always win). Faster helps, better strategy helps more, maybe we can't say the AI is doing as well as the top humans at strategy since it's faster, but we…

AI that ships with games (even the hardest difficulty) are made to be beatable. Of course pros beat them.

You misunderstand me, I'm not talking about the AI that shipped with broodwar. I'm talking about third party AIs made via bwapi competing against eachother in the various starcraft AI tournaments, such as

    AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment (AIIDE)
    IEEE Conference on Computational Intelligence and Games (CIG)
    Student StarCraft AI (SSCAI) Tournament
    BWAPI Bots Ladder
https://bwapi.github.io/

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#238

Post by antepodius »

scott_s wrote: post #236
Strom wrote: post #205

> Agents were capped at a max of 22 agent actions per 5 seconds, where one agent action corresponds to a selection, an ability and a target unit or point, which counts as up to 3 actions towards the in-game APM counter. Moving the camera also counts as an agent action, despite not being counted towards APM. I'm happy to see that they've greatly improved the APM cap. During the earlier showmatch they had an extremely…

How much do misclicks or accidental double clicks matter at the pro level?

A lot. In the same way that errors reduce data bandwidth, misclicks reduce effective APM in hectic situations. Classic examples are moving marines into banelings or units into disruptor shots.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#239

Post by enturn »

EGreg wrote: post #217
Strom wrote: post #205

> Agents were capped at a max of 22 agent actions per 5 seconds, where one agent action corresponds to a selection, an ability and a target unit or point, which counts as up to 3 actions towards the in-game APM counter. Moving the camera also counts as an agent action, despite not being counted towards APM. I'm happy to see that they've greatly improved the APM cap. During the earlier showmatch they had an extremely…

Why don’t they ever just have a virtual mouse api or whatever, with some lag and some jitter just to make it an actual apples to apples comparison? The AI is supposed to call this API. Then actions per minute would be irrelevant.

I'm just guessing but I think it would be easier to start from a perfect system and progressively work towards an accurate imperfect system rather than the other way of guessing an imperfect system and working back towards an accurate imperfect system.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#240

Post by scott_s »

antepodius wrote: post #238
scott_s wrote: post #236

Earlier quoted context omitted.

How much do misclicks or accidental double clicks matter at the pro level?

A lot. In the same way that errors reduce data bandwidth, misclicks reduce effective APM in hectic situations. Classic examples are moving marines into banelings or units into disruptor shots.

Sorry, I asked that wrong. Clearly misclicks matter at the pro level, as pros are going to exploit their competitors mistakes ruthlessly. What I should have asked is, at the pro level, how often do misclicks affect the outcome of a match?
Post reply on HN