Grandmaster level in StarCraft II using multi-agent reinforcement learning • Hacker News Board

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#211

Post by Erlich_Bachman » Thu, Oct 31, 2019, 7:35 PM UTC

pmoriarty wrote: post #191

Starcraft is a relatively simple game (which is one of the main reasons for its enormous popularity), so seeing an AI do well at it is not particularly impressive. I'd have been far more impressed had an AI beaten some of the best wargamers at a complex wargame, or had it beaten some of the best text-adventure game players at novel text adventure games neither of them had played before. The former would be difficult…

> Starcraft is a relatively simple game (which is one of the main reasons for its enormous popularity)

In the most practicle uses of the word "simple", especially in this context (number of possible actions and search space and whatnot) your statement is just incorrect.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#212

Post by Strom » Thu, Oct 31, 2019, 7:37 PM UTC

Erlich_Bachman wrote: post #206

Strom wrote: post #205

> Agents were capped at a max of 22 agent actions per 5 seconds, where one agent action corresponds to a selection, an ability and a target unit or point, which counts as up to 3 actions towards the in-game APM counter. Moving the camera also counts as an agent action, despite not being counted towards APM. I'm happy to see that they've greatly improved the APM cap. During the earlier showmatch they had an extremely…

Yes, those might have some impact, but it is clear that the progress is there, with this new APM cap and camera movement etc. You can also see in replays that the AI often makes mechanical mistakes, missing spells, missing units, even ordering wrong units from outside the screen - so it surely seems that if it's win rate was conditioned in any strong way on its sheer mechanical ability, it would have learned to not m…

The mistakes it makes are due to bad decisions. There have been no claims by DeepMind that they have some sort of chaos engineering [1] going on where the AI decides one thing and then the output system actually does another thing.

Also I think you overestimate the AI/IT knowledge of these top players that they're consulting. I have great respect towards them, but they're not renaissance men [2] who both play 10 hours of StarCraft per day and also know the subtleties of how computers work, not to mention bleeding edge AI. You can watch the previous showmatch [3] to see how DeepMind people lack knowledge of StarCraft and how the pro players they're consulting lack understanding of the AI and both are learning new things live on air. Its obvious that their cooperation is bearing fruit as they spend more time talking to each other, as evidenced by the new APM limits. However I'm willing to bet that they would reach many more good conclusions if they just continue working together.

--

[1] https://en.wikipedia.org/wiki/Chaos_engineering

[2] The problem with both parties (the pros & deepmind) is that they're so overspecialized. I'm nowhere near as good as them at their respective fields, but I am a professional programmer and diamond in StarCraft II. In addition I've built StarCraft II AI myself, although with different goals related to finding optimal strategies.

[3] https://www.twitch.tv/videos/369062832

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#213

Post by Erlich_Bachman » Thu, Oct 31, 2019, 7:38 PM UTC

CydeWeys wrote: post #116

plopz wrote: post #3

> placing within the top 0.15% of the region's 90,000 players > 61 wins out of 90 games against high-ranking players This doesn't seem to be quite as commanding as it was in Go. Do we know what MMR it reached or if it consistently beat players like Serral?

It seems like it's only a matter of time though. StarCraft II also has a rock-paper-scissors nature to it though, so you wouldn't expect even a perfect player to win 100% of the time. There are some strategies that are hard-counters to other strategies, and because of the imperfect information nature of the game, by the time you scout your opponent and see what they're doing, it may be too late to shift and deal with…

> by the time you scout your opponent and see what they're doing, it may be too late to shift and deal with it.

Unless you're Serral and your observers magically come out of nowhere and cover every single pixel of your base.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#214

Post by Mirioron » Thu, Oct 31, 2019, 7:38 PM UTC

skybrian wrote: post #168

Mirioron wrote: post #132

Earlier quoted context omitted.

What makes this amazing isn't specific to StarCraft 2. AI in strategy games has been really lackluster. I can't think of a single example of a strategy game where an AI was competitive against experienced players due to strategy and tactics, rather than inhuman speed, accuracy or cheating. So it's not just about AI in StarCraft 2, but rather AI in essentially any (strategy) game. Now we have an example of an AI that…

There is a genuine advance here, but keep in mind that when an AI is developed by the game developers, they're not necessarily playing to win, but to make the AI fun to beat, and without using too much computer power, which would make the game slower. Also, due to commercial pressures it's tough to put a lot of effort into the AI for a game that's still being changed to make it more fun. Even now, I wouldn't really e…

>There is a genuine advance here, but keep in mind that when an AI is developed by the game developers, they're not necessarily playing to win, but to make the AI fun to beat, and without using too much computer power, which would make the game slower.

This point is being brought up a lot, but I don't really buy it. Yes, there have been instances where the AI being too good discouraged players from playing the game as much, but this almost always happens due to the AI having some inhuman advantage that a person could not really replicate. I think players do want the AI to pose a challenge - that's what a lot of casual PvP games are about. Humans are (were?) the only ones that could offer a fair match against another human.

I will absolutely grant you the computational power point though. However, as hardware advances, this cost will become more and more acceptable.

I don't expect to see widedspread adoption of this in commercial games within a decade, but I think that in 2 or 3 decades this will be the norm. In fact, I would bet that once we can make an AI that is good at a game, we can also make it weaker. A game could estimate a player's skill rating silently and then adjust the AI's strength to give the player a difficult/fun time.

Of course, it could just be that AlphaStar is able to play well against humans, because players treat it like a human. Maybe the AI still has standard game AI-like weaknesses that can be exploited if people know that they're playing against an AI. Eg some Diamond league player went mass ravens and kicked AlphaStar's ass. The AI would have to learn how to deal with stuff like this on the fly and I don't think we're there yet.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#215

Post by hartator » Thu, Oct 31, 2019, 7:40 PM UTC

Instead of playing the APM rate, maybe they can slow down the game for the human players? If you play at 0.25x or 0.5x, it will be harder for the AI to out micro. Couple of micro strategies used by the AI, like staying at a 7 range when enemy can fire at 6 range is still a big reason why they are so effective.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#216

Post by sonofhans » Thu, Oct 31, 2019, 7:41 PM UTC

missosoup wrote: post #20

nightcracker wrote: post #5

> After 50 games, however, DeepMind hit a snag. Some players had noticed that three user accounts on the Battle.net gaming platform had played the exact same number of StarCraft II games over a similar time frame — the three accounts that AlphaStar was secretly using. When watching replays of these matches, players noticed that the account owner was performing actions that would be extremely difficult, if not impossi…

I'm not sure this is unfairness. IIRC they put in a fair bit of effort to put it on a level playing field with humans by limiting APM and not allowing it to observe multiple areas simultaneously by spam moving the camera. It might have some minor unfair advantage in terms of being able to click with pixel perfect accuracy, but they're marginal and from watching this project evolve, it's pretty clear that the strategi…

"... the strategic planning aspect of alphastar has indeed become better than humans."

This is not true. Top-level human players take into account who their opponent is, what builds they've used in recent games, their proclivities, strengths, and weaknesses. In a 7-game match they'll intentionally use builds which (in order to deceive) appear the same but have very different effects.

AlphaStar has better strategy than other AIs, but that's a low bar. My young son has better strategic appreciation of the game, and easily points out the difference between good strategy and AlphaStar's next-level micro.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#217

Post by EGreg » Thu, Oct 31, 2019, 7:47 PM UTC

Strom wrote: post #205

> Agents were capped at a max of 22 agent actions per 5 seconds, where one agent action corresponds to a selection, an ability and a target unit or point, which counts as up to 3 actions towards the in-game APM counter. Moving the camera also counts as an agent action, despite not being counted towards APM. I'm happy to see that they've greatly improved the APM cap. During the earlier showmatch they had an extremely…

Why don’t they ever just have a virtual mouse api or whatever, with some lag and some jitter just to make it an actual apples to apples comparison?

The AI is supposed to call this API. Then actions per minute would be irrelevant.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#218

Post by timeattack » Thu, Oct 31, 2019, 7:49 PM UTC

sgillen wrote: post #197

I see a lot of comments downplaying this achievement, saying it's not impressive or it won't be impressive until X condition is met. I welcome skepticism and criticism for this sort of thing, and think most of it that I've seen here is well founded. But I would like to take a second to explain why I think this, and really all the progress in this area, is actually a really impressive achievement to me. Let me try and…

I think that downplaying the achievement is more like defensive mechanism which exists to protect illusion of human brain superiority.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#219

Post by bsaul » Thu, Oct 31, 2019, 7:50 PM UTC

NoodleIncident wrote: post #176

bsaul wrote: post #58

Has anyone read the actual paper ? This summary really makes it look like "mission accomplished", but this was much much more interesting than that. We saw AI do "obviously stupid things", and we also saw them improve a lot in the middle of the trial, as many youtubers showed. AI was also much more interesting when playing the protoss race, and really felt like it was responding to the opponents actions, and the othe…

This sounds like it was posted before the link was changed, but at least one pro was impressed by its novel style: > AlphaStar is an intriguing and unorthodox player – one with the reflexes and speed of the best pros but strategies and a style that are entirely its own. The way AlphaStar was trained, with agents competing against each other in a league, has resulted in gameplay that’s unimaginably unusual; it really…

yes indeed, it seems the most recent batch of games brings interesting new things

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#220

Post by jamra » Thu, Oct 31, 2019, 7:55 PM UTC

goalieca wrote: post #85

jamra wrote: post #71

Earlier quoted context omitted.

However the computer can macro while in a fight which is something people can’t dI. We would miss the fight and lose the game. It’s not just a tactics game. Attention span and where your eyes are matter.

It’s so bad that below master people shouldn’t really be doing too much micro during a fight because they’ll lose out on macro. Even in pro matches you see attention span issues and sometimes avoidance of too much micro to win. Really good Micro is going to win you any battle.

I’m in diamond 2 and I definitely micro in battles. The difference between platinum 1 and diamond 3 is largely macro but it very soon becomes micro as well.