Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Grandmaster level in StarCraft II using multi-agent reinforcement learning

deepmind.com

Post reply on HN
Print view View original

221–230 of 321 posts

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#221

Post by sgillen »

timeattack wrote: post #218
sgillen wrote: post #197

I see a lot of comments downplaying this achievement, saying it's not impressive or it won't be impressive until X condition is met. I welcome skepticism and criticism for this sort of thing, and think most of it that I've seen here is well founded. But I would like to take a second to explain why I think this, and really all the progress in this area, is actually a really impressive achievement to me. Let me try and…

I think that downplaying the achievement is more like defensive mechanism which exists to protect illusion of human brain superiority.

Part of the problem is the hype in the other direction. I guess there are those who claim that this sort of result constitutes super human intelligence or something close to AGI. When in reality this is a hyper specialized system that has taken many man years of work to realize.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#222

Post by sidlls »

tiborsaas wrote: post #52
mieseratte wrote: post #30

Earlier quoted context omitted.

> For humans that would be almost impossible, because the screen would scroll when the mouse approaches the border. That humans cannot reliably perform these actions because of the limitations of our corporeal form means that Alphastar has an advantage over a human player. Limiting APM isn't enough.

It's an AI system, of course it has some advantage, that's why we are building them.

The advantage here is due to artifice and not necessarily generalizable.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#223

Post by vkou »

Jyaif wrote: post #92
bsaul wrote: post #58

Has anyone read the actual paper ? This summary really makes it look like "mission accomplished", but this was much much more interesting than that. We saw AI do "obviously stupid things", and we also saw them improve a lot in the middle of the trial, as many youtubers showed. AI was also much more interesting when playing the protoss race, and really felt like it was responding to the opponents actions, and the othe…

> Actually not a single game made pro player realize something new about the game. People are now over saturating their mineral line (making more probes than before), so I don't think that's true.

Why?

Is it because they expect to lose workers to harassment? Or is it so that they can saturate new expansions quicker?

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#225

Post by floitsch »

MiracleUser wrote: post #80
floitsch wrote: post #49

Earlier quoted context omitted.

The question isn't whether the AI still has advantages, but whether it's an unfair one. The current restrictions were implemented with input from TLO, and seem to work quite nicely. One could, of course, add more restrictions (like not noticing things on the minimap all the time...), but that's not what's it about anymore. At this point, micro is comparable to humans, and we can start to compare macro and strategy. F…

> a well executed strategy with few errors often wins, even if it wasn't optimal I think this is hits the chord with why a lot of people are talking about this bot as cheating more than they are being outplayed. It's like fighting an aimbot in an FPS. They arent outsmarting you, but the mechanical consistency is inhuman and you are unable to force mistakes. At best itll feel like playing someone on their best day. Th…

I don't think the comparison with aimbots is fair. Aimbots are more like a perfect micro. In the beginning (when they did their first show match), Alphastar was insane at that. Now the micro is not giving an advantage anymore.

Where alphastar shines, is in making a strategy work. It won't attack into something that looks dangerous. It won't forget to build more units. It won't be distracted on two fronts. (That said: Alphastar did some really stupid mistakes as well.)

I agree with you, though: it would be much more interesting, if Alphastar was actually handicapped in the micro (at least at a later point in the game). Then it would really need to find interesting strategies and counter opposing units with more cost-efficient ones.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#226

Post by floitsch »

skybrian wrote: post #168
Mirioron wrote: post #132

Earlier quoted context omitted.

What makes this amazing isn't specific to StarCraft 2. AI in strategy games has been really lackluster. I can't think of a single example of a strategy game where an AI was competitive against experienced players due to strategy and tactics, rather than inhuman speed, accuracy or cheating. So it's not just about AI in StarCraft 2, but rather AI in essentially any (strategy) game. Now we have an example of an AI that…

There is a genuine advance here, but keep in mind that when an AI is developed by the game developers, they're not necessarily playing to win, but to make the AI fun to beat, and without using too much computer power, which would make the game slower. Also, due to commercial pressures it's tough to put a lot of effort into the AI for a game that's still being changed to make it more fun. Even now, I wouldn't really e…

Hi Brian,

I really hope this will change, though. Skilled AIs could make so many games more interesting and better.

Many online games really suffer from bad multiplayer experiences. Players that drop in the middle of a game, ...

If Overwatch, Destiny, ... had really good AIs, that would make the game more attractive. If a player drops, they could replace it with similarly skilled AI. Instead of waiting for 12 players of the same skill-level, they could just play with some AIs.

They could even make it such, that humans win 70% of the time, and let the AIs just lose more often.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#227

Post by tialaramex »

dmreedy wrote: post #190
jrx wrote: post #159

Earlier quoted context omitted.

> This is like if the world's best chess AI had gone from losing high school tournaments to being competitive with Kasparov in less than 3 years. I don't think it's like that at all. On the high level, there is no "chess AI", "go AI", "image classification AI" and "dexterous manipulation AI". These are all sides of the same coin, that gets significantly better every year. Adding support for the new game or new "envir…

> On the high level, there is no "chess AI", "go AI", "image classification AI" and "dexterous manipulation AI". These are all sides of the same coin, that gets significantly better every year. On a practical level, this is not true. There are different algorithms, different architectures, different hyperparameters required for each of these problems, and often for each subdomain within each of these problems, and of…

Are you writing this from last century?

Deepmind's best-in-class chess and Go AIs are the same code (AlphaZero) just given respectively rules and game state input for either chess or Go and then allowed to train on the target game.

One of the fun works in progress in this space is teaching AIs to play a suite of 80s video games. Getting quite good at several games where the idea is to go right and not die is pretty easy these days, but Deepmind's work can do a broader variety only coming badly unstuck on games where it's hard to discern your progress at all without some meta-knowledge.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#228

Post by MichaelDickens »

Strom wrote: post #205

> Agents were capped at a max of 22 agent actions per 5 seconds, where one agent action corresponds to a selection, an ability and a target unit or point, which counts as up to 3 actions towards the in-game APM counter. Moving the camera also counts as an agent action, despite not being counted towards APM. I'm happy to see that they've greatly improved the APM cap. During the earlier showmatch they had an extremely…

It's not exactly the same as a showmatch, but you can find lots of YouTube videos analyzing replays of AlphaStar playing on the ladder. For example, this game against 2018 world champion Serral: https://www.youtube.com/watch?v=_BOp10v8kuM

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#229

Post by skybrian »

floitsch wrote: post #226
skybrian wrote: post #168

Earlier quoted context omitted.

There is a genuine advance here, but keep in mind that when an AI is developed by the game developers, they're not necessarily playing to win, but to make the AI fun to beat, and without using too much computer power, which would make the game slower. Also, due to commercial pressures it's tough to put a lot of effort into the AI for a game that's still being changed to make it more fun. Even now, I wouldn't really e…

Hi Brian, I really hope this will change, though. Skilled AIs could make so many games more interesting and better. Many online games really suffer from bad multiplayer experiences. Players that drop in the middle of a game, ... If Overwatch, Destiny, ... had really good AIs, that would make the game more attractive. If a player drops, they could replace it with similarly skilled AI. Instead of waiting for 12 players…

Yep, I'd like to see it too. If one popular game really did single-player with AI opponents well, maybe it would start a trend?

AI development probably needs to be made easier, somehow. Game developers have a lot to do.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#230

Post by DuskStar »

JamesBarney wrote: post #202
papreclip wrote: post #157

Earlier quoted context omitted.

IMO it's a testament to how games like SC are collectively and thoroughly "solved" by the community, and how the games aren't that complex after all. I never followed SC or SC2, but my observation of the pro scene and competitive ladder for Warcraft 3 was that cookie cutter strats dominated. Pro players were typically those who executed best, not those who innovated best. Personally I felt disappointed by the fact th…

There is a lot of strategy in RTS's but alpha star doesn't need to exploit any of it because it can win on mechanics. It has perfect macro, and perfect micro, which allows it to beat players without having to learn strategies or tactics. It's kind of like competing against a gorilla in boxing chess. Just because a gorilla is dominant doesn't mean boxing chess doesn't require chess skills, only that a gorilla doesn't…

Are you sure it has perfect micro? Can you even execute perfect micro on 30 actions per 5 seconds?
Post reply on HN