Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Grandmaster level in StarCraft II using multi-agent reinforcement learning

deepmind.com

Post reply on HN
Print view View original

251–260 of 321 posts

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#251

Post by TaupeRanger »

timeattack wrote: post #218
sgillen wrote: post #197

I see a lot of comments downplaying this achievement, saying it's not impressive or it won't be impressive until X condition is met. I welcome skepticism and criticism for this sort of thing, and think most of it that I've seen here is well founded. But I would like to take a second to explain why I think this, and really all the progress in this area, is actually a really impressive achievement to me. Let me try and…

I think that downplaying the achievement is more like defensive mechanism which exists to protect illusion of human brain superiority.

Not at all. We've known machines are better than humans at machine-like tasks for hundreds of years. IMO responses like yours are defense mechanisms of AI researchers/enthusiasts who want to protect the illusion that they are creating things that do anything remotely resembling what the human brain does.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#252

Post by TaupeRanger »

jplayer01 wrote: post #106
Nimitz14 wrote: post #86

Earlier quoted context omitted.

I don't think you understood his point. A player doesn't need to have seen a strategy before to react correctly to it. Sometimes a player will pull something completely unexpected out of a hat and lose, because the other guy reacted correctly thanks to his game experience. If you watch any high GM player stream half the time his reaction to what his (worse) opponent is doing is "WTF is this?" as he then proceeds to c…

I do understand his point. There are just as many examples of people being so overwhelmed by the new strategy that they don't know how to respond to it and lose. And then learn how to deal with it in later matches. Also, downvoting me for disagreeing is a dick move.

And yet, in Go, the AI adapted to any "weird strategy" and won regardless. The fact that AlphaStar can't is an obvious weakness.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#253

Post by hervature »

Having a hard time to parse what is the action space here.

The paper claims: AlphaStar’s action space is defined as a set of functions with typed arguments

Looking at citation 7, it seems like they are structuring the action space as (First pick high level action)->(Pick argument 1 for action)->...->(Pick argument n for action). If this is the case, this seems to be "cheating" calling this AI as humans have completely picked out the actions. That is, the achievement here this: given what humans consider useful actions, AlphaStar can play at a grandmaster level.

The achievement here is mostly engineering in my opinion. One that extends far further than the 40ish people list on the paper. Probably an effort of over 1,000 people. From casually looking over the paper, there is nothing significantly different than AlphaZero or previous art. Again, the achievement here is listed under the infrastructure section of the paper.

In summary, this is a great step forward but now we need to start developing techniques to learn these action space hierarchies instead of throwing more power at increasingly difficult games.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#254

Post by Nimitz14 »

Jyaif wrote: post #92
bsaul wrote: post #58

Has anyone read the actual paper ? This summary really makes it look like "mission accomplished", but this was much much more interesting than that. We saw AI do "obviously stupid things", and we also saw them improve a lot in the middle of the trial, as many youtubers showed. AI was also much more interesting when playing the protoss race, and really felt like it was responding to the opponents actions, and the othe…

> Actually not a single game made pro player realize something new about the game. People are now over saturating their mineral line (making more probes than before), so I don't think that's true.

This is wrong. See posts here from ptitdrogo: https://www.reddit.com/r/starcraft/comments/d4n3tw/alphastar...

> People oversaturated in wol and hots because you didn't expand a lot in these games and bases have a lot less minerals in LOTV, not because of some kind of lost knowledge like some comments seem to think here.

[...]

> Meanwhile alphastar was going 2 gate robo staying on one base making a fuck ton of probes to take a super late natural, it was bad, and anybody calling it the future really bothered me.

Alphastar has not introduced anything new. Its play is strategically poor.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#255

Post by tialaramex »

dmreedy wrote: post #234
tialaramex wrote: post #227

Earlier quoted context omitted.

Are you writing this from last century? Deepmind's best-in-class chess and Go AIs are the same code (AlphaZero) just given respectively rules and game state input for either chess or Go and then allowed to train on the target game. One of the fun works in progress in this space is teaching AIs to play a suite of 80s video games. Getting quite good at several games where the idea is to go right and not die is pretty e…

I don't mean to imply AlphaZero is not impressive; it surely is. Nor do I mean to imply that any of these advances aren't impressive. I do mean to imply that "closed-world games with well-defined rules" is a relatively small subdomain of problems. And that BERT looks pretty different from AlphaZero.

The post you disputed pointed out that there aren't separate AIs needed for things like Go or Chess. Because there aren't (any more) the Deepmind work showed that you can just generalize to learn all games in this class the same way.

You claimed that "different architectures" are needed. Not true. And further you claimed this is true even for "each subdomain". This would have been a fair point in 1989. Traditional chess AIs approach the opening very differently for example, relying on fixed "books" of known good openings. But AlphaZero since it is a generalist doesn't do this, it plays every part of a match the same way.

Now you've gone from asserting that Chess and Go need separate AIs to claiming that since BERT and AlphaZero are different software it makes your point. Humans pretty clearly don't have a single structure that's doing all the work in both playing Go (AlphaZero) and understanding English (BERT) either - so that's a pretty bold bit of goalpost moving.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#257

Post by drchewbacca »

TaupeRanger wrote: post #251
timeattack wrote: post #218

Earlier quoted context omitted.

I think that downplaying the achievement is more like defensive mechanism which exists to protect illusion of human brain superiority.

Not at all. We've known machines are better than humans at machine-like tasks for hundreds of years. IMO responses like yours are defense mechanisms of AI researchers/enthusiasts who want to protect the illusion that they are creating things that do anything remotely resembling what the human brain does.

Kasparov comments on chess computers in an interview with Thierry Paunin on pages 4-5 of issue 55 of Jeux & Stratégie (published in 1989):

‘Question: ... Two top grandmasters have gone down to chess computers: Portisch against “Leonardo” and Larsen against “Deep Thought”. It is well known that you have strong views on this subject. Will a computer be world champion, one day ...?

Kasparov: Ridiculous! A machine will always remain a machine, that is to say a tool to help the player work and prepare. Never shall I be beaten by a machine! Never will a program be invented which surpasses human intelligence. And when I say intelligence, I also mean intuition and imagination. Can you see a machine writing a novel or poetry? Better still, can you imagine a machine conducting this interview instead of you? With me replying to its questions?’

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#258

Post by YeGoblynQueenne »

emerongi wrote: post #201
gambler wrote: post #198

Earlier quoted context omitted.

>much of the feedback (here and elsewhere) is about the fundamental challenge of assessing human vs. machine in an RTS It's amazing that most people here don't understand that AI performance in any one computer game relative to humans is largely irrelevant. A system that can play many games at a mediocre level, but does it without any hand-holding, clever APIs or architecture adaptation is infinitely more impressive…

Most of the approaches used are re-usable, which is a big part of why we can develop new AIs for games faster than ever. You can take the algorithm(s) that was used in one game and use it in another. Yes, a human is still needed to decide which approach to use, but we are slowly approaching a world where building an AI becomes easier and faster. It will become absolutely irrelevant that a single AI can not play all g…

>> The brains of an AI are also transferrable. Built a superhuman AI? Send it to a friend!

So far I haven't seen Google sending their AIs to a friend.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#259

Post by empath75 »

Strom wrote: post #205

> Agents were capped at a max of 22 agent actions per 5 seconds, where one agent action corresponds to a selection, an ability and a target unit or point, which counts as up to 3 actions towards the in-game APM counter. Moving the camera also counts as an agent action, despite not being counted towards APM. I'm happy to see that they've greatly improved the APM cap. During the earlier showmatch they had an extremely…

Grandmaster level players make those kinds of mistakes extremely rarely and if they’re the only difference between winning and losing, the ai is an extraordinary achievement.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#260

Post by empath75 »

tryptophan wrote: post #149
300bps wrote: post #136

Earlier quoted context omitted.

Starcraft is an incredibly complex game. Even basic strategies will win if they’re done faster. APM (actions per minute) is a very significant factor into who is winning. Apparently they limited their AI player to 264 APM but that’s still incredibly high and done with machine level consistency. That’s almost 4.5 actions per second!! I know there are human level players at and probably above that level but that really…

And the bot also can parse the entire screen in .03 seconds and then jump to a new area of the map. No human can monitor the entire map like the bot can.

Sometimes these critiques seem to be complaining that the ai is too intelligent. That’s sort of the point isn’t it.
Post reply on HN