Grandmaster level in StarCraft II using multi-agent reinforcement learning • Hacker News Board

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#201

Post by emerongi » Thu, Oct 31, 2019, 7:06 PM UTC

gambler wrote: post #198

interblag wrote: post #59

This is a really interesting one to digest. As with previous announcements about AlphaStar, much of the feedback (here and elsewhere) is about the fundamental challenge of assessing human vs. machine in an RTS. These points are very valid - stepping back however, this still feels like a pretty incredible accomplishment. I'm a gold league SC2 player, so maybe in the 30th-50th percentile. Three years ago, when DeepMind…

>much of the feedback (here and elsewhere) is about the fundamental challenge of assessing human vs. machine in an RTS It's amazing that most people here don't understand that AI performance in any one computer game relative to humans is largely irrelevant. A system that can play many games at a mediocre level, but does it without any hand-holding, clever APIs or architecture adaptation is infinitely more impressive…

Most of the approaches used are re-usable, which is a big part of why we can develop new AIs for games faster than ever. You can take the algorithm(s) that was used in one game and use it in another.

Yes, a human is still needed to decide which approach to use, but we are slowly approaching a world where building an AI becomes easier and faster. It will become absolutely irrelevant that a single AI can not play all games, because whenever a new game comes out, someone will be able to build a superhuman AI on it within a week/month.

The brains of an AI are also transferrable. Built a superhuman AI? Send it to a friend! Compare it to a human, who would need to spend enormous amounts of time to transfer their game-brain to someone else. If you want, you can bundle all those AIs into one and pretend it can play any game.

"It's amazing that most people here don't understand that human performance in all computer games relative to AIs is largely irrelevant"

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#202

Post by JamesBarney » Thu, Oct 31, 2019, 7:11 PM UTC

papreclip wrote: post #157

bsaul wrote: post #58

Has anyone read the actual paper ? This summary really makes it look like "mission accomplished", but this was much much more interesting than that. We saw AI do "obviously stupid things", and we also saw them improve a lot in the middle of the trial, as many youtubers showed. AI was also much more interesting when playing the protoss race, and really felt like it was responding to the opponents actions, and the othe…

IMO it's a testament to how games like SC are collectively and thoroughly "solved" by the community, and how the games aren't that complex after all. I never followed SC or SC2, but my observation of the pro scene and competitive ladder for Warcraft 3 was that cookie cutter strats dominated. Pro players were typically those who executed best, not those who innovated best. Personally I felt disappointed by the fact th…

There is a lot of strategy in RTS's but alpha star doesn't need to exploit any of it because it can win on mechanics. It has perfect macro, and perfect micro, which allows it to beat players without having to learn strategies or tactics.

It's kind of like competing against a gorilla in boxing chess. Just because a gorilla is dominant doesn't mean boxing chess doesn't require chess skills, only that a gorilla doesn't need them.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#203

Post by jetbooster » Thu, Oct 31, 2019, 7:12 PM UTC

dave_b wrote: post #155

gwd wrote: post #66

Earlier quoted context omitted.

> These extremely difficult/impossible things didn't really give an advantage. One of the videos I watched compared APM (Actions Per Minute) with EPM (Effective actions Per Minute). AlphaStar always has them nearly identical, which would be (according to him) basically impossible for humans.

Right, one example I remember from watching the games back in January was stalker micro. The high EPM of the AI allowed it to micro them picture perfectly and blink every single one away right before it died. In doing so it won a battle against impossible odds, and the announcers even commented on how it was way beyond the capabilities of the best players.

That exact strat is the reason they implemented the APM limit, it was a cheese strat only a machine could implement.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#204

Post by Sohcahtoa82 » Thu, Oct 31, 2019, 7:15 PM UTC

CzarnyZiutek wrote: post #57

calculator beats top human at multiplication (1957) AI is just a f.... calculator.

I learned how to use a calculator in Elementary school, that doesn't mean I knew how to solve calculus problems.

AI needs to know what to calculate. According to the article, at any step in the game, there are 10^26 possible actions to take. Good luck calculating which ones to try via brute force.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#205

Post by Strom » Thu, Oct 31, 2019, 7:15 PM UTC

> Agents were capped at a max of 22 agent actions per 5 seconds, where one agent action corresponds to a selection, an ability and a target unit or point, which counts as up to 3 actions towards the in-game APM counter. Moving the camera also counts as an agent action, despite not being counted towards APM.

I'm happy to see that they've greatly improved the APM cap. During the earlier showmatch they had an extremely naive cap on the average APM during the whole game. Which meant that the AI could take it easy for most of the time and then during battle sequences hit peaks of 2000 APM. Also the previous cap was based on the in-game APM counter, which doesn't count some things like moving the camera, which is now addressed. The current state sounds a lot better.

However it seems still superhuman in mechanical non-strategy ways, e.g. a human can misclick (click too early/late when moving the mouse, or just miss the target with the cursor completely) or do accidental double clicks etc. These end up being very costly mistakes against a zero mechanical mistake AI. Which in turn means that the AI can win with an inferior strategy due to the extra gains it has via superior mechanical accuracy. In other words this artificial intelligence is still under significacnt artificial motor skills welfare, and thus even if it starts beating pro players we shouldn't be too quick to talk about how it's the intelligence part that got it the win.

All of that said, I'm liking the progress and am excited to see what they achieve with this next. Would love another showmatch against pro players.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#206

Post by Erlich_Bachman » Thu, Oct 31, 2019, 7:24 PM UTC

Strom wrote: post #205

> Agents were capped at a max of 22 agent actions per 5 seconds, where one agent action corresponds to a selection, an ability and a target unit or point, which counts as up to 3 actions towards the in-game APM counter. Moving the camera also counts as an agent action, despite not being counted towards APM. I'm happy to see that they've greatly improved the APM cap. During the earlier showmatch they had an extremely…

Yes, those might have some impact, but it is clear that the progress is there, with this new APM cap and camera movement etc.

You can also see in replays that the AI often makes mechanical mistakes, missing spells, missing units, even ordering wrong units from outside the screen - so it surely seems that if it's win rate was conditioned in any strong way on its sheer mechanical ability, it would have learned to not make any such mistakes. Since the mistakes are clearly there - it seems that its power comes primarily from somewhere else, likely from the AI ability to choose the right actions, not from mechanical power of executing them perfectly.

Thus it is not likely that "this artificial intelligence is still under significacnt artificial motor skills welfare". They have consulted top players before they set this AI up, and if those details were important, they would have baked them into the limitations list already.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#207

Post by lsorber » Thu, Oct 31, 2019, 7:28 PM UTC

Strom wrote: post #205

> Agents were capped at a max of 22 agent actions per 5 seconds, where one agent action corresponds to a selection, an ability and a target unit or point, which counts as up to 3 actions towards the in-game APM counter. Moving the camera also counts as an agent action, despite not being counted towards APM. I'm happy to see that they've greatly improved the APM cap. During the earlier showmatch they had an extremely…

And even with limited APM, the response time can still be superhuman. Bit disappointed that they didn't add some human-level latency to the agents' actions.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#208

Post by Erlich_Bachman » Thu, Oct 31, 2019, 7:29 PM UTC

hoseja wrote: post #15

There has always been the issue of interface when playing videogames AI vs human. Either give the human a brain-computer interface or give the AI a mouse, keyboard, monitor, robot hands and a camera. Anything else seems inherently unfair.

If you read the paper, this is exactly what they have done, and this is why this achievement is so interesting.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#209

Post by fwip » Thu, Oct 31, 2019, 7:29 PM UTC

gpm wrote: post #163

300bps wrote: post #136

Earlier quoted context omitted.

Starcraft is an incredibly complex game. Even basic strategies will win if they’re done faster. APM (actions per minute) is a very significant factor into who is winning. Apparently they limited their AI player to 264 APM but that’s still incredibly high and done with machine level consistency. That’s almost 4.5 actions per second!! I know there are human level players at and probably above that level but that really…

> Even basic strategies will win if they’re done faster. This isn't really true. Basic strategies done faster still lose miserably to humans, we can see this from the long history of SC Broodwar AI tournaments where they have a human play the best AI at the end (and always win). Faster helps, better strategy helps more, maybe we can't say the AI is doing as well as the top humans at strategy since it's faster, but we…

AI that ships with games (even the hardest difficulty) are made to be beatable. Of course pros beat them.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#210

Post by Erlich_Bachman » Thu, Oct 31, 2019, 7:34 PM UTC

gambler wrote: post #192

redkoala wrote: post #188

Earlier quoted context omitted.

It's meant to operate purely on data viewable on the screen to mimic a human player's experience and provide an even playing ground.

Can you cite something that confirms your assertion? From what I'm reading in the paper, it sounds like there is some custom interface in play: >AlphaStar can target locations more accurately than humans outside the camera, although less accurately within it because target locations (selected on a 256x256 grid) are treated the same inside and outside the camera. It's really hard to parse what such statements mean. Th…

> The fact that someone who is cited as a co-author of the paper

They are not just some random person, they were top SCII players. Who else if not them would know this well enough to make assumptions?

There is a custom interface in the sense that the bot does not read pixels from the screen - it reads the information through the API, same information that is usually presented on a screen. But the amount of information is limited to exactly what a human would see at the same time using the standard SCII interface.