Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Grandmaster level in StarCraft II using multi-agent reinforcement learning

deepmind.com

Post reply on HN
Print view View original

141–150 of 321 posts

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#141

Post by anonymfus »

wpasc wrote: post #96
MiracleUser wrote: post #80

Earlier quoted context omitted.

> a well executed strategy with few errors often wins, even if it wasn't optimal I think this is hits the chord with why a lot of people are talking about this bot as cheating more than they are being outplayed. It's like fighting an aimbot in an FPS. They arent outsmarting you, but the mechanical consistency is inhuman and you are unable to force mistakes. At best itll feel like playing someone on their best day. Th…

The issue with limiting it past what humans can do is that in the self play training they'd never encounter what humans could do. The benefits of trying to make the limitations somewhat human-like is that the self play optimizes around roughly what it will encounter when it goes against humans

May be make self-play asymmetrical so sides in the game would have different sets of limitations? So main agent would have subhuman micro and exploiter agents would have superhuman micro.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#143

Post by 6gvONxR4sf7o »

angel_j wrote: post #111

I don't think folks realize how much of a game changer AlphaStar is. If it can do the same "job" those high level players, it can probably do any job that doesn't require lifting. It could be playing all the traffic lights in a city. It could manage a fleet of factory drones. It could coach humans engaged in sport or business. It could probably figure out spreadsheets. Drill down, and perhaps it can play with chemica…

Sure, if you give it a few bazillion years of practice. We haven't figured out sample complexity yet (the amount of experience required to learn).

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#144

Post by Mirioron »

gwd wrote: post #66
floitsch wrote: post #16

Earlier quoted context omitted.

These extremely difficult/impossible things didn't really give an advantage. For example, AlphaStar would sometimes click on an object at the border of the screen. For humans that would be almost impossible, because the screen would scroll when the mouse approaches the border. Similarly, AlphaStar would not play with group hotkeys, but use a different technique. However, in none of the analyses, people noticed things…

> These extremely difficult/impossible things didn't really give an advantage. One of the videos I watched compared APM (Actions Per Minute) with EPM (Effective actions Per Minute). AlphaStar always has them nearly identical, which would be (according to him) basically impossible for humans.

Of course it's impossible for humans. Humans are used to clicking multiple times to do an action. Some actions you want to take in the game are very important, failure to do them correctly can mean a lost game, for example moving units in combat properly. This means that it's better to spam click that action a few times to make sure that the button press is registered properly, because it is possible for button presses to fail due to hardware, software, and most often a coordination error. This means that essentially every player has a much higher APM than EPM, but it's really EPM that counts. APM itself is mostly irrelevant.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#145

Post by jcampbell1 »

Jyaif wrote: post #92
bsaul wrote: post #58

Has anyone read the actual paper ? This summary really makes it look like "mission accomplished", but this was much much more interesting than that. We saw AI do "obviously stupid things", and we also saw them improve a lot in the middle of the trial, as many youtubers showed. AI was also much more interesting when playing the protoss race, and really felt like it was responding to the opponents actions, and the othe…

> Actually not a single game made pro player realize something new about the game. People are now over saturating their mineral line (making more probes than before), so I don't think that's true.

There were other cases of this such as 3 rather than 2 early air units. The AI seemed to anticipate losses in a way that is almost more rational than humans.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#146

Post by ionforce »

Mirioron wrote: post #137
ionforce wrote: post #117

Earlier quoted context omitted.

> But most surprising is that it didn't make any "breathtaking" moves or actions, as opposed to AlphaGO. I don't play any competitive StarCraft so my view might be moot but I was surprised at the number of siege tanks it uses. It made me wonder if there's some critical advantage to having so many tanks stacked up in a line so that the splash is spread out. Also I find it weird that it did not build any marauders at a…

> Also I find it weird that it did not build any marauders at all, in any of the games I've seen. This is off-topic, but I was thinking about this the other day - couldn't AI be used to balance a competitive game in this sense? Imagine that the AI becomes so good that human players very rarely win against the AI in a best of 7 series. Then we find out that AI doesn't ever build a specific unit. That would be a pretty…

You wouldn't be the first person to say this. But it only proves that the unit isn't useful __ as piloted by AlphaStar __.

Based on the initial AlphaStar against TLO/Mana, you would think that Stalkers are insanely OP and the only thing worth building.

I (and others) have wondered that if you continue to lower AlphaStar's APM, you would see a diversity of units. The money would be in where AlphaStar decides to spend its really tight APM budget. Is it worth casting that Psionic Storm?

I personally feel like it has an insane micro advantage by being able to select arbitrary units on the battlefield, as opposed to dragged squares or control groups. But I'm not a pro gamer, so I don't know what that feels like.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#147

Post by mattnewton »

Mirioron wrote: post #137
ionforce wrote: post #117

Earlier quoted context omitted.

> But most surprising is that it didn't make any "breathtaking" moves or actions, as opposed to AlphaGO. I don't play any competitive StarCraft so my view might be moot but I was surprised at the number of siege tanks it uses. It made me wonder if there's some critical advantage to having so many tanks stacked up in a line so that the splash is spread out. Also I find it weird that it did not build any marauders at a…

> Also I find it weird that it did not build any marauders at all, in any of the games I've seen. This is off-topic, but I was thinking about this the other day - couldn't AI be used to balance a competitive game in this sense? Imagine that the AI becomes so good that human players very rarely win against the AI in a best of 7 series. Then we find out that AI doesn't ever build a specific unit. That would be a pretty…

You can also do this directly with metrics on human games. It’s something that I know wizards of the coast does with Magic the Gathering decklists, just counts the copies of cards and keeps track of the “conversion rate” of them (how likely is this deck to pass some threshold, like have a positive win percentage, given that it has this card). They are doing it primarily to spot ban targets (the paper aspect of cards means you can buff without printing a new set of cards), but it is the same idea.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#148

Post by UserIsUnused »

bsaul wrote: post #58

Has anyone read the actual paper ? This summary really makes it look like "mission accomplished", but this was much much more interesting than that. We saw AI do "obviously stupid things", and we also saw them improve a lot in the middle of the trial, as many youtubers showed. AI was also much more interesting when playing the protoss race, and really felt like it was responding to the opponents actions, and the othe…

Interesting.

I watched Dota2 first OpenAI 5v5 show at TI8, and they did had different actions, like which spells used to initiated, how some spells are used, etc.

But tht AI was very limited in Hero pool.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#149

Post by tryptophan »

300bps wrote: post #136
minblaster wrote: post #81

Earlier quoted context omitted.

Starcraft is an incredibly complex game. 10^26 possible moves at any point (you can click/drag anywhere on the screen, pressing a keyboard button as you do so), imperfect information, real-time constraints, etc.

Starcraft is an incredibly complex game. Even basic strategies will win if they’re done faster. APM (actions per minute) is a very significant factor into who is winning. Apparently they limited their AI player to 264 APM but that’s still incredibly high and done with machine level consistency. That’s almost 4.5 actions per second!! I know there are human level players at and probably above that level but that really…

And the bot also can parse the entire screen in .03 seconds and then jump to a new area of the map. No human can monitor the entire map like the bot can.
Post reply on HN