Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Grandmaster level in StarCraft II using multi-agent reinforcement learning

deepmind.com

Post reply on HN
Print view View original

41–50 of 321 posts

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#41

Post by interblag »

plopz wrote: post #29
missosoup wrote: post #20

Earlier quoted context omitted.

I'm not sure this is unfairness. IIRC they put in a fair bit of effort to put it on a level playing field with humans by limiting APM and not allowing it to observe multiple areas simultaneously by spam moving the camera. It might have some minor unfair advantage in terms of being able to click with pixel perfect accuracy, but they're marginal and from watching this project evolve, it's pretty clear that the strategi…

Wasn't blink stalker micro pretty central to its playstyle as protoss, the race it was best at? Why would you say that is minor and marginal?

That was from the version of AlphaStar that they demo'd in the earlier show matches. From what I gathered in the articles over the last few days, they took in a lot of feedback from that and adjusted this to make strategies like that inhuman blink micro less possible...

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#42

Post by jplayer01 »

Honestly, the article needs to be replaced with https://deepmind.com/blog/article/AlphaStar-Grandmaster-leve... which actually goes into some technical detail. Nature.com's article is purely for laypersons and, imo, not particularly useful for HN's crowd because of how little insight it gives.

It also provides the paper and an archive of all of the AI's matches for anybody who wants to take a closer look. These can be viewed with the free version of SC2 (afaik).

Further links:

https://doi.org/10.1038/s41586-019-1724-z (supplementary data available here in json form)

https://rdcu.be/bVI7G (public paper)

https://deepmind.com/research/open-source/alphastar-resource... (replays)

https://www.youtube.com/playlist?list=PLtFBLTxDxWOSrWZ8krQt6... (list of older AlphaStar matches cast by an SC2 player)

https://www.youtube.com/watch?v=l82wBa3UoZU (one of the newer matches cast by another player)

https://old.reddit.com/r/starcraft/comments/dpaunw/deepminds... (win/loss rates across all the played matches by race, includes apm)

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#43

Post by fenwick67 »

scoopertrooper wrote: post #13
Iv wrote: post #10

Earlier quoted context omitted.

A bit further away I would say. So far it trains on a deterministic model by playing millions of games against itself. It can be thrown off by a single rule change. The challenge here in making an AI usable on the real battlefield is more on making a realistic simulation rather than on a better AI.

So, what I'm hearing here is that we should start a series of endless wars?

And now we know the plot for Metal Gear Solid 6

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#44

Post by rantwasp »

tomcooks wrote: post #9
dmichulke wrote: post #7

Earlier quoted context omitted.

I assume obtaining a decent-sized training dataset is gonna be a bloody business.

for starters you just need a gps antenna on each soldier, an ammo count and a health meter (healthy, hit, safe). these 3 pieces of info already allow you to create a strategy framework

yes. and the soldiers need to do exactly what you tell them. good luck

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#45

Post by username90 »

plopz wrote: post #29
missosoup wrote: post #20

Earlier quoted context omitted.

I'm not sure this is unfairness. IIRC they put in a fair bit of effort to put it on a level playing field with humans by limiting APM and not allowing it to observe multiple areas simultaneously by spam moving the camera. It might have some minor unfair advantage in terms of being able to click with pixel perfect accuracy, but they're marginal and from watching this project evolve, it's pretty clear that the strategi…

Wasn't blink stalker micro pretty central to its playstyle as protoss, the race it was best at? Why would you say that is minor and marginal?

It can no longer do micro like that, I have watched most of its games several times and it never did any micro feat like that again. On the contrary its micro was often lacking compared to humans, it won mostly through very strong timing attacks and uncanny ability to pick when to fight.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#47

Post by diggan »

Nimitz14 wrote: post #35

The title seems to contradict the subtitle: > Google AI beats top human players at strategy game StarCraft II vs > DeepMind’s AlphaStar beat all but the very best humans at the fast-paced sci-fi video game.

Yeah, subtitle seems much better. My impression before reading was that it did beat the top players but reading the articles makes it clear it beats everyone BUT the top players...

Also, this part seems a bit weird from the article:

> The AI wasn’t able to beat the best player in the world, as AIs have in chess and Go, but DeepMind considers its benchmark met, and says it has completed the StarCraft II challenge.

So they didn't manage to beat the best players but consider the challenge complete anyways? I thought the goal was to build something that could things better than humans.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#48

Post by neogodless »

v4dok wrote: post #33
Iv wrote: post #12

Earlier quoted context omitted.

To play against alphastar, you have to opt-in. IIRC it states Alphastar will hide itself. One of the things people noticed in replays was the lack of control groups and in the case of zergs, the ability to select larvas directly, which no player ever does. It could have been as simple as removing these quirks.

Nobody selects larvas directly? I guess I am a computer then. Way worse than Alphastar tho.

Ha! So I always did that in Brood Wars, but in SC2, I put each hatchery on a hotkey (0, 9, 8, etc.) and then just pick the units I want to queue up. No need to select individual larva any more.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#49

Post by floitsch »

mieseratte wrote: post #30
floitsch wrote: post #16

Earlier quoted context omitted.

These extremely difficult/impossible things didn't really give an advantage. For example, AlphaStar would sometimes click on an object at the border of the screen. For humans that would be almost impossible, because the screen would scroll when the mouse approaches the border. Similarly, AlphaStar would not play with group hotkeys, but use a different technique. However, in none of the analyses, people noticed things…

> For humans that would be almost impossible, because the screen would scroll when the mouse approaches the border. That humans cannot reliably perform these actions because of the limitations of our corporeal form means that Alphastar has an advantage over a human player. Limiting APM isn't enough.

The question isn't whether the AI still has advantages, but whether it's an unfair one. The current restrictions were implemented with input from TLO, and seem to work quite nicely.

One could, of course, add more restrictions (like not noticing things on the minimap all the time...), but that's not what's it about anymore. At this point, micro is comparable to humans, and we can start to compare macro and strategy.

Fwiw, when I watched some of the replays, I was disappointed by AlphaStar. It's a very consistent player with few mistakes, but it isn't very reactive and definitely not inventive. Instead of switching tactics when things don't work out, it generally continues with the chosen strategy. Often, that's enough: a well executed strategy with few errors often wins, even if it wasn't optimal.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#50

Post by [deleted] »

nightcracker wrote: post #5

> After 50 games, however, DeepMind hit a snag. Some players had noticed that three user accounts on the Battle.net gaming platform had played the exact same number of StarCraft II games over a similar time frame — the three accounts that AlphaStar was secretly using. When watching replays of these matches, players noticed that the account owner was performing actions that would be extremely difficult, if not impossi…

[deleted]
Post reply on HN