Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Grandmaster level in StarCraft II using multi-agent reinforcement learning

deepmind.com

Post reply on HN
Print view View original

271–280 of 321 posts

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#271

Post by ionforce »

callmekit wrote: post #270
ionforce wrote: post #231

Earlier quoted context omitted.

Everything about AlphaStar is typed and discrete. It has perfect inputs because it uses an API (and does not read pixel data). Human limitations that AlphaStar shares: - Data that requires the camera to see (e.g. enemy location, enemy HP) - Inability to examine/target cloaked units Possibly unfair, super-human things AlphaStar has access to: - Instantaneous awareness of cloaked units - Knowledge of things humans need…

Do you know if it can target a particular unit from a clump of air units? If yes, then there is another "unfair" thing. Also I wonder how their "camera-like interface" works with tactics like fly a building above units to make them harder to target.

Yeah. I think watching it split a line of Stalkers into two flanks that isn't done with a rectangular selection is crazy unfair.

I think you're saying does it suffer from occlusion during selection. Based on how APIs typically work, I would say no it does not. So yeah that's another thing humans can't do. AlphaStar could hypothetically stack units into a singular mass and it would be discretely untargetable by human players.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#272

Post by callmekit »

kyberias wrote: post #189
gambler wrote: post #187

From the paper: >Humans play StarCraftthrough a screen that displays only part of the map along with a high-level view of the entire map, to e.g. avoid information overload. The agent interacts with the game through a similar camera-like interface What exactly does that mean? Does it or does it not play by operating purely on image data human players would see on the screen? How much of the system's interaction with…

Purely on image data.

From the article, they have a special interface, "developed in consultation with professional StarCraft II players and Blizzard employees". They are a bit vague on the details, but, for example, "Agents can also select sets of units anywhere, which humans can do less flexibly using control groups".

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#273

Post by chillacy »

jcampbell1 wrote: post #145
Jyaif wrote: post #92

Earlier quoted context omitted.

> Actually not a single game made pro player realize something new about the game. People are now over saturating their mineral line (making more probes than before), so I don't think that's true.

There were other cases of this such as 3 rather than 2 early air units. The AI seemed to anticipate losses in a way that is almost more rational than humans.

I remember it also built like 8 observers in one match, supposedly because the AI was so good at sniping them (and it trained on itself).

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#274

Post by chillacy »

perseusmandate wrote: post #249
vkou wrote: post #223

Earlier quoted context omitted.

Why? Is it because they expect to lose workers to harassment? Or is it so that they can saturate new expansions quicker?

In terms of total mineral output oversaturating would seem to be disadvantageous since being able to get more workers on the new mineral line is offset by the cost of not expanding sooner and then being able to build from both bases. I'm interested to understand why Alpha did this since worker production seems like one of the most solved and optimized parts of the games and not where you'd expect innovation

Most people speculate it's to account for losses from harassment where it somehow finds easier to overprobe than bother with more defense.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#275

Post by chillacy »

sgillen wrote: post #197

I see a lot of comments downplaying this achievement, saying it's not impressive or it won't be impressive until X condition is met. I welcome skepticism and criticism for this sort of thing, and think most of it that I've seen here is well founded. But I would like to take a second to explain why I think this, and really all the progress in this area, is actually a really impressive achievement to me. Let me try and…

I don't think the AI learns from pixels (nor does it move through mouse/kb commands), it has direct API access to the game. Nonetheless it is impressive.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#277

Post by chillacy »

king-rat wrote: post #79
jplayer01 wrote: post #69

Earlier quoted context omitted.

> Not being robust to strategies it hasn't seen before is a serious shortcoming in a real time strategy game. I'm not sure if you've actually played or followed competitive SC2. This is absolutely normal . Players will pull something completely unexpected out of a hat and win. The losing player will learn from it in future games. That's just how it goes. Unexpected strategies are really hard to counter when you've ne…

I do actually follow competitive SC2. My point is that for DeepMind to master this game in the same way it has with Go and chess, it must be able to anticipate counter plays ad infinitum. Otherwise humans will continue to beat it.

Are new strategies more important in SC2 than go/chess? From the blog it seems like the AI learns from self-play so if it doesn't come up with a strategy, it won't know how to counter it. But that seems like it should apply to chess and go as well.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#278

Post by chillacy »

yters wrote: post #105

What would be interesting is to limit the AI processing speed to human capacity, which is something like 60 bits per second. In all these AI v. Human games I see, it is really apples to oranges because the human consumes vastly less resources and compute cycles to perform at the same level as the AI. And when I say 'vast' I mean Vast. There is like a quintillion factor difference between the AI and the human. There i…

I think we should just accept that humans and computers will be good at different things. Otherwise when AlphaStar 5 walks on stage and smiles at the camera we'll be criticizing its use of fuel cells instead of digesting food.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#279

Post by freddie_mercury »

the_af wrote: post #241
Mirioron wrote: post #214

Earlier quoted context omitted.

> There is a genuine advance here, but keep in mind that when an AI is developed by the game developers, they're not necessarily playing to win, but to make the AI fun to beat, and without using too much computer power, which would make the game slower. This point is being brought up a lot, but I don't really buy it. Yes, there have been instances where the AI being too good discouraged players from playing the game…

I think part of the problem is that every videogame is a parlor trick in a sense. Unless you are a tournament player or someone who enjoys a really hard challenge, the bulk of the customers who buy the game just want to enjoy the illusion of a challenge. You don't really need a complex AI directing the hordes of enemies you want to defeat; you just want the illusion of battling enemies and emerging the victor, and ho…

To extend your point, it is also clear from analytics in Steam & mobile gaming that games have massive amounts of player dropout. It is pretty rare, even in relatively short single player campaigns, to see 30% of people who bought the game finish it. So effort making the AI even better is always going to only be for an extremely small niche of players that play a game long enough to exhaust the limits of simple AI.

Just look at the achievements for Left for Dead. Only 40% of all players have got the achievement for "kill one of each Uncommon type" which is trivial to get if you play for a few hours.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#280

Post by baddox »

Mirioron wrote: post #132
baddox wrote: post #70

Earlier quoted context omitted.

> The speed at which AlphaStar overtook previous AIs seems to me to be nearly unprecedented in AI research. Is it not simply the case that, before AlphaStar, very little money and effort was being put into developing AIs for Starcraft 2?

What makes this amazing isn't specific to StarCraft 2. AI in strategy games has been really lackluster. I can't think of a single example of a strategy game where an AI was competitive against experienced players due to strategy and tactics, rather than inhuman speed, accuracy or cheating. So it's not just about AI in StarCraft 2, but rather AI in essentially any (strategy) game. Now we have an example of an AI that…

Is this AI better due to strategy and tactics? I am a low-level player but a fairly consistent spectacle, and in the videos I have seen I didn’t spot anything that looks like superior strategy or tactics. Just insanely good mechanics (which is still awesome!)
Post reply on HN