Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Grandmaster level in StarCraft II using multi-agent reinforcement learning

deepmind.com

Post reply on HN
Print view View original

181–190 of 321 posts

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#181

Post by evandijk70 »

Mirioron wrote: post #132
baddox wrote: post #70

Earlier quoted context omitted.

> The speed at which AlphaStar overtook previous AIs seems to me to be nearly unprecedented in AI research. Is it not simply the case that, before AlphaStar, very little money and effort was being put into developing AIs for Starcraft 2?

What makes this amazing isn't specific to StarCraft 2. AI in strategy games has been really lackluster. I can't think of a single example of a strategy game where an AI was competitive against experienced players due to strategy and tactics, rather than inhuman speed, accuracy or cheating. So it's not just about AI in StarCraft 2, but rather AI in essentially any (strategy) game. Now we have an example of an AI that…

I am not sure what you mean by 'strategy' here, don't Chess, Go, and No-Limit Hold'em fall under those categories?

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#182

Post by gpm »

Razengan wrote: post #82

1. Create a realistic military combat game with AI that learns from thousands of players and builds on that knowledge across multiple games/matches. 2. Load that AI into real-world Terminators.

3. Discover that your realistic military combat game neglected the effects of the sun heating your robots and that they now consistently overshoot targets during the day and undershoot targets during the night.

(And the million other complexities that exist in the real world).

Fundamentally I think the biggest advance in AI for real world applications (military and otherwise), is one that can learn quickly enough that we can train it in the real world situations.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#183

Post by heywot »

SketchySeaBeast wrote: post #173
pps wrote: post #154

Earlier quoted context omitted.

264 is not that high. People (if Serral is not a cyborg) can achieve even 1000 APM; and ~300 EPM average for a game.

Those aren't meaningful actions, though, are they? That's 17 actions a second.

Yes and no. There are many repeat actions in SC but at peak they can definitely reach that high. Microing units while simultaneously performing larger macro actions at the same time is a staple of any decent player and is really obvious at the higher levels. I personally play on an average of 100-120 APM for the game but regularly see 200+ in Platinum/Diamond.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#184

Post by jfoutz »

the_af wrote: post #128
PeterisP wrote: post #107

Earlier quoted context omitted.

In strategy games like Civilization the "preferred" difficulty level is that hard/easy to beat because it gets extra resources. It would be preferrable to have the same difficulty level through opponents that play better/smarter while having the same game mechanic consequences as players if they make the same actions, but we currently can't, so they get artificial production multipliers and such.

Yes, I understand this. I guess I disagree better/smarter would be better, because in videogames what matters is the illusion of challenge, not a real challenge. So spending resources into developing a real AI for Civilization is probably not the best idea; as long as it tricks casual players into believing it's putting up a fight, that's good enough.

I mostly agree with you - there is a little bit of a fine line though. I can learn from my opponents strategy playing against someone a little bit better than me. Maybe i lost because i need more or less of X? Tough to learn from when the opponent is creating the illusion of playing well, rather than playing well.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#185

Post by username90 »

evandijk70 wrote: post #181
Mirioron wrote: post #132

Earlier quoted context omitted.

What makes this amazing isn't specific to StarCraft 2. AI in strategy games has been really lackluster. I can't think of a single example of a strategy game where an AI was competitive against experienced players due to strategy and tactics, rather than inhuman speed, accuracy or cheating. So it's not just about AI in StarCraft 2, but rather AI in essentially any (strategy) game. Now we have an example of an AI that…

I am not sure what you mean by 'strategy' here, don't Chess, Go, and No-Limit Hold'em fall under those categories?

They don't really count as the computer game genre "strategy games". Computer strategy games tend to have many orders of magnitude more complex rules and state than Chess, Go or poker, so the game-play is very different and making an AI for them is also very different.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#186

Post by magnamerc »

bakuninsbart wrote: post #78
ropman76 wrote: post #27

Earlier quoted context omitted.

Or even a turn based game, where click speed does not matter...

If Google could tackle Civilizations god-aweful AI next, I'd be over the moon.

I would pay a lot of money to be able to play a human level Emperor+ level AI on civ

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#187

Post by gambler »

From the paper:

>Humans play StarCraftthrough a screen that displays only part of the map along with a high-level view of the entire map, to e.g. avoid information overload. The agent interacts with the game through a similar camera-like interface

What exactly does that mean? Does it or does it not play by operating purely on image data human players would see on the screen?

How much of the system's interaction with game's interface is learned as opposed to hand-crafred and filtered through APIs?

It's amazing that most people here seem to think that system's ranking in a computer game are more important than its ability to learn from and interact with unstructured data.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#188

Post by redkoala »

gambler wrote: post #187

From the paper: >Humans play StarCraftthrough a screen that displays only part of the map along with a high-level view of the entire map, to e.g. avoid information overload. The agent interacts with the game through a similar camera-like interface What exactly does that mean? Does it or does it not play by operating purely on image data human players would see on the screen? How much of the system's interaction with…

It's meant to operate purely on data viewable on the screen to mimic a human player's experience and provide an even playing ground.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#189

Post by kyberias »

gambler wrote: post #187

From the paper: >Humans play StarCraftthrough a screen that displays only part of the map along with a high-level view of the entire map, to e.g. avoid information overload. The agent interacts with the game through a similar camera-like interface What exactly does that mean? Does it or does it not play by operating purely on image data human players would see on the screen? How much of the system's interaction with…

Purely on image data.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#190

Post by dmreedy »

jrx wrote: post #159
interblag wrote: post #59

This is a really interesting one to digest. As with previous announcements about AlphaStar, much of the feedback (here and elsewhere) is about the fundamental challenge of assessing human vs. machine in an RTS. These points are very valid - stepping back however, this still feels like a pretty incredible accomplishment. I'm a gold league SC2 player, so maybe in the 30th-50th percentile. Three years ago, when DeepMind…

> This is like if the world's best chess AI had gone from losing high school tournaments to being competitive with Kasparov in less than 3 years. I don't think it's like that at all. On the high level, there is no "chess AI", "go AI", "image classification AI" and "dexterous manipulation AI". These are all sides of the same coin, that gets significantly better every year. Adding support for the new game or new "envir…

> On the high level, there is no "chess AI", "go AI", "image classification AI" and "dexterous manipulation AI". These are all sides of the same coin, that gets significantly better every year.

On a practical level, this is not true. There are different algorithms, different architectures, different hyperparameters required for each of these problems, and often for each subdomain within each of these problems, and often for each specific instance of these problems. It's difficult to draw any kind of holistic picture that combines all of the individual advances in each of these problem instances; that's why progress in AI is so hard to measure, and why a statement like "each of these toy problems...brings us closer and closer to solving the 'real problems'" is probably a bit too coarse-grained to be fair as well.

Post reply on HN