Live data from Hacker News

AlphaStar: Mastering the Real-Time Strategy Game StarCraft II

deepmind.com

221–230 of 459 posts

Re: AlphaStar: Mastering the Real-Time Strategy Game StarCraft II

#221

Impressive, but there's a couple of things I'd like to see them try one day. Plug AlphaStar into a robot that physically interacts with a keyboard and mouse to control the game. This "robot" should only have what's relevant for playing the game and emulates a human i.e. a camera that looks at a screen (this is the only knowledge it has of the game), and two arms & hands with five digits that control the mouse and key…

Human pro gamers have sevaral advantages that mean they should need slightly (but probably not that much) less game time.

Transfer learning from the rest of life, games are designed to be understandable to humans with familiar concepts, that AIs don't start knowing.

Discussion with other players. Mana benefits immensely from every else's 20k hours of SC2 as well.

Selection bias, there are many many people who try SC2, only the people who are naturally good at it succeed. So in some sense we need to be counting the rejects training hours as well.

I would like to see advances on training AI using less data. I just wanted to comment that the comparison in number of hours isn't quite fair.

Re: AlphaStar: Mastering the Real-Time Strategy Game StarCraft II

#222
post #56

Earlier quoted context omitted.

In the showmatched they made the computer have to look at a regular screen to control, the stalker micro was much less impressive - and mana won.

For now. Give them another month. This is like AlphaGo vs Fan Hui all over again -- people knocked that accomplishment at the time because he was just a master, not one of the top players in the world. Well, not much longer, AlphaGo beat Lee Sedol, the best player in the world. The ceiling here is going to be incredibly high, much higher than the level of play that people are capable of, even when restricted to a sin…

This doesn't nullify the observations that people are making here.

Part of the difficulty here is describing what a 'fair' match might be. Specifically, I think fairness has to do with a goal many people have for AI: to improve human play. The strategies in Chess or Go that were employed could conceivably be used by human players. There aren't any hard restrictions preventing humans from learning from that play, even if the AI is entirely superior.

It would follow that a 'fair' SCII match would employ strategies that humans could implement. Making extra workers, for instance, might be a real lesson from AlphaStar play. The insane stalker micro, however, could never be done by a human.

From this perspective, I think the important takeaways were:

* The AI leaned heavily on super-human stalker micro.

* The AI had some strategic blind-spots, namely the immortal harass.

* The APM comparison isn't terribly meaningful; a lot of human APM is spammy/twitchy button presses that doesn't do all that much, whereas the AI can presumably make each action count. There were also AlphaStar APM spikes that likely go along with the stalker-micro issue.

Re: AlphaStar: Mastering the Real-Time Strategy Game StarCraft II

#223
post #205
post #146

Earlier quoted context omitted.

Another way to think about it is that a human brain is mostly doing transfer-learning, on top of a 99%-baked deep net that was wired up during foetal development from our DNA, where that DNA-persisted model has "seen" hundreds of millions of years of training data. Humans don't have to learn to process, recognize, and classify objects in visual sense-data, for example. We can do that from the moment we're born, becau…

Perhaps a nit, but still fascinating: the human visual cortex finishes developing after birth. A newborn can't really distinguish between objects. The ability to differentiate, focus on and track objects is developed over the course of several months.

True. Humans are pretty unique in that regard, though; pretty much no other animal is like that. It's easier to understand human neonatal development if you just considering all humans to be born premature. (It'd be really interesting to know whether that's literally true—whether keeping a human baby in the womb for an extra few months would actually result in the same stages of mental development being passed that occur in a regular baby of that age who has been sensing and interacting with the world.)

Re: AlphaStar: Mastering the Real-Time Strategy Game StarCraft II

#224
post #23

Does anyone know if this opens the door to using these techniques for poker (given that they've now show success on games of imperfect information)? Thus far the solutions to poker have involved solving the game tree through raw computational power and clever methods of information collapsing: http://science.sciencemag.org/content/347/6218/145 But it seems the techniques used here might be both far more efficient, as…

So, your link is about Cepheus. It's important to understand that Cepheus isn't AI, it's an (asymptotically close to) optimal strategy for Limit Heads Up ("Limit" means you don't need to choose bet sizes, they are fixed, which makes the problem much simpler), and since poker is a game of probabilities the strategy is probabilistic too. ie this is like when somebody explains Tic-Tac-Toe, there isn't anything interesti…

Yes, I understood that.

I was essentially asking if the techniques used by DeepMind could be leveraged to create a much more powerful version of AI's like Libratus, or to become very strong at games too big to solve for GTO solutions, such as full ring.

> For "Full Ring", the game of Poker as you've probably seen it played, which has more than half a dozen independent players, AI would be very challenging

This has already been done. There was an AI called Sonia which played both HU and full ring at expert human level or beyond, and was not based on GTO solutions. It created a model of every player which updated in real time, and exploited them the way expert humans do.

I'm just curious if DeepMind would be able to achieve similar or better results.

Re: AlphaStar: Mastering the Real-Time Strategy Game StarCraft II

#225
post #9

I would really like to see this for Age of Empires II. I think AOE has far more races and is a far more complex game ( although I'm biased because I haven't played SC2 as much as AOEII ).

I have played both games and a fan of both. Starcraft is definitely more complex than AoE for AI development and that's why the researchers must have chosen it. The complexity of AI depends on how many potential decisions you can make at any point of time. Here are a few reasons why: 1) Starcraft races have completely different build trees and different advantages. This has a large cascading effect of early decisions…

AoE maps are not flat and units have an attack bonus when uphill (and defense penalty when downhill). Top players will place castles on hills for example and micro their units so that they are more elevated than their opponents.

AoE has monks which can take control of enemy units.

The fact that Starcraft has more micro potential should make AI development easier, not harder. Micro management is a relatively mechanic task that is time consuming for humans but which a computer should excel at.

Re: AlphaStar: Mastering the Real-Time Strategy Game StarCraft II

#226
post #221

Impressive, but there's a couple of things I'd like to see them try one day. Plug AlphaStar into a robot that physically interacts with a keyboard and mouse to control the game. This "robot" should only have what's relevant for playing the game and emulates a human i.e. a camera that looks at a screen (this is the only knowledge it has of the game), and two arms & hands with five digits that control the mouse and key…

Human pro gamers have sevaral advantages that mean they should need slightly (but probably not that much) less game time. Transfer learning from the rest of life, games are designed to be understandable to humans with familiar concepts, that AIs don't start knowing. Discussion with other players. Mana benefits immensely from every else's 20k hours of SC2 as well. Selection bias, there are many many people who try SC2…

Those are really good points.

Re: AlphaStar: Mastering the Real-Time Strategy Game StarCraft II

#227
post #164

Earlier quoted context omitted.

This is a widely underappreciated fact when it comes to comes to comparing the 'training experience' of humans versus bots. And it extends far beyond processing 'sense data' - A human likely has some level of understanding of how the game works based on experience from other games it has played and from 'real life' - we know almost instinctively that 'high ground' is likely to give a combat advantage without having t…

All of our knowledge of how to play games and so on has come from our current lifetime. We do not have a "genetic memory" that means we have learnings from cavemen or some other such nonsense. Our DNA contains instructions on how to grow a human, it's not a mega hard drive with millions of years of collective memory. If a 19 year old is good at Starcraft, he's good at Starcraft because he spent two or three years pla…

We may not have "genetic memory" but a ton of human capabilities are baked in at the DNA level. Sure, we need to practice in order to specialise those abilities for particular tasks, but that's more of a calibration phase on a fantastically capable machine, rather than a construction phase.

Totally agree with how impressive humans are, though. In fact, one of the most amazing things to me about robotics is finding out how close to global optimal some humans can actually get.

Re: AlphaStar: Mastering the Real-Time Strategy Game StarCraft II

#228
post #204

Earlier quoted context omitted.

Is a AI that wins at Starcraft only because it has crazy high APM really going to help get to the next X? We could have built that 10 years ago. All it proves is that computers have faster reflexes then humans. That won’t help them become problem solvers for the future.

You seem to forget the way it learned to play every part of the game (not just micro fights). That is, not by having any developer code any rules, but simply by "looking" and "playing". That's the great accomplishment and nothing like that could have been done 10 years ago.

What makes this interesting is if they can make a computer program better at Starcraft strategy then a human. How they did that is irrelevant. If having developers code rules makes a better AI then deep learning, then the former is the most impressive solution. What they did is a great accomplishment and the AI they created was amazing, but I feel like the faster-then-humanly-possible micro makes any accomplishment hollow, because that is really nothing new.

Re: AlphaStar: Mastering the Real-Time Strategy Game StarCraft II

#229
post #23

Does anyone know if this opens the door to using these techniques for poker (given that they've now show success on games of imperfect information)? Thus far the solutions to poker have involved solving the game tree through raw computational power and clever methods of information collapsing: http://science.sciencemag.org/content/347/6218/145 But it seems the techniques used here might be both far more efficient, as…

So, your link is about Cepheus. It's important to understand that Cepheus isn't AI, it's an (asymptotically close to) optimal strategy for Limit Heads Up ("Limit" means you don't need to choose bet sizes, they are fixed, which makes the problem much simpler), and since poker is a game of probabilities the strategy is probabilistic too. ie this is like when somebody explains Tic-Tac-Toe, there isn't anything interesti…

> not least because if humans realise they're at a disadvantage it would be essentially impossible to prevent them from colluding with other humans to get an advantage, even to some extent unconsciously

Indeed versus a single opponent, you can play an "optimal" (e.g. Nash equilibrium strategy) and basically win/solve the game. But in a multiplayer setting (in poker and similar games), you rely on your ability to predict opponents. In the former you could essentially assume your opponent is as smart as possible ('rational' in game theory language), and if it is not rational it will necessarily do worse. In the latter however, by assuming every player is rational (e.g. playing N.E. strategies) you lack the ability to exploit weak players which are decidedly suboptimal, predict their moves, and capitalize. Thus winning is conditioned as much as the knowledge of your opponents, their skill and style, as in your own power to play probabilistic 'optimal' moves.

A good simple example is also Rock-Paper-Scissors: a N.E. strategy just plays randomly -- it clearly cannot lose even against the most skilled players every -- but it also cannot win versus weak, predictable opponents. So e.g. in a tournament setting it would always lose, depending on the tournament structure (in particular if tournament stages are not all pairwise elimination).

I find it a fascinating contrast to traditional game theory, and sort of conventional view of games, that there are "good players" and "bad players" in the sense of their execution being superior in an absolute sense, independent of opponent. In reality success in a variety of games (and real world scenarios) is won by tailoring your strategy to a particular opponent, using information from outside the game, and specifically predicting his plays vs. utilizing universal strategies (again this is particularly relevant in non-pairwise tournaments, non-zero sum games, etc).

In real life, almost always the "game" is non-zero sum, multiplayer, with non-rational players, etc., even in a sports setting. Roger Federer might play slack and save his body in the early stages of a tournament vs a known weaker player, and give it all vs his most fearsome opponents.

And also finally to quote "The Art of Strategy" (introductory game theory classic), 'There's always a bigger game.' (in real life it's the ultimately the entire Universe, and no one is quite sure of the Rules :p )

Re: AlphaStar: Mastering the Real-Time Strategy Game StarCraft II

#230
Many comments here are about how the AI information advantage (seeing the whole map at once sans fog of war except the last game; seeing exact unit stats like health etc) leads to higher APM-value, whether APM itself is higher or each Action is more meaningful, and discussing different ways to nerf it to bring it down to a human level.

I'm more interested in the limits that an AI could be pushed to vs humans, and if humans can't match the AI's APM, just add more humans until they can. E.g. 1v7 would allow humans to manage multiple disparate flanks at once just like an AI, and still leave someone to free to manage macro play etc that suffers when a human focuses on micro.

Post reply on HN