Live data from Hacker News

AlphaStar: Mastering the Real-Time Strategy Game StarCraft II

deepmind.com

231–240 of 459 posts

Re: AlphaStar: Mastering the Real-Time Strategy Game StarCraft II

#231

Earlier quoted context omitted.

Is a AI that wins at Starcraft only because it has crazy high APM really going to help get to the next X? We could have built that 10 years ago. All it proves is that computers have faster reflexes then humans. That won’t help them become problem solvers for the future.

Speed of play is a fundamentally important gameplay mechanic of any real-time game. One of the main reasons the pros are better than amateurs at these types of game is because they play and react faster. And yes, of course computers are much better at doing things more quickly than humans. It's not even remotely close for us. The AIs are clearly better. It's not cheating either; they are legitimately better at it tha…

So all they really proved is computers are faster then humans. I knew that before this started.

The Deepmind team knows the challenge isn’t to beat humans at Starcraft. That is trivially easy with the advantages you mentioned. The challenge is to be better at strategy then a human. That is why they tried to add artificial rules to make the AI have similar physical limitations to a human (emulated mouse, rate limited actions, emulated screen and visibility). There have been micro AI bots for years that could out preform any human. They knew they weren’t just trying to build another micro bot, because if they were it wouldn’t be much of an accomplishment.

Re: AlphaStar: Mastering the Real-Time Strategy Game StarCraft II

#232
What are the games that, so far, still look like they will be too difficult for ML to play them at the highest level? I know Go was held to be this kind of game for a long time and is now close to being dominated by AI. Magic: the gathering perhaps?

Re: AlphaStar: Mastering the Real-Time Strategy Game StarCraft II

#233
post #146

Earlier quoted context omitted.

Another way to think about it is that a human brain is mostly doing transfer-learning, on top of a 99%-baked deep net that was wired up during foetal development from our DNA, where that DNA-persisted model has "seen" hundreds of millions of years of training data. Humans don't have to learn to process, recognize, and classify objects in visual sense-data, for example. We can do that from the moment we're born, becau…

Is there evidence for this? My mental model has been that DNA encodes more along the lines of hyperparameters: amount of gray matter vs white matter, locations of brain regions and folds, etc, but the connections between neurons, and their weights, were all learned. There isn't that much information you can stuff into DNA, after all.

Connections between neurons, the synapses, are encoded. So much so that they are given individual names. This is a fun one to read about to get an idea:

https://en.wikipedia.org/wiki/Calyx_of_Held

Re: AlphaStar: Mastering the Real-Time Strategy Game StarCraft II

#234
post #181
post #150

Earlier quoted context omitted.

Even the 200ms reaction time seemed overly slanted towards the AI. I don't think that is the actual reaction time of top pros, in the matches the AI played the human player would teleport in from complete invisibility and try to use an instant cast spell and the AI would have already teleported out. Yes the theoretically may have been constrained to a 200ms reaction time, but in practice the AI was playing at a super…

Another advantage was that the AI is just reading the game state through an API, it doesn't have to look on the screen. The game can be difficult to watch from a pro's perspective since they have to constantly click around the map to see what's happening, but the AI has perfect knowledge of everything it is capable of seeing, all without having to physically move a mouse to click on the screen.

Is the bit about reading the game through an api true? Earlier iterations of this same rl based agent that played Atari games would read just raw pixels not an api.

Re: AlphaStar: Mastering the Real-Time Strategy Game StarCraft II

#235
post #46

This is really impressive, I didn't expect starcraft to be played this well by a machine learning based AI. I'm excited to read the paper when it comes out! That said, I'm not sure I agree that it was winning mainly due to better decision making. For context, I've been ranked in the top 0.1% of players and beaten pros in Starcraft 2, and also work as a machine learning engineer. The stalker micro in particular looked…

> For context, I've been ranked in the top 0.1% of players and beaten pros in Starcraft 2, and also work as a machine learning engineer.

What's that hireability like?

Re: AlphaStar: Mastering the Real-Time Strategy Game StarCraft II

#236
post #54

Earlier quoted context omitted.

It's extremely impressive, but at the same time it's still an interesting and fair question to say, "can we make an AI that beats humans while playing in a 'human like' fashion?", or alternatively, could we make an AI that would win if we put it inside a human body and made it play through those physical input restrictions? (I do agree though that HN is way too negative overall. Partly it's just because, negative com…

> "can we make an AI that beats humans while playing in a 'human like' fashion?" Yes, now. From the article: “I was impressed to see AlphaStar pull off advanced moves and different strategies across almost every game, using a very human style of gameplay I wouldn’t have expected,” he said. “I’ve realised how much my gameplay relies on forcing mistakes and being able to exploit human reactions, so this has put the gam…

It was performing over 1000 pixel perfect (presumably) and meaningful actions a minute during fights. That's super-human.

Re: AlphaStar: Mastering the Real-Time Strategy Game StarCraft II

#237
post #89

Earlier quoted context omitted.

It is certainly a real advantage, but I think the argument is that it's not as interesting as an AI that could win on the strength of better decision-making, or the innovation of novel strategies, etc.

AI wins on the strength of better decision-making and novel strategies in Chess and Go, though. I have no doubt we'll see this in RTSes in the near future as well. For now we may not be quite there yet, as this is simply the first time it's beaten a pro player in any way. Compare with the AlphaGo match vs Fan Hui. A year later and it was dominant over all pro players.

> AI wins on the strength of better decision-making and novel strategies in Chess and Go, though. I have no doubt we'll see this in RTSes in the near future as well.

Yes, likely! I wasn't doubting it's possible or even likely. Only that seeing an AI do flawless 1000 APM stalker micro and macroing perfectly, while pretty cool, is not as exciting as seeing an AI use a novel strategy (edit: especially one that a human could theoretically execute)

Re: AlphaStar: Mastering the Real-Time Strategy Game StarCraft II

#238
post #220

Earlier quoted context omitted.

That's not how any of this works. We do not have "millions of years" of information encoded into DNA. DNA doesn't store that much data. In fact, it's about 1.6 gigabytes only! And most of that information is basically a ruleset for growing proteins which become our body. All the stuff we've learned about games and so on have come from our current lifetime. I don't have caveman memory for how to fight a tiger.

I said "deep net" for a reason. A DNN model almost always turns out to be far, far smaller than the training data that was used to create it. For one example: any smartphone's face-recognition feature. Each such feature is a DNN which took millions of hours of face data to train... but the resultant model fits on an ASIC. Our DNA doesn't directly encode such a model, but it encodes a particular morphogenic chemical g…

Our DNA is NOT a trained deep net, nor is it a deep net period. Our DNA is a string of proteins which encode other proteins which gives the series of tasks needed to create and operate all the structures of the brain and body.

The "training" of our deep net happens during our lifetime. We are not born with a trained deep net so your analogy that somehow we are born with a highly capable deep-net encoded into 1.6GB of DNA makes no sense.

Can you imagine how capable a human being would be if it was born into a world with no other humans or learning sources? Imagine a new born baby born into a world with some accessible food/water close by so it wouldn't die from lack of nutrition or wild animals, but crucially without any other humans. It would be utterly fucking useless, no language/reading means no way of assimiliating new knowledge. That baby would end up being a totally incapable human, regardless of the DNA or structure of the brain.

As far as we currently understand, if infants aren't exposed to language and communication at a very young age, they are either incapable or severely stunted in terms of communication for the rest of their life.

My point is, that we are very much dependent on the learning that we get from the point of birth ONWARDS. We get the amazing capacity to learn from the structure of our brain and body, but we'd be absolutely incapable idiots without other people to teach us, our books, language etc. We understand "games" and game theory from playing games with other kids, we're not born with "game theory" encoded into our DNA as one other commenter seemed to think, the same for language learning, and everything else.

Anyway, the point of this whole debate was that it's incredibly impressive that humans can learn to play a game as complex as SC2 in a tiny fraction of the time it takes a cluster of GPUs using a huge amount of energy and resources. Not forgetting that we also have to use a physical body to control our actions in the game, which adds a whole other level of complexity since we have to understand how to manipulate a mouse/keyboard etc, whereas the AI is essentially acting directly with the game, like a human with a neural link. The other kicker, is that if you just changed one aspect, like picking a new map neither player had seen, the AI would be sent hurtling back to square one whereas the human would only be partially affected. These series of demos only make me more impressed that given the huge resources given to Google, they can just about beat a human and even then after 200 years of training time and various other artificial advantages.

Re: AlphaStar: Mastering the Real-Time Strategy Game StarCraft II

#239
post #110

Earlier quoted context omitted.

Presumably completely invisible, as it was looking at raw unit stats rather than the visuals.

I wonder if that would let you win with something like mass dark templar with phoenix's to snipe observers. You could run right past it, and it could never anticipate you. Or better yet, imagine zerg where you can burrow every unit.

It would be the same as with a player: as soon as you do something with those invisible units, or imply that you have it (eg dt shrine), its sufficient to say that invisibility is in play, and appropriate tools should be used. Its not like you can do anything about dark templars even if you see the shimmer, if you have no sight, beyond body blocking.

Regardless, the article describes cheesing as the common tactic in early iterations, with economic-play being learned later — one of the described cheeses is dt rushes, which the AI apparently learned to deal with, so it should have some understanding of invisible units (alternatively it learned to ignore the dts and base trade or something).

I don’t think the shimmer is useful enough to be a significant loss for these prospective AI’s quests for world (sc2) domination

Re: AlphaStar: Mastering the Real-Time Strategy Game StarCraft II

#240
post #181

Earlier quoted context omitted.

Another advantage was that the AI is just reading the game state through an API, it doesn't have to look on the screen. The game can be difficult to watch from a pro's perspective since they have to constantly click around the map to see what's happening, but the AI has perfect knowledge of everything it is capable of seeing, all without having to physically move a mouse to click on the screen.

Is the bit about reading the game through an api true? Earlier iterations of this same rl based agent that played Atari games would read just raw pixels not an api.

Any game that is specifically going out of its way to support these ai’s will naturally do it through an api, though I’m only aware of dota2 and sc2 (sc:bw also does, through a community-modified client that serves the api, iirc). For adhoc games, eg atari, pixel-parsing is the natural result, but no one would intentionally set it up like that
Post reply on HN