Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AlphaStar: Mastering the Real-Time Strategy Game StarCraft II

deepmind.com

Post reply on HN
Print view View original

451–459 of 459 posts

Re: AlphaStar: Mastering the Real-Time Strategy Game StarCraft II

#451

Post by erikpukinskis »

En_gr_Student wrote: post #445

And now they are superhuman with regards to "Fog of war"! A truly crushing next step would be to make "World in Flames" into a computer game, and have Alpha-Star-Zero, which is coming, to become the best it can on that. http://www.matrixgames.com/products/296/details/World.In.Fla... If they do that, every single military on the planet should crap themselves, including the US, Nato, Russia, China, and EU. It means tha…

You are comparing formal systems, like Starcraft, where the entire game world is quantized, with the real world where the games are not quantized.

Not only is the game of war not “written down” in digital form, there is far too much data to ever write down. Military strategists have been trying to model war forever. But no one can even agree on what the “game pieces” are, let alone list all of the Deus Ex Machina that might show up. An AI that is better than any human commander at tank warfare would have been just as dead as any human army when the opponent shows up with an H-bomb.

But being good at tank warfare also isn’t anything like being good at a tank game. To be good at tank warfare you need to be good at building factories. Pouring concrete in swamps. Convincing grannies to buy fewer cans of beans, and eat more squash. Figuring out when your workers really need to go home and sleep. Part of the game is just realizing thosethings are even things you might want to think strategize around.

And the nature of competition in real life is that as players gain advantages, opponents copy their skills and those advantages cease to work. Then you have to find new advantages in some aspect of the game that has never been documented before.

In a sense, high level real world competition is more like MAKING games than playing them. It’s about designing a competitive landscape where your opponent won’t be able to design their way out of it. Something AIs haven’t, to my knowledge, even begun to be able to think about.

Re: AlphaStar: Mastering the Real-Time Strategy Game StarCraft II

#452

Post by dmoy »

swaggyBoatswain wrote: post #441
dmoy wrote: post #192

Earlier quoted context omitted.

It is why I stopped playing SC, and I was never any good anyways. Still fun, but it just hurt real bad.

I stopped playing SC competitively because it's too stressful. Both physically and mentally. Hitting 300 APM continously in a game for up to 60 minutes at a time makes your hands go numb. And the adrenaline rush makes you want to go running afterwards. With games like LoL/DoTA at least you have a chance to take a break after a gank/farming/ team wipes. With starcraft every decision has a significantly higher compound…

Hell I never played it competitively. I had to stop playing it even casually because it physically hurt.

Re: AlphaStar: Mastering the Real-Time Strategy Game StarCraft II

#453

Post by notSupplied »

setr wrote: post #442
feanaro wrote: post #343

Earlier quoted context omitted.

That's a good insight and yes, humans would probably be overpowered eventually. However, this is just the consequence of the fact that all games are similar if you remove external limitations such as reaction time (or, alternatively, produce a more efficient "being" which is not as subject to these limitations as some other). Starcraft is like chess in some sense. The largest fundamental difference is that it isn't a…

Tbh starcraft and dota shouldn’t really be the test games atm; turn based strategies (or rather, grand strategies) would be the far more appropriate evolution after chess and go, since we’re clearly more interested in AI macro than micro, and too much of its learning process is in trying to push the AI beyond micro-oriented thinking (probably many rounds of the AI tournament are lost simply because one AI found a new…

Yes but a turn based game drastically reduces the action space compared to a real time game, something the DeepMind folks pointed out as a particularly interesting problem they wanted to tackle.

Re: AlphaStar: Mastering the Real-Time Strategy Game StarCraft II

#455

Post by notSupplied »

rcheu wrote: post #46

This is really impressive, I didn't expect starcraft to be played this well by a machine learning based AI. I'm excited to read the paper when it comes out! That said, I'm not sure I agree that it was winning mainly due to better decision making. For context, I've been ranked in the top 0.1% of players and beaten pros in Starcraft 2, and also work as a machine learning engineer. The stalker micro in particular looked…

It's a question of whether "played with human level latency and precision" be a part of the rules of the game we are making the AI play.

I would say yes, because StarCraft was very clearly balanced for human players. We already saw some indication that when played with super-human micro, mass blink stalkers is a stronger strategy than when humans are in control. Without the active intervention of game balancing, RTS metas tend to devolve into "mass one or two units" which was what happenes to every Command & Conquer game (and why SC is a respected eSport while C&C is not).

I suspect this will happen when you have agents playing parameters that don't match what the game was balanced for. The strategic landscape will shrivel up and the game cease to captivate us.

Re: AlphaStar: Mastering the Real-Time Strategy Game StarCraft II

#456

Post by rcthompson »

ajuc wrote: post #435
rcthompson wrote: post #294

Earlier quoted context omitted.

I don't know if we can absolutely claim that the limited viewport was the deciding factor in the 11th game, but it did seem to me that the Alphastar agent's blink stalker micro was somewhat compromised in that game compared to the seemingly superhuman blink micro in previous games.

You can see the alphastar perspective of that 11th game here: https://youtu.be/H3MCb4W7-kM?t=5195 It struggles with camera placement like real players :) And uses popular divert-attention tactics, which shows it understand that part of the game - for example when it sends oracles to mineral line at the same time as it attacks in front. Previous versions didn't do that, because they were taught playing vs cheating AI…

Yeah, that's actually a huge point that I didn't even consider. Regardless of whether the AI itself is playing with a limited viewport, the fact that its opponent has a limited viewport opens up the opportunity to learn attention diversion tactics during the training process, which would otherwise be impossible.

Re: AlphaStar: Mastering the Real-Time Strategy Game StarCraft II

#457

Post by setr »

notSupplied wrote: post #453
setr wrote: post #442

Earlier quoted context omitted.

Tbh starcraft and dota shouldn’t really be the test games atm; turn based strategies (or rather, grand strategies) would be the far more appropriate evolution after chess and go, since we’re clearly more interested in AI macro than micro, and too much of its learning process is in trying to push the AI beyond micro-oriented thinking (probably many rounds of the AI tournament are lost simply because one AI found a new…

Yes but a turn based game drastically reduces the action space compared to a real time game, something the DeepMind folks pointed out as a particularly interesting problem they wanted to tackle.

>a turn based game drastically reduces the action space compared to a real time game,

That's the primary benefit imo. The bigger action space is largely composed of non-strategic elements, at least in the sense of long-term strategies, eg micro and mini-skirmish tactics, that I don't think are as interesting. Ofc its clearly a conflict of interest, but my feeling was the most interesting aspect of Go/Chess is the AI making unintuitive discoveries that benefit the long-term. The human-collective machine is pretty good on its own at finding the shorter-term strategies; I don't think AI will make much significant impact in that space.

games as a medium to study upcoming real-world applications (eg cars), RTS makes sense; but as a medium to study AI beating humans, TBS is more appropriate (their ability to explore large search-spaces is far more interesting/potentially impactful). Studying both would be ideal ofc, but in a pick-one situation, TBS is better imo. But only RTS are even really viable atm, which is disappointing.

Re: AlphaStar: Mastering the Real-Time Strategy Game StarCraft II

#458

Post by sam_dusk »

celeritascelery wrote: post #228
bsaul wrote: post #204

Earlier quoted context omitted.

You seem to forget the way it learned to play every part of the game (not just micro fights). That is, not by having any developer code any rules, but simply by "looking" and "playing". That's the great accomplishment and nothing like that could have been done 10 years ago.

What makes this interesting is if they can make a computer program better at Starcraft strategy then a human. How they did that is irrelevant. If having developers code rules makes a better AI then deep learning, then the former is the most impressive solution. What they did is a great accomplishment and the AI they created was amazing, but I feel like the faster-then-humanly-possible micro makes any accomplishment h…

If you think the how is irrelevant you are completely missing the point of this exercise. Maybe to you only the result matters but for every other task and humanity the how matters. Simply imagine next taking on a different Game like one version of the Anno series. If developers did it by hand, you need 50 devs sitting there for probably a couple of months, figuring out the best, rules their sequence and putting them in. That is about $20 Million just to get a similar AI for the next game. Compare that to download all available replays, requiring maybe 2-3 data scientist to get the data into shape, renting some compute in the google cloud and you get the same or a better result for probably half a million $.

Watch and learn from data alone is why modern machine learning is considered a revolution and novelty. Buying compute time in the cloud is in comparison (to devs and hand coding) dirt cheap and the results are often better.

Deepmind is not working on this problem for the benefit of gamers or the Starcraft community. Making the perfect bot is not the aim. Tackling the next hurdle, next hardest problem in machine learning is. On the way to become better at generalizing the learning algorithms.

Re: AlphaStar: Mastering the Real-Time Strategy Game StarCraft II

#459

Post by sidusknight »

rcheu wrote: post #46

This is really impressive, I didn't expect starcraft to be played this well by a machine learning based AI. I'm excited to read the paper when it comes out! That said, I'm not sure I agree that it was winning mainly due to better decision making. For context, I've been ranked in the top 0.1% of players and beaten pros in Starcraft 2, and also work as a machine learning engineer. The stalker micro in particular looked…

What was your SC2 alias? I played at a similar level as you.
Post reply on HN