Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Grandmaster level in StarCraft II using multi-agent reinforcement learning

deepmind.com

Post reply on HN
Print view View original

241–250 of 321 posts

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#241

Post by the_af »

Mirioron wrote: post #214
skybrian wrote: post #168

Earlier quoted context omitted.

There is a genuine advance here, but keep in mind that when an AI is developed by the game developers, they're not necessarily playing to win, but to make the AI fun to beat, and without using too much computer power, which would make the game slower. Also, due to commercial pressures it's tough to put a lot of effort into the AI for a game that's still being changed to make it more fun. Even now, I wouldn't really e…

> There is a genuine advance here, but keep in mind that when an AI is developed by the game developers, they're not necessarily playing to win, but to make the AI fun to beat, and without using too much computer power, which would make the game slower. This point is being brought up a lot, but I don't really buy it. Yes, there have been instances where the AI being too good discouraged players from playing the game…

I think part of the problem is that every videogame is a parlor trick in a sense. Unless you are a tournament player or someone who enjoys a really hard challenge, the bulk of the customers who buy the game just want to enjoy the illusion of a challenge. You don't really need a complex AI directing the hordes of enemies you want to defeat; you just want the illusion of battling enemies and emerging the victor, and how the game accomplishes this is not that relevant... unless the trick is so blatant that it breaks the suspension of disbelief, of course.

Why does this matter? Because the trick is often less effort than the real deal. The point of videogames is to entertain you, and the goal of game publishers is to turn a profit while doing so. Obviously there are some exceptions, and indie/free games might have different goals (we all know indie games that can be punishing, like Dwarf Fortress).

I remember with the original Left 4 Dead by Valve, they claimed there was an "AI Director" that modified the map according to how well you were doing, "adapting" to how you were playing. In practice it amounted to closing some sections of the map according to how easy you were killing zombies without dying. So this wasn't really AI but a glorified IF-THEN-ELSE trick, but what does it matter? It served its purpose. Do we really need something more complex (and expensive!) when killing zombies in L4D?

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#242

Post by Mikeb85 »

nullbyte wrote: post #152
Mikeb85 wrote: post #91

As impressive as this is, APM matters a lot in an RTS such as Starcraft and computers have a massive advantage here. I'd like to see them tackle an RTS like Civ where APM doesn't matter whatsoever, only decision making.

Interestingly, AlphaStar has very low APM compared to human players. This is because humans do a lot of insignificant actions when button mashing, whereas AlphaStar is extremely precise with its moves. That's why when you compare APM graphs between Alpha and a human player, the human almost always ranks higher in actions-per-minute.

It's still a high APM game. Reducing redundant button presses still doesn't change the fact that APM is very significant and that being able to process information and act quickly is more important than pure strategy.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#244

Post by friendlybus »

papreclip wrote: post #157
bsaul wrote: post #58

Has anyone read the actual paper ? This summary really makes it look like "mission accomplished", but this was much much more interesting than that. We saw AI do "obviously stupid things", and we also saw them improve a lot in the middle of the trial, as many youtubers showed. AI was also much more interesting when playing the protoss race, and really felt like it was responding to the opponents actions, and the othe…

IMO it's a testament to how games like SC are collectively and thoroughly "solved" by the community, and how the games aren't that complex after all. I never followed SC or SC2, but my observation of the pro scene and competitive ladder for Warcraft 3 was that cookie cutter strats dominated. Pro players were typically those who executed best, not those who innovated best. Personally I felt disappointed by the fact th…

Cookie cutter strats or the metagame changes over time. Knowing what strategies you are likely to face influences what strategy you are likely to try. The meta is different across regions. The europeans play differently to the americans who play differently to the koreans. Some people dedicate themselves to different playstyles and your 'cookie cutter' strat will have to grow with everyone else's. They patch the game often to update the balance.

Designing a game that is a fun and balanced experience whilst also encouraging true innovation is a difficult problem. Every game ever has this problem, putting thousands of minds on a problem and letting them confer about their results usually solves all the low-hanging fruit in about a week. Creating a game where you only play to share a new idea makes each match feel like a dice roll.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#245

Post by Ruphin »

scott_s wrote: post #240
antepodius wrote: post #238

Earlier quoted context omitted.

A lot. In the same way that errors reduce data bandwidth, misclicks reduce effective APM in hectic situations. Classic examples are moving marines into banelings or units into disruptor shots.

Sorry, I asked that wrong. Clearly misclicks matter at the pro level, as pros are going to exploit their competitors mistakes ruthlessly. What I should have asked is, at the pro level, how often do misclicks affect the outcome of a match?

Every game of Starcraft is decided by an extremely large amount of small events. It is very rare that a single event or misclick is responsible for the outcome of a game, but it happens occasionally.

Since players take so many actions during the course of the game (well into the tens of thousands), inevitably some clicks will be sub-optimal, and they all have a tiny impact on the outcome. Some professional players do specific exercises to improve their clicking accuracy in order to gain efficiency by reducing misclicks, but generally clicking accuracy is not considered a big factor compared to raw speed. Most players try to attain the highest possible clicking speed while maintaining an accuracy level that is "good enough".

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#246

Post by friendlybus »

CydeWeys wrote: post #116
plopz wrote: post #3

> placing within the top 0.15% of the region's 90,000 players > 61 wins out of 90 games against high-ranking players This doesn't seem to be quite as commanding as it was in Go. Do we know what MMR it reached or if it consistently beat players like Serral?

It seems like it's only a matter of time though. StarCraft II also has a rock-paper-scissors nature to it though, so you wouldn't expect even a perfect player to win 100% of the time. There are some strategies that are hard-counters to other strategies, and because of the imperfect information nature of the game, by the time you scout your opponent and see what they're doing, it may be too late to shift and deal with…

There aren't really 'hard' counters, there's just a high chance you'll win.

Flash had a 70% winrate overall and had time periods of 90%+ winrate.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#247

Post by friendlybus »

Razengan wrote: post #82

1. Create a realistic military combat game with AI that learns from thousands of players and builds on that knowledge across multiple games/matches. 2. Load that AI into real-world Terminators.

3. Discover that a kid holding a fork can walk up to your terminator due to RoE and disconnect it's battery pack.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#248

Post by TaupeRanger »

Nimitz14 wrote: post #83
diggan wrote: post #47

Earlier quoted context omitted.

Yeah, subtitle seems much better. My impression before reading was that it did beat the top players but reading the articles makes it clear it beats everyone BUT the top players... Also, this part seems a bit weird from the article: > The AI wasn’t able to beat the best player in the world, as AIs have in chess and Go, but DeepMind considers its benchmark met, and says it has completed the StarCraft II challenge. So…

They're framing it positively to distract from the fact they can't do it, the reality is they've spent millions of $ on compute and yet their agent is still terrible at strategy (and tactics sometimes too), they probably decided it's best to stop now before sinking even more money into it.

You're getting downvoted for tone probably, but I think you're right overall. They simply can't "win" this game in the same way they "won" Go.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#249

Post by perseusmandate »

vkou wrote: post #223
Jyaif wrote: post #92

Earlier quoted context omitted.

> Actually not a single game made pro player realize something new about the game. People are now over saturating their mineral line (making more probes than before), so I don't think that's true.

Why? Is it because they expect to lose workers to harassment? Or is it so that they can saturate new expansions quicker?

In terms of total mineral output oversaturating would seem to be disadvantageous since being able to get more workers on the new mineral line is offset by the cost of not expanding sooner and then being able to build from both bases.

I'm interested to understand why Alpha did this since worker production seems like one of the most solved and optimized parts of the games and not where you'd expect innovation

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#250

Post by kllrnohj »

doc_gunthrop wrote: post #109
username90 wrote: post #45

Earlier quoted context omitted.

It can no longer do micro like that, I have watched most of its games several times and it never did any micro feat like that again. On the contrary its micro was often lacking compared to humans, it won mostly through very strong timing attacks and uncanny ability to pick when to fight.

> It can no longer do micro like that Is that supposed to be an intention in design? I'd figure an AI could easily outplay a human if that weren't the case, given its inherent advantages (e.g. better accuracy, the ability to instantaneously prioritize which units to blink in/out). For example, [this]( https://youtu.be/pETcAm82vXU?t=322 ) game is one example where an AI could perform even better using such tactics. ED…

Just as an example of how extremely unfair it would end up being, flawless AI micro looks like this: https://www.youtube.com/watch?v=IKVFZ28ybQs

100 zerglings vs. 20 siege tanks. Without insane micro the zerglings barely kill 2 siege tanks. With insane micro the 100 zerglings mop up the whole army with ease.

It's fascinating & fun to watch, but if your goal is to make an AI that can out-think a human it's super not useful, either.

Post reply on HN