Grandmaster level in StarCraft II using multi-agent reinforcement learning • Hacker News Board

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#121

Post by ionforce » Thu, Oct 31, 2019, 4:15 PM UTC

Nimitz14 wrote: post #39

nightcracker wrote: post #5

> After 50 games, however, DeepMind hit a snag. Some players had noticed that three user accounts on the Battle.net gaming platform had played the exact same number of StarCraft II games over a similar time frame — the three accounts that AlphaStar was secretly using. When watching replays of these matches, players noticed that the account owner was performing actions that would be extremely difficult, if not impossi…

Funny how they put it as: > When watching replays of these matches, players noticed that the account owner was performing actions that would be extremely difficult, if not impossible, for a human. framing it like the agent was doing some sort insane play, when the reality is the main evidence of a player being alphastar (other than its garbage decision making) was the fact that it wasn't using hotkeys!

The correct terminology would be "control groups".

The configuration of control groups is visible in replays; AlphaStar's replay data does not have any control groups in it.

Though, via its API, it is able to select arbitrary groups of units from anywhere on the map.

Personally, I would have loved to see it work through control group management because I think that is something that is important and steals attention away from humans. But probably to the researchers, it is just annoying data to model that doesn't get to the "core" of StarCraft.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#122

Post by dannypgh » Thu, Oct 31, 2019, 4:17 PM UTC

TremendousJudge wrote: post #113

rate_ofchange wrote: post #98

Earlier quoted context omitted.

> The speed at which AlphaStar overtook previous AIs seems to me to be nearly unprecedented in AI research In pretty much any field, top performing humans are at the physical limitation level, you will not see any sort of breakthrough, just incremental improvement. Machines on the other side, can be scaled arbitrarily. Once you've built a small crane, you can build even increasing ones, it's just a function of money…

>Machines on the other side, can be scaled arbitrarily. Once you've built a small crane, you can build even increasing ones, it's just a function of money and interest. It doesn't matter how much money or interest we have, but right now it isn't technically feasible to build a 36000 km tall crane (also known as a space elevator). Humanity simply couldn't get it done even if we poured all our current resources into th…

> Following the analogy, a general purpose AI simply isn't a bigger Deep Blue or AlphaGo; it's probably something different that requires knowledge that we currently don't have.

I agree with this, but it isn't clear to me that a general AI will have a significantly different impact on society than a world where task-specific well performing AIs are easy for anyone to develop.

Sure, a general AI has a set of properties that are really fascinating to discuss and debate (including what is consciousness and whether AIs should be given rights), and perhaps a general AI is required for doomsday computers-taking-over scenarios, but the impacts that AI will have on our economy and politics don't require general AI.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#123

Post by ionforce » Thu, Oct 31, 2019, 4:17 PM UTC

username90 wrote: post #54

smolder wrote: post #40

Earlier quoted context omitted.

New players (like when I first played) are likely to select individual larva before they do tutorials or learn the hotkeys, but you're right that high level players would almost never select one directly. Maybe after a hatchery has died and there are still larva remaining?

No, it doesn't select the larvae by dragging, it just selects them instantly from the other side of the map. I'd guess it either has a hotkey for the hatcheries and press it and the larvae key instantly without it showing in the replay, or it has a cheat ability to instantly select larvae.

Its API allows it to select units arbitrarily from anywhere (regardless of camera). Camera vision only controls the specificity of data it sees for on-screen units.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#124

Post by 83457 » Thu, Oct 31, 2019, 4:19 PM UTC

empath75 wrote: post #115

angel_j wrote: post #111

I don't think folks realize how much of a game changer AlphaStar is. If it can do the same "job" those high level players, it can probably do any job that doesn't require lifting. It could be playing all the traffic lights in a city. It could manage a fleet of factory drones. It could coach humans engaged in sport or business. It could probably figure out spreadsheets. Drill down, and perhaps it can play with chemica…

Or control a fleet of armed drones, more likely. I’d be surprised if darpa, etc aren’t investigating applications like that right now.

Isn't it likely they have been doing that for decades?

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#125

Post by sinuhe69 » Thu, Oct 31, 2019, 4:21 PM UTC

linuxftw wrote: post #100

_f9cu wrote: post #46

Why they wouldn't solve more impactful problems rather than playing games?

Because that's all they're capable of doing with it. Games can be easily repeated, have clear outcomes. There's no need for nuanced thinking or actual problem-solving. Just approximate the current scenario against previously successful strategies and pursue that strategy. A real challenge would be to invent a new game, have both a human and an "AI" read the rules for the first time, and then compete in their first ga…

There is no deep learning without massive data. What do you expect? :D

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#126

Post by dang » Thu, Oct 31, 2019, 4:22 PM UTC

jplayer01 wrote: post #42

Honestly, the article needs to be replaced with https://deepmind.com/blog/article/AlphaStar-Grandmaster-leve... which actually goes into some technical detail. Nature.com's article is purely for laypersons and, imo, not particularly useful for HN's crowd because of how little insight it gives. It also provides the paper and an archive of all of the AI's matches for anybody who wants to take a closer look. These can b…

This comment was originally posted on https://news.ycombinator.com/item?id=21408024, but we've merged it into the earlier submission, which used the link you mentioned.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#127

Post by dang » Thu, Oct 31, 2019, 4:22 PM UTC

Nimitz14 wrote: post #35

The title seems to contradict the subtitle: > Google AI beats top human players at strategy game StarCraft II vs > DeepMind’s AlphaStar beat all but the very best humans at the fast-paced sci-fi video game.

This comment was originally posted on https://news.ycombinator.com/item?id=21408024, but we've merged it into the earlier submission.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#128

Post by the_af » Thu, Oct 31, 2019, 4:24 PM UTC

PeterisP wrote: post #107

the_af wrote: post #99

Earlier quoted context omitted.

Do you mean Google should write a better AI than the one in Civilization? Interesting. But also consider this: in casual strategy videogames -- actually, strike "strategy" and just consider videogames -- most players don't want a really hard opponent. A computer opponent that is really very hard to beat is not what we want, because that'd be frustrating and many of us play videogames (yes, even strategy games!) to un…

In strategy games like Civilization the "preferred" difficulty level is that hard/easy to beat because it gets extra resources. It would be preferrable to have the same difficulty level through opponents that play better/smarter while having the same game mechanic consequences as players if they make the same actions, but we currently can't, so they get artificial production multipliers and such.

Yes, I understand this. I guess I disagree better/smarter would be better, because in videogames what matters is the illusion of challenge, not a real challenge. So spending resources into developing a real AI for Civilization is probably not the best idea; as long as it tricks casual players into believing it's putting up a fight, that's good enough.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#129

Post by 6gvONxR4sf7o » Thu, Oct 31, 2019, 4:25 PM UTC

username90 wrote: post #31

hoseja wrote: post #15

There has always been the issue of interface when playing videogames AI vs human. Either give the human a brain-computer interface or give the AI a mouse, keyboard, monitor, robot hands and a camera. Anything else seems inherently unfair.

They are testing the AI's ability to do tactics and strategy, not motor and visual tasks. It has built in delay and needs to move around the map to gather information just like a human, so it doesn't have any significant unfair advantages. Edit: Note that the version they sent out to the ladder had significant larger delay, significant lower APM, and didn't get any information not visible on the screen unlike the fir…

Strategy is limited by available tactics, and the ability to execute them. Your (human) tactics and strategy would necessarily change if you and your opponent were required to use a mouse that can only move across the screen over the course of a couple seconds, or if you could only click once per second, or if the computer added noise to when/where your mouse click arrives.

Strategy and tactics are a function of constraints. If an unconstrained computer can beat a constrained human, have we really shown that its strategy and tactics are better? To do that, you'd have them to play under the same constraints. Or at least have the constrains be close enough, which is what people are debating.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#130

Post by ogig » Thu, Oct 31, 2019, 4:25 PM UTC

aketchum wrote: post #112

ogig wrote: post #110

So they added exploiter agents showing the AI what strats are on the meta. When the meta changes, will it still perform? Maybe im reading it wrong but seems much less impressive with the exploiter agents in the mix.

My understanding is that the exploiter agents are not based on the meta. My take is that the exploiter agents have a different goal than the main agent. While the main agent is trying to develop a strategy to win against as many opponents as possible, the exploiters are focused on finding new ways to beat just the main agent. It seems similar to the inspiration behind GANs

I think you are right, from the paper:

>During training we used three agent types that differ only in the distribution of opponents they train against, when they are snapshotted to create a new player, and the probability of resetting to the supervised parameters.

So exploiter agents aren't fed a specific strat, instead they discover the weak spots in the same way as the main agent tries to win. The GANs similarity is there.