I don't think folks realize how much of a game changer AlphaStar is. If it can do the same "job" those high level players, it can probably do any job that doesn't require lifting. It could be playing all the traffic lights in a city. It could manage a fleet of factory drones. It could coach humans engaged in sport or business. It could probably figure out spreadsheets. Drill down, and perhaps it can play with chemical synthesis and quantum theories, or drive a truck.
Grandmaster level in StarCraft II using multi-agent reinforcement learning
111–120 of 321 posts
Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning
#112So they added exploiter agents showing the AI what strats are on the meta. When the meta changes, will it still perform? Maybe im reading it wrong but seems much less impressive with the exploiter agents in the mix.
My understanding is that the exploiter agents are not based on the meta. My take is that the exploiter agents have a different goal than the main agent. While the main agent is trying to develop a strategy to win against as many opponents as possible, the exploiters are focused on finding new ways to beat just the main agent. It seems similar to the inspiration behind GANs
Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning
#113This is a really interesting one to digest. As with previous announcements about AlphaStar, much of the feedback (here and elsewhere) is about the fundamental challenge of assessing human vs. machine in an RTS. These points are very valid - stepping back however, this still feels like a pretty incredible accomplishment. I'm a gold league SC2 player, so maybe in the 30th-50th percentile. Three years ago, when DeepMind…
> The speed at which AlphaStar overtook previous AIs seems to me to be nearly unprecedented in AI research In pretty much any field, top performing humans are at the physical limitation level, you will not see any sort of breakthrough, just incremental improvement. Machines on the other side, can be scaled arbitrarily. Once you've built a small crane, you can build even increasing ones, it's just a function of money…
>Machines on the other side, can be scaled arbitrarily. Once you've built a small crane, you can build even increasing ones, it's just a function of money and interest.
It doesn't matter how much money or interest we have, but right now it isn't technically feasible to build a 36000 km tall crane (also known as a space elevator). Humanity simply couldn't get it done even if we poured all our current resources into that project. It isn't physically impossible, but such a behemoth has requirements that current materials science cannot meet. Building tall cranes generates useful know-how for a space elevator, but it's a really different problem, not just a matter of resources.
Following the analogy, a general purpose AI simply isn't a bigger Deep Blue or AlphaGo; it's probably something different that requires knowledge that we currently don't have. Sure, building Deep Blue and AlphaGo most likely generated part of that money, but that doesn't mean we have everything we need.
You can argue that is simply also money and interest, and that's true, but it's not in the same as building a 10 meter crane and a 20 meter crane.
Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning
#114I don't think folks realize how much of a game changer AlphaStar is. If it can do the same "job" those high level players, it can probably do any job that doesn't require lifting. It could be playing all the traffic lights in a city. It could manage a fleet of factory drones. It could coach humans engaged in sport or business. It could probably figure out spreadsheets. Drill down, and perhaps it can play with chemica…
It is definitely amazing and a huge step forward for RL. However, the paper says that it started off with a policy that beats 84% of players based on imitation learning, so it didn't learn this strategy all on its own from scratch. Also, this required hundreds of thousands (millions? some large number) of simulations to learn, it is much more difficult reach that scale of learning in the real world.
Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning
#115I don't think folks realize how much of a game changer AlphaStar is. If it can do the same "job" those high level players, it can probably do any job that doesn't require lifting. It could be playing all the traffic lights in a city. It could manage a fleet of factory drones. It could coach humans engaged in sport or business. It could probably figure out spreadsheets. Drill down, and perhaps it can play with chemica…
Or control a fleet of armed drones, more likely. I’d be surprised if darpa, etc aren’t investigating applications like that right now.
Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning
#116> placing within the top 0.15% of the region's 90,000 players > 61 wins out of 90 games against high-ranking players This doesn't seem to be quite as commanding as it was in Go. Do we know what MMR it reached or if it consistently beat players like Serral?
It seems like it's only a matter of time though.
StarCraft II also has a rock-paper-scissors nature to it though, so you wouldn't expect even a perfect player to win 100% of the time. There are some strategies that are hard-counters to other strategies, and because of the imperfect information nature of the game, by the time you scout your opponent and see what they're doing, it may be too late to shift and deal with it.
Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning
#117Has anyone read the actual paper ? This summary really makes it look like "mission accomplished", but this was much much more interesting than that. We saw AI do "obviously stupid things", and we also saw them improve a lot in the middle of the trial, as many youtubers showed. AI was also much more interesting when playing the protoss race, and really felt like it was responding to the opponents actions, and the othe…
> But most surprising is that it didn't make any "breathtaking" moves or actions, as opposed to AlphaGO.
I don't play any competitive StarCraft so my view might be moot but I was surprised at the number of siege tanks it uses. It made me wonder if there's some critical advantage to having so many tanks stacked up in a line so that the splash is spread out.
Also I find it weird that it did not build any marauders at all, in any of the games I've seen.
And just in general, no sexy units. No battlecruisers, no infestors, no swarm hosts...
Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning
#118What would be interesting is to limit the AI processing speed to human capacity, which is something like 60 bits per second. In all these AI v. Human games I see, it is really apples to oranges because the human consumes vastly less resources and compute cycles to perform at the same level as the AI. And when I say 'vast' I mean Vast. There is like a quintillion factor difference between the AI and the human. There i…
There's a lot of knobs to wiggle here. I'm more interested in finding out what emerges in the system that we can actually call "AI" the more restraints we put on it.
Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning
#119Earlier quoted context omitted.
> The speed at which AlphaStar overtook previous AIs seems to me to be nearly unprecedented in AI research. Is it not simply the case that, before AlphaStar, very little money and effort was being put into developing AIs for Starcraft 2?
Starcraft is an incredibly complex game. 10^26 possible moves at any point (you can click/drag anywhere on the screen, pressing a keyboard button as you do so), imperfect information, real-time constraints, etc.
That pales in comparison to starcraft at a higher resolution, which has even more points to click on screen! And even that is nothing compared to playing horseshoes in the physical world. There are an infinite number of moves at any given moment! And an infinite number of moments! Horseshoes is clearly the most complex game of all.
Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning
#120I won’t agree until they beat South Korean players at Starcraft I.
They didn't even beat South Koreans at Starcraft 2 in the article