Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Grandmaster level in StarCraft II using multi-agent reinforcement learning

deepmind.com

Post reply on HN
Print view View original

51–60 of 321 posts

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#51

Post by WhompingWindows »

hoseja wrote: post #15

There has always been the issue of interface when playing videogames AI vs human. Either give the human a brain-computer interface or give the AI a mouse, keyboard, monitor, robot hands and a camera. Anything else seems inherently unfair.

Then we are stuck with an inherently unfair system, as robotics is nowhere near ready for keyboard/mouse, and neuroscience is nowhere near ready for CBI.

This current modality is important, IMO, because we could potentially see neural networks performing tasks on other software, not just SC2. Imagine a neural network performing copy-editing in Word, writing code for CRUD applications, etc. Those are some mind-blowing potentials, we'd be losing out if we slowed down to work on robot hands.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#52

Post by tiborsaas »

mieseratte wrote: post #30
floitsch wrote: post #16

Earlier quoted context omitted.

These extremely difficult/impossible things didn't really give an advantage. For example, AlphaStar would sometimes click on an object at the border of the screen. For humans that would be almost impossible, because the screen would scroll when the mouse approaches the border. Similarly, AlphaStar would not play with group hotkeys, but use a different technique. However, in none of the analyses, people noticed things…

> For humans that would be almost impossible, because the screen would scroll when the mouse approaches the border. That humans cannot reliably perform these actions because of the limitations of our corporeal form means that Alphastar has an advantage over a human player. Limiting APM isn't enough.

It's an AI system, of course it has some advantage, that's why we are building them.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#54

Post by username90 »

smolder wrote: post #40
Iv wrote: post #12

Earlier quoted context omitted.

To play against alphastar, you have to opt-in. IIRC it states Alphastar will hide itself. One of the things people noticed in replays was the lack of control groups and in the case of zergs, the ability to select larvas directly, which no player ever does. It could have been as simple as removing these quirks.

New players (like when I first played) are likely to select individual larva before they do tutorials or learn the hotkeys, but you're right that high level players would almost never select one directly. Maybe after a hatchery has died and there are still larva remaining?

No, it doesn't select the larvae by dragging, it just selects them instantly from the other side of the map. I'd guess it either has a hotkey for the hatcheries and press it and the larvae key instantly without it showing in the replay, or it has a cheat ability to instantly select larvae.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#56

Post by ryanmercer »

Am I the only one that doesn't care about software beating humans at video games?

I'm sure there's some sort of useful learning being done here by the people that created the software that might some day help them create software that can better predict what is needed for a specific application but to me it just feels like entities like OpenAI and Google researching this predictive software are just wasting obscene amounts of money.

How about train the stuff on better typed OCR? Better handwriting OCR? This will have actual commercial application.

Why not make something for grading papers. English papers, math homework, etc? Start at a first grade level and as you train the software up move on to higher levels of education. My fiance is a high school teacher, she currently only teaches math but previously has done math and English, she sits there grading papers off the clock while watching television in the evenings and on the weekends... MANY high school teachers are in this situation, think of how much free time could be reclaimed by training this instead of teaching software how to beat humans in video games!!!

This would even help teachers have more time during school hours to help struggling students, if you aren't trying to grade papers in class while students are doing work you free up time that you could actively be assisting one or more students. Instead, these "AI" researchers keep training software to be the best at video games... facepalm

Then take that and apply it to something like my job. I clear international freight through customs for a living, I look at paperwork all day and then have to determine what tariff number I should use for something (cell phone 8517120050, silver ring over 1.50 USD 7113115000) and classify every single line on an invoice by using familiarity with the tariff schedule/description(s) on the invoice/any other supporting documentation/customer profiles for customers that have paid to have product databases on file with us. My employer, we do thousands of these things a day and have to keep everything for several years (5 IIRC) in the evening CBP or any other OGA wants to see it during that time period.

So take that huge, pre-existing, data set like that, identify which shipments were classified correctly and which were not, and then let the software have a stab at trying to do it with that better OCR you created.

Then, you know, actually get rid of (or drastically reduce) soul crushing, mind-numbing, highly repetitive digital paperwork jobs like mine.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#58

Post by bsaul »

Has anyone read the actual paper ? This summary really makes it look like "mission accomplished", but this was much much more interesting than that. We saw AI do "obviously stupid things", and we also saw them improve a lot in the middle of the trial, as many youtubers showed.

AI was also much more interesting when playing the protoss race, and really felt like it was responding to the opponents actions, and the other races not so much.

But most surprising is that it didn't make any "breathtaking" moves or actions, as opposed to AlphaGO. Actually not a single game made pro player realize something new about the game. Which is really embarassing, because it suggests that the AI just was able to correctly reproduce existing strategies and build orders, that it probably "learned" from existing pro games in the training sample.

I was really hoping for a more interesting report, honestly explaining the shortcomings of the technics used and giving hints for the obvious blunders. As well as a roadmap for a second round, this time aiming at beating the very top players.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#59

Post by interblag »

This is a really interesting one to digest. As with previous announcements about AlphaStar, much of the feedback (here and elsewhere) is about the fundamental challenge of assessing human vs. machine in an RTS. These points are very valid - stepping back however, this still feels like a pretty incredible accomplishment.

I'm a gold league SC2 player, so maybe in the 30th-50th percentile. Three years ago, when DeepMind started this project (and after nearly two decades of research into SC/SC2) I could probably have beaten the best non-cheating AI. Now, after 3 years, this AI is playing at a Grandmaster level, under at least a reasonable approach to fairness. By comparison, according to the AlphaGo paper [1] the best Go AIs prior to AlphaGo were playing at a 6 Dan level, which looks to be somewhere in the 90-98th percentile [2].

The speed at which AlphaStar overtook previous AIs seems to me to be nearly unprecedented in AI research. This is like if the world's best chess AI had gone from losing high school tournaments to being competitive with Kasparov in less than 3 years. Valid criticisms aside, this feels like an incredible achievement.

[1] https://www.nature.com/articles/nature16961.pdf [2] https://senseis.xmp.net/?KGSRankHistogram

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#60

Post by xorfish »

Nimitz14 wrote: post #35

The title seems to contradict the subtitle: > Google AI beats top human players at strategy game StarCraft II vs > DeepMind’s AlphaStar beat all but the very best humans at the fast-paced sci-fi video game.

Professional players that play in tournaments know each other and how they play. This might be a further weakness even if Alphastar can beat the best players. Other players may adapt to the play of Alphastar when they know that they play against it.
Post reply on HN