Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Human-level performance in 3D multiplayer games with population-based RL

science.sciencemag.org

Post reply on HN
Print view View original

11–16 of 16 posts

Re: Human-level performance in 3D multiplayer games with population-based RL

#11

Post by HanayamaTriplet »

d--b wrote: post #9

Do they handicap the AI to have sub-perfect aiming? 3D shooters and Quake in particular are aiming games above. If you aim perfectly, you will always win against humans. Adding randomness to aiming is the way to provide fair comparison to give any conclusion regarding true AI features.

The ML-based AI (or "agent" as the authors referred to it) in the article never achieved perfect aiming, only up to 80% at close range and 0.5% at long range. After artificially delaying reaction time, a team of one strong human plus one delayed agent was only able to win against a team of two delayed agents only 21% of the time. While this only controlled for reaction time and not accuracy, the authors note that the human and delayed agents had similar numbers of hits per game. Meanwhile, the agents had 4-5x more flag captures than humans, leading them to believe that it had developed superior strategy which carried the games even without god-like aim.

Re: Human-level performance in 3D multiplayer games with population-based RL

#12

Post by Mirioron »

Creationer wrote: post #10
dontreact wrote: post #6

What is deepmind going to do once they solve all video games? It seems like they are leveraging the massive economic demand for entertaining simulations of the world. But once they hit the limit of that they will have to make their own simulations.

AI in an RTS, like Deepmind in Starcraft, can simply issue more commands and react faster than a human - since game information can be gleaned directly from the frame buffer, and instructions issued without going through the analogue of a mouse and keyboard.

I want to nitpick a bit: AFAIK DeepMind didn't use the frame buffer in StarCraft. In the widely reported matches DeepMind didn't even have to move the camera. This was apparent in one of the matches against Man's, where DeepMind blink microd its Stalkers over what would be multiple screens for a human player (so you would have to move the camera to do the same, which is impossible at that speed). They did play one match, where DeepMind was basically controlling the camera too (separately trained) and Mana beat it.

On another note, even though it was said that DeepMind was APM limited in StarCraft, it did spike to 1500 at some points, which is not doable for a human. So, I'd say that DeepMind hasn't quite got there yet in beating SC2 pros. They could definitely make very interesting single player opponents though. I hope that in the future I can play a strategy game alone and have some interesting AI opponents to play against.

Re: Human-level performance in 3D multiplayer games with population-based RL

#13

Post by de_watcher »

HanayamaTriplet wrote: post #11
d--b wrote: post #9

Do they handicap the AI to have sub-perfect aiming? 3D shooters and Quake in particular are aiming games above. If you aim perfectly, you will always win against humans. Adding randomness to aiming is the way to provide fair comparison to give any conclusion regarding true AI features.

The ML-based AI (or "agent" as the authors referred to it) in the article never achieved perfect aiming, only up to 80% at close range and 0.5% at long range. After artificially delaying reaction time, a team of one strong human plus one delayed agent was only able to win against a team of two delayed agents only 21% of the time. While this only controlled for reaction time and not accuracy, the authors note that the…

> it had developed superior strategy

The problem is that when it starts to play against humans the humans develop an anti-strategy after just a several matches. Feeding that back into AI doesn't immediately give enough data points for training.

Re: Human-level performance in 3D multiplayer games with population-based RL

#14

Post by ctoth »

d--b wrote: post #9

Do they handicap the AI to have sub-perfect aiming? 3D shooters and Quake in particular are aiming games above. If you aim perfectly, you will always win against humans. Adding randomness to aiming is the way to provide fair comparison to give any conclusion regarding true AI features.

But did they ... but did they ...

But did you ever consider that an AI can just aim better than you period?

I'm totally blind. I could enter a marksmanship competition and demand that everybody be made to play by rules that would work for me, but I then wouldn't be able to say that I'm the best marksman if I win.

Re: Human-level performance in 3D multiplayer games with population-based RL

#15

Post by nightski »

Mirioron wrote: post #12
Creationer wrote: post #10

Earlier quoted context omitted.

AI in an RTS, like Deepmind in Starcraft, can simply issue more commands and react faster than a human - since game information can be gleaned directly from the frame buffer, and instructions issued without going through the analogue of a mouse and keyboard.

I want to nitpick a bit: AFAIK DeepMind didn't use the frame buffer in StarCraft. In the widely reported matches DeepMind didn't even have to move the camera. This was apparent in one of the matches against Man's, where DeepMind blink microd its Stalkers over what would be multiple screens for a human player (so you would have to move the camera to do the same, which is impossible at that speed). They did play one ma…

Can't you jump the camera by clicking on the minimap to anywhere else on the map? Or are you saying the view did not move at all? Just stayed on their base the entire time or something?

Re: Human-level performance in 3D multiplayer games with population-based RL

#16

Post by sjg007 »

dontreact wrote: post #8
jimmy_dean wrote: post #7

Earlier quoted context omitted.

Deepmind can always transfer learning to the robotics industry, if they haven't already. There's bound to be a space they can fill with the progress they've made in these video game reinforcement learning tasks. Maybe aiding in medical discoveries?

The sim to real gap is very large and it honestly seems to me like maybe most of the work is not in the RL part of it but rather instrumenting the robot properly, setting up the right simulation, figuring out how to scale up the real world learning. Haven't seen much robotics stuff come out of Deepmind yet, maybe for these reasons.

You need algorithms that can learn complex tasks and also reason under uncertainty. Not just uncertainty of sensor inputs but also uncertainty of state. The goal of all of these video games is to develop the algorithms over a wide range of difficulty. So algorithms that have memory, learn / plan and are reasonably robust etc...
Post reply on HN