Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

More on Dota 2

blog.openai.com

Post reply on HN
Print view View original

11–20 of 106 posts

Re: More on Dota 2

#11

Post by gcp »

mannykannot wrote: post #9
sanxiyn wrote: post #8

Earlier quoted context omitted.

The linked article says "The bot received incentives for winning and basic metrics like health and last hits". So apart from losing/winning, losing health is bad, last hitting is good. You could add more, but apparently that's all OpenAI used.

It also says "We also separately trained the initial creep block using traditional RL techniques." I have no idea how significant that is, but it seems to be getting a fair amount of attention.

It's a highly specific procedure that happens before there is interaction with the opponent, so without "handing over" the understanding that having creeps on your high-ground is good, it's very hard for the learning to see through the noise and discover this.

Re: More on Dota 2

#13

Post by gcp »

noway421 wrote: post #10
NamTaf wrote: post #5

Earlier quoted context omitted.

Reinforcement learning isn't a new idea - I did a Berkeley-based edX course on it a few years ago now and it was not state-of-the-art to my knowledge. That had no deep aspect to it, we just generated a reinfrocement algorithm that utilised a good measure of performance (specifically, it was pacman and the score value is pretty good at that) and changed a few algorithm weighting variables at each iteration. My underst…

Why wouldn't algorithm reach a local maxima when playing with itself, or even degrade over time by opening up to unknown attacks?

One typically keeps pools of trained networks to combat this.

Re: More on Dota 2

#14

Post by mannykannot »

NamTaf wrote: post #5
omarforgotpwd wrote: post #4

I’m by no means an expert, but I’m fascinated by the idea that a neural net playing against itself can substantially outperform a supervised learning approach with a large training data set. I mean, gathering training data and making sure it’s labeled correctly and all that is a huge hassle so if you could eliminate that step or even reduce the amount or quality of training data required that should be a big win for…

Reinforcement learning isn't a new idea - I did a Berkeley-based edX course on it a few years ago now and it was not state-of-the-art to my knowledge. That had no deep aspect to it, we just generated a reinfrocement algorithm that utilised a good measure of performance (specifically, it was pacman and the score value is pretty good at that) and changed a few algorithm weighting variables at each iteration. My underst…

> The key becomes finding the right scoring algorithms to tweak the neural net correctly towards the desired outcome.

Does this not become something similar to supervised learning if you are scoring internal states of the game? (i.e. scoring on more than just the outcome and things that violate the rules?)

Re: More on Dota 2

#15

Post by lvoudour »

I know it has been mentioned a lot the past few days, but since the articles keep flowing about it I'll mention it again:

It's a great feat and kudos to the openai team, but it is VERY unfair for the human players who rely on a sensory interface vs a direct API connection. That's unlike chess or go where the interface isn't important. The really impressive feat will be an AI that uses the same sensory information to make decisions (and I really hope that's where the openai will head next)

Re: More on Dota 2

#16

Post by fridek »

lvoudour wrote: post #15

I know it has been mentioned a lot the past few days, but since the articles keep flowing about it I'll mention it again: It's a great feat and kudos to the openai team, but it is VERY unfair for the human players who rely on a sensory interface vs a direct API connection. That's unlike chess or go where the interface isn't important. The really impressive feat will be an AI that uses the same sensory information to…

Exactly my thoughts after Elon Musk's tweet: https://twitter.com/elonmusk/status/896163163581825025

Unless AI is constrained to pro player max pointer move delta, click rate, and vision latency, I don't really see much difference between AI and a team of kids running with aimbot shouting "cyka cyka".

Re: More on Dota 2

#18

Post by tictacttoe »

omarforgotpwd wrote: post #4

I’m by no means an expert, but I’m fascinated by the idea that a neural net playing against itself can substantially outperform a supervised learning approach with a large training data set. I mean, gathering training data and making sure it’s labeled correctly and all that is a huge hassle so if you could eliminate that step or even reduce the amount or quality of training data required that should be a big win for…

It's very cool, but I think it also requires a very specific "adversarial" problem with a well defined notion of success (winning the game). If your machine learning task is something more nuanced and harder to define, e.g. identify word synonyms, I don't see how you can get around having a training data set.

Re: More on Dota 2

#19

Post by hobofan »

lvoudour wrote: post #15

I know it has been mentioned a lot the past few days, but since the articles keep flowing about it I'll mention it again: It's a great feat and kudos to the openai team, but it is VERY unfair for the human players who rely on a sensory interface vs a direct API connection. That's unlike chess or go where the interface isn't important. The really impressive feat will be an AI that uses the same sensory information to…

And the response as I've seen it on other threads: It probably doesn't make a big difference, and will outperform humans there too, and it would be a huge waste of computing power to train it that way.

I think OpenAI should show that the AI can derive (a close aproximation of) the API data from videos, but I don't think that building a closed training loop would add much value here.

Re: More on Dota 2

#20

Post by hobofan »

fridek wrote: post #16
lvoudour wrote: post #15

I know it has been mentioned a lot the past few days, but since the articles keep flowing about it I'll mention it again: It's a great feat and kudos to the openai team, but it is VERY unfair for the human players who rely on a sensory interface vs a direct API connection. That's unlike chess or go where the interface isn't important. The really impressive feat will be an AI that uses the same sensory information to…

Exactly my thoughts after Elon Musk's tweet: https://twitter.com/elonmusk/status/896163163581825025 Unless AI is constrained to pro player max pointer move delta, click rate, and vision latency, I don't really see much difference between AI and a team of kids running with aimbot shouting "cyka cyka".

> Actions accessible by the bot API, chosen at a frequency comparable to humans

I would guess that this already includes all the delays you are asking for.

Post reply on HN