More on Dota 2
blog.openai.com
More on Dota 2
1–10 of 106 posts
Re: More on Dota 2
#2Props for the $12k donation to OpenDota. That's really awesome! Tho I personally always preferred Dotabuff.
Re: More on Dota 2
#3Dupe, more discussion at https://news.ycombinator.com/item?id=15031470
Re: More on Dota 2
#4I’m by no means an expert, but I’m fascinated by the idea that a neural net playing against itself can substantially outperform a supervised learning approach with a large training data set. I mean, gathering training data and making sure it’s labeled correctly and all that is a huge hassle so if you could eliminate that step or even reduce the amount or quality of training data required that should be a big win for AI, right? Especially if doing this not only makes things easier but also improves the performance of the model.
Re: More on Dota 2
#5I’m by no means an expert, but I’m fascinated by the idea that a neural net playing against itself can substantially outperform a supervised learning approach with a large training data set. I mean, gathering training data and making sure it’s labeled correctly and all that is a huge hassle so if you could eliminate that step or even reduce the amount or quality of training data required that should be a big win for…
Reinforcement learning isn't a new idea - I did a Berkeley-based edX course on it a few years ago now and it was not state-of-the-art to my knowledge. That had no deep aspect to it, we just generated a reinfrocement algorithm that utilised a good measure of performance (specifically, it was pacman and the score value is pretty good at that) and changed a few algorithm weighting variables at each iteration.
My understanding, from talking to a ML friend this morning, is that the latest progress is taking reinforcement learning and applying deep learning approaches (nets, etc.) to it. The key becomes finding the right scoring algorithms to tweak the neural net correctly towards the desired outcome.
The self-play really is the reinforcement side of things at work. How you take that 'score' and use it to correctly modify the input weightings - be them in a neural net, traditional algorithm, etc. - is the key.
Re: More on Dota 2
#6Is there a good self-contained example of how people set up learning in ways where "the AI doesn't initially know the rules"?
I've heard this many times and conceptually I get the principle, but I have a hard time understanding how you create a legitimate starting position or measurement mechanism beyond "losing/winning".
Re: More on Dota 2
#7Is there a good self-contained example of how people set up learning in ways where "the AI doesn't initially know the rules"? I've heard this many times and conceptually I get the principle, but I have a hard time understanding how you create a legitimate starting position or measurement mechanism beyond "losing/winning".
[deleted]
Re: More on Dota 2
#8Is there a good self-contained example of how people set up learning in ways where "the AI doesn't initially know the rules"? I've heard this many times and conceptually I get the principle, but I have a hard time understanding how you create a legitimate starting position or measurement mechanism beyond "losing/winning".
The linked article says "The bot received incentives for winning and basic metrics like health and last hits". So apart from losing/winning, losing health is bad, last hitting is good. You could add more, but apparently that's all OpenAI used.
Re: More on Dota 2
#9Is there a good self-contained example of how people set up learning in ways where "the AI doesn't initially know the rules"? I've heard this many times and conceptually I get the principle, but I have a hard time understanding how you create a legitimate starting position or measurement mechanism beyond "losing/winning".
The linked article says "The bot received incentives for winning and basic metrics like health and last hits". So apart from losing/winning, losing health is bad, last hitting is good. You could add more, but apparently that's all OpenAI used.
It also says "We also separately trained the initial creep block using traditional RL techniques." I have no idea how significant that is, but it seems to be getting a fair amount of attention.
Re: More on Dota 2
#10I’m by no means an expert, but I’m fascinated by the idea that a neural net playing against itself can substantially outperform a supervised learning approach with a large training data set. I mean, gathering training data and making sure it’s labeled correctly and all that is a huge hassle so if you could eliminate that step or even reduce the amount or quality of training data required that should be a big win for…
Reinforcement learning isn't a new idea - I did a Berkeley-based edX course on it a few years ago now and it was not state-of-the-art to my knowledge. That had no deep aspect to it, we just generated a reinfrocement algorithm that utilised a good measure of performance (specifically, it was pacman and the score value is pretty good at that) and changed a few algorithm weighting variables at each iteration. My underst…
Why wouldn't algorithm reach a local maxima when playing with itself, or even degrade over time by opening up to unknown attacks?