Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI at the Dota 2 World Championships

openai.com

Post reply on HN
Print view View original

111–120 of 219 posts

Re: OpenAI at the Dota 2 World Championships

#111

Post by electrograv »

obastani wrote: post #99

I think one of the biggest challenges in moving from 1v1 to 5v5 is the substantially increased possibility of creative strategies. In a 1v1, there isn't that much room for innovation, making it a perfect target for existing AI techniques, which are good at learning what is "in the data" but have not yet been shown to be capable of thinking outside the box. In contrast, in a 5v5 matchup, a much wider range of strategi…

Without even getting into misconceptions and inaccuracies here, the following quotes illustrate "the moving goalposts of AI": The almost-metaphysical "mystique" that many attribute (perhaps subconsciously) to human intelligence vs machine intelligence:

  > AI techniques, which are good at learning what is "in the data" 
  but have not yet been shown to be capable of thinking outside the box

  > AIs would be highly vulnerable

  > exploitable quirks in the AI

  > established lack of robustness of neural nets

  > clever human players will be quick to exploit them
Of course, it is important to identify pitfalls, so as to address them. But it is sad that no matter how good AI gets, there will always be pessimistic reasoning downplaying its success. With every step forward, there are the predictions of impassible future failures -- "just around the corner"!

Due to this mystical "cleverness" attributed exclusively to humans, I'm not convinced a majority of humans will ever allow machines to be considered 'intelligent', no matter how far the technology advances. I'd love to be proven wrong, though.

Re: OpenAI at the Dota 2 World Championships

#112

Post by popcorncolonel »

ted12345 wrote: post #83

I wonder if some of the things the guy says are misleading. In the video, we see the bot "creep blocking." For those unfamiliar with dota, players can use the model of the unit they control to obstruct the movement of allied computer controlled units in order to gain a favorable position. I suppose it's possible that over millions and millions of matches played against itself, the OpenAI bot "invented" this behavior…

It would be pretty much impossible for the programmers to "build the behavior in" to the neural network, unless you mean training on supervised data or something.

Re: OpenAI at the Dota 2 World Championships

#114

Post by croon »

DannyDaemonic wrote: post #75
habitue wrote: post #59

So an interesting thing you could do as a game company with an "unbeatable" bot is to use it to balance a metagame. Let the bots all play each other and tweak character stats etc until they win a proportional amount of the time. (This presupposes that the bot learns the game the way OpenAI claims it is, without needing to learn from player replays or from playing in a very different way than human players do with tre…

I think balance is a bigger problem than people realize. It's much harder than you'd think, and the most common way of balancing things is to simply make them more "samey". I was part of the Warcraft 3 beta. When it started all the races felt so unique. It's a bit foggy now but I think, for example, all the elves buildings - which appeared tree like - would automatically regenerate. Once they started balancing the ga…

> I think balance is a bigger problem than people realize. It's much harder than you'd think, and the most common way of balancing things is to simply make them more "samey".

I completely agree, but Dota is the very rare exception to that rule.

If one heroes ability is "too strong", most games would turn it down 20%, but dota instead perhaps lowers the hero's armor by 1, or lowers their sight range, or base strength, or something even less intuitive and tangentially related.

What you get is 100+ very unique heroes, that somehow manages to stay fairly balanced. It's quite an amazing game, if you can handle the timesink.

Re: OpenAI at the Dota 2 World Championships

#115

Post by arnioxux »

thefreeman wrote: post #74
ahh wrote: post #54

The developer interviewed claimed that there was no domain-specific knowledge and implied (though didn't explicitly state) there wasn't any training against non-OpenAI bots or players. (I'd love to know the reward function they used for whatever Q-learning variation they ran with.) If this is accurate, one of the things I'm most impressed with is that the bot figured out creep-blocking. (I can't find a good GIF, but…

I don't know that they specifically said there was no domain-specific knowledge. iirc they said they didn't "teach it the rules of dota" but they also said the training involved "coaching". I interpret that to include showing the bot useful techniques (like creep blocking) which the AI then learned leads to higher win rates, etc.

I think "coaching" just means "curriculum learning" (which is basically as you've described).

But in the future coaching might indeed just be a human expert talking in natural language to teach it how to be better! https://www.youtube.com/watch?v=L_jGDV_5gPA&t=6

Re: OpenAI at the Dota 2 World Championships

#116

Post by screye »

ahh wrote: post #54

The developer interviewed claimed that there was no domain-specific knowledge and implied (though didn't explicitly state) there wasn't any training against non-OpenAI bots or players. (I'd love to know the reward function they used for whatever Q-learning variation they ran with.) If this is accurate, one of the things I'm most impressed with is that the bot figured out creep-blocking. (I can't find a good GIF, but…

> The developer

I was surprised to learn that he is the CTO of Open AI.

> If this is accurate, one of the things I'm most impressed with is that the bot figured out creep-blocking

Same. It is such long set of moves to get a perfect block. To have such a long move-set figured out as something advantageous is very high level of planning from the bot.

Re: OpenAI at the Dota 2 World Championships

#117

Post by Noos »

vecter wrote: post #103
jmkapz wrote: post #92

Earlier quoted context omitted.

I'm talking mid to long term here. To rephrase, is there a sizeable amount of players who will stop playing, knowing that they cannot ever be the best? Because when AI beats one, I don't believe there is a coming back. For the concrete AlphaGo example, of course it was hugely popular but I would be interested in the evolution on the number of players, how many of those viewers started playing balanced by how many exi…

I don't think it will matter at all. Literally, zero impact. Chess computers crush the world's best humans, yet here we are in meatspace, still vying to win the human world chess championship. Also, 99.99999% of players realize they'll never ever be close to being the best in the world, yet they still play the game because it is inherently fun.

the problem is that players will use those bots against other players, just like they do in online chess. The burden of reporting them will be too much for online play probably. It's the ultimate cheat.

Re: OpenAI at the Dota 2 World Championships

#118

Post by simonebrunozzi »

eljimmy wrote: post #81
darrenkopp wrote: post #60

Really was quite interesting to watch ( https://www.twitch.tv/videos/166172514?t=7h3m10s ). Honestly, the bot played _extremely_ well, but I think the biggest advantage was how much faster it's reaction time was and it's movements were likely much more precise than a human is with a mouse. I'm pretty interested in seeing their 5v5 results as well. It seems like that will have similar results as the bots can coordinat…

I'm very curious to see how a 5-AI team would "plan" smoke ganks. Really exciting time for technology as deep learning is moving into gaming. I'm reminded of the games played in the movie 'Her'.

For the non-DoTA players out there, there's a lot of specific jargon in the above comments.

I've played a few dozen hours, and I understand most of them.

1) "smoke ganks" means that a team goes invisible, in order to surprise the opposing team, and kill as many as possible in a short time.

2) laner: one of the main 3 roles that one of the five players in a team can take (e.g. support, laner, etc).

3) and so on : "shrine", "regen", "soul ring", etc, are all very specific locations or objects or concepts (regen = regeneration).

DoTA is a really, really interesting game. Very strategic, great lore and graphics, and a huge fan base.

Re: OpenAI at the Dota 2 World Championships

#119

Post by flunhat »

electrograv wrote: post #111
obastani wrote: post #99

I think one of the biggest challenges in moving from 1v1 to 5v5 is the substantially increased possibility of creative strategies. In a 1v1, there isn't that much room for innovation, making it a perfect target for existing AI techniques, which are good at learning what is "in the data" but have not yet been shown to be capable of thinking outside the box. In contrast, in a 5v5 matchup, a much wider range of strategi…

Without even getting into misconceptions and inaccuracies here, the following quotes illustrate "the moving goalposts of AI": The almost-metaphysical "mystique" that many attribute (perhaps subconsciously) to human intelligence vs machine intelligence: > AI techniques, which are good at learning what is "in the data" but have not yet been shown to be capable of thinking outside the box > AIs would be highly vulnerabl…

I will consider an A.I. truly intelligent when it passes the Turing test with flying colors. And as a native English speaker, too (not just barely passing by posing as a 13 year old Ukrainian boy with a poor grasp of English, which is what happened last time).

As far I know, no A.I. developed has ever met this bar. It seems that the nuances, inconsistencies, and total freedom of language make it a totally different kind of problem to solve than a simple game-playing algorithm.

Or maybe I'm wrong and getting an A.I. to pass the Turing test is just another kind of game-playing algorithm with different parameters. At any rate, we're quite a way off.

Post reply on HN