OpenAI at the Dota 2 World Championships • Hacker News Board

Re: OpenAI at the Dota 2 World Championships

#141

Post by intended » Sat, Aug 12, 2017, 4:48 AM UTC

Zarath wrote: post #86

DannyDaemonic wrote: post #75

Earlier quoted context omitted.

I think balance is a bigger problem than people realize. It's much harder than you'd think, and the most common way of balancing things is to simply make them more "samey". I was part of the Warcraft 3 beta. When it started all the races felt so unique. It's a bit foggy now but I think, for example, all the elves buildings - which appeared tree like - would automatically regenerate. Once they started balancing the ga…

Homogenization is a huge problem with nearly every major online game I've played (WoW, LoL, Hearthstone, etc.)

Your looking at the blizzard school of balance.

Think going to a theme park. It's very pretty and attractive but underneath it all, the experience is always on rails without any deviation.

That's one way to deal with complexity. I hate those games, because the moment you become human, and try to do something intuitive but undefined and unwanted by the programmers, you get penalized.

Dota doesnt do that, although the recent patch feels like a shadow of such design: but nothing like league or blizzard

Re: OpenAI at the Dota 2 World Championships

#142

Post by visarga » Sat, Aug 12, 2017, 4:51 AM UTC

popcorncolonel wrote: post #112

ted12345 wrote: post #83

I wonder if some of the things the guy says are misleading. In the video, we see the bot "creep blocking." For those unfamiliar with dota, players can use the model of the unit they control to obstruct the movement of allied computer controlled units in order to gain a favorable position. I suppose it's possible that over millions and millions of matches played against itself, the OpenAI bot "invented" this behavior…

It would be pretty much impossible for the programmers to "build the behavior in" to the neural network, unless you mean training on supervised data or something.

It's not impossible, it's called inverse reinforcement learning, where they learn a value function from an external demonstration. Then they use this value function for teaching the bot an action policy. Intuitively, the idea is to learn first what are a good state and a bad state, based on external demonstrations, then use that to teach the bot how to act.

This kind of learning is similar to GANs, where the discriminator learns from real data and the generator learns from the discriminator.

Re: OpenAI at the Dota 2 World Championships

#143

Post by asddddd » Sat, Aug 12, 2017, 5:02 AM UTC

electrograv wrote: post #111

obastani wrote: post #99

I think one of the biggest challenges in moving from 1v1 to 5v5 is the substantially increased possibility of creative strategies. In a 1v1, there isn't that much room for innovation, making it a perfect target for existing AI techniques, which are good at learning what is "in the data" but have not yet been shown to be capable of thinking outside the box. In contrast, in a 5v5 matchup, a much wider range of strategi…

Without even getting into misconceptions and inaccuracies here, the following quotes illustrate "the moving goalposts of AI": The almost-metaphysical "mystique" that many attribute (perhaps subconsciously) to human intelligence vs machine intelligence: > AI techniques, which are good at learning what is "in the data" but have not yet been shown to be capable of thinking outside the box > AIs would be highly vulnerabl…

Gaming skill is often divided into mechanical skill and decision making. 1v1 like this is dominated heavily by mechanical skill (frame-perfect last hits/denies, the absurd creep block, etc). Generally, it's the sort of thing you can "download skill" in (cheat at) because computers are very good at it.

Decision making is how quake pros can beat aimbotters, and starts to become very important in 5v5 DotA, particularly at a higher level. You'd see it in terms of drafting, ganks, map vision/control, map positioning, sneaking rosh, and other unexpected meta strats. Considering the real-time computation requirements, this is almost certainly a much harder problem.

So while it's entirely possible bot-team would win just through insane mechanics snowballing the game (ganking 0ms reaction times would be fun), or through good-enough decision making + insane mechanics, they could also be outplayed.

Re: OpenAI at the Dota 2 World Championships

#144

Post by dvt » Sat, Aug 12, 2017, 5:02 AM UTC

keerthiko wrote: post #136

dvt wrote: post #129

Earlier quoted context omitted.

In the interview, they literally said they coached it to have certain behaviors.

They said "yes, it does that"/"it's been known to do that", not "it was coached to do that". Just as a "We've seen it do that before."

Yeah, just re-watched the clip, I think you're right. My mistake (the word "coached" confused me).

Re: OpenAI at the Dota 2 World Championships

#145

Post by visarga » Sat, Aug 12, 2017, 5:02 AM UTC

xapata wrote: post #63

ahh wrote: post #54

The developer interviewed claimed that there was no domain-specific knowledge and implied (though didn't explicitly state) there wasn't any training against non-OpenAI bots or players. (I'd love to know the reward function they used for whatever Q-learning variation they ran with.) If this is accurate, one of the things I'm most impressed with is that the bot figured out creep-blocking. (I can't find a good GIF, but…

I expect your conjecture is correct. That's the whole point of deep learning -- there are many layers that automate what would otherwise be human feature extraction.

The magic here doesn't come solely from deep learning, but also from having access to massive simulation. Simulation can make an almost average human-level deep neural net become better than the best human. It happened for Go as well, where AlphaGo learned by self playing millions of games.

I think there is a deep link between simulation and AGI. An AGI would need to be able to imagine how people and objects would act and react in any situation, which is the same as the ability to simulate the world, or to imagine. We might be able to create small simulations like Dota2, but the real world will be much harder.

Re: OpenAI at the Dota 2 World Championships

#146

Post by Swizec » Sat, Aug 12, 2017, 5:20 AM UTC

intended wrote: post #141

Zarath wrote: post #86

Earlier quoted context omitted.

Homogenization is a huge problem with nearly every major online game I've played (WoW, LoL, Hearthstone, etc.)

Your looking at the blizzard school of balance. Think going to a theme park. It's very pretty and attractive but underneath it all, the experience is always on rails without any deviation. That's one way to deal with complexity. I hate those games, because the moment you become human, and try to do something intuitive but undefined and unwanted by the programmers, you get penalized. Dota doesnt do that, although the…

> I hate those games, because the moment you become human, and try to do something intuitive but undefined and unwanted by the programmers, you get penalized.

That's what I loved about Ultima Online. Characters were basically "Here are 100 possible skills to learn and 5 base stats. You get 700 points to distribute among your skills. Good luck"

My fav part was that doing things that require strength made you stronger, doing dexterous tasks increased dexterity etc.

While fun, the game was also an absolute nightmare for beginners. You really needed to find friends that knew what they were doing to help you out. This of course created a wonderful community.

Re: OpenAI at the Dota 2 World Championships

#147

Post by NegativeLatency » Sat, Aug 12, 2017, 6:24 AM UTC

simonebrunozzi wrote: post #118

eljimmy wrote: post #81

Earlier quoted context omitted.

I'm very curious to see how a 5-AI team would "plan" smoke ganks. Really exciting time for technology as deep learning is moving into gaming. I'm reminded of the games played in the movie 'Her'.

For the non-DoTA players out there, there's a lot of specific jargon in the above comments. I've played a few dozen hours, and I understand most of them. 1) "smoke ganks" means that a team goes invisible, in order to surprise the opposing team, and kill as many as possible in a short time. 2) laner: one of the main 3 roles that one of the five players in a team can take (e.g. support, laner, etc). 3) and so on : "shr…

laner refers to the initial resource gathering period where you kill the minions spawned from each team's base

Re: OpenAI at the Dota 2 World Championships

#148

Post by ajsalminen » Sat, Aug 12, 2017, 6:28 AM UTC

croon wrote: post #114

DannyDaemonic wrote: post #75

Earlier quoted context omitted.

I think balance is a bigger problem than people realize. It's much harder than you'd think, and the most common way of balancing things is to simply make them more "samey". I was part of the Warcraft 3 beta. When it started all the races felt so unique. It's a bit foggy now but I think, for example, all the elves buildings - which appeared tree like - would automatically regenerate. Once they started balancing the ga…

> I think balance is a bigger problem than people realize. It's much harder than you'd think, and the most common way of balancing things is to simply make them more "samey". I completely agree, but Dota is the very rare exception to that rule. If one heroes ability is "too strong", most games would turn it down 20%, but dota instead perhaps lowers the hero's armor by 1, or lowers their sight range, or base strength,…

A big hero pool you can only pick one of each from is probably easier to balance than something like an RTS where there are a fixed set of units for just a few alternate races and the players typically choose between them once at the start of their career.

You have to consider how every single one of them interacts with all of the others and even less serious imbalances will be present in the games all the times for players that are unable to switch to playing anything else.

Re: OpenAI at the Dota 2 World Championships

#149

Post by ajsalminen » Sat, Aug 12, 2017, 6:33 AM UTC

CrystalLangUser wrote: post #78

jsmthrowaway wrote: post #65

Earlier quoted context omitted.

DotA and Dota 2 are the games that helped define the term MOBA. I'm not sure why you're saying it isn't one, especially since it's in the first sentence of both Wikipedia articles.

Sometimes people prefer to label dota as an ARTS: Action Real Time Strategy. Some people dislike the MOBA nomenclature because it comes from Riot (maker of League of Legends; a clone of dota). In addition the term is pretty vague. Call of Duty, Starcraft, and DotA could all technically be “MOBAS”. ARTS describes it a little better but I don’t think that term is great either.

In my opinion ARTS doesn't describe it better but introduces confusion since it implies these games have something in common with real time strategy games. I don't think they really have anything to do with RTS despite being originally conceived on the Warcraft 3 engine and assets. At least MOBA makes it clear that it's an entirely separate category.

Re: OpenAI at the Dota 2 World Championships

#150

Post by Tomminn » Sat, Aug 12, 2017, 7:05 AM UTC

unrealhoang wrote: post #105

hmate9 wrote: post #85

Earlier quoted context omitted.

It is extremely difficult. 1v1 is like beating humans at chess. 5v5 is like beating humans at Go. That's the kind of jump it is.

I would even bring that to one more level of difficulty. There are so many choices can be made in-game to gain advantages over enemy: item build, gank/farm distribution, resource sharing, vision fight, choose to defense or let go/trade of tower/objectives... If we can have AI that beat Pro at 5v5 in Dota, it would very likely be an AGI.

No. The value function for the AI will be way too specialized to look anything like AGI.