Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The use of embeddings in OpenAI Five

neuro.cs.ut.ee

Post reply on HN
Print view View original

21–29 of 29 posts

Re: The use of embeddings in OpenAI Five

#21

Post by kenhwang »

nindalf wrote: post #17
kenhwang wrote: post #16

Earlier quoted context omitted.

Off the top of my head: - using skills against targets where the skill would have no effect - using skills on nothing with all plausible targets not remotely nearby - using items to turn invisible and then immediately taking an action to remove invisibility - everything regarding vision/detection: placing vision where vision already exists via structures, placing multiple sources of vision on top of each other, buyin…

You're missing the forest for the trees. Yes, it is true that the bots was suboptimal and made elementary errors, notably around vision. However, overall they did really well. The mere fact that they could coordinate with each other without explicit communication was very impressive. On each of the poor utilization of skills, I could counter by saying there were other objectives in play. For example, you didn't menti…

Sure, it stuns, but it was often stacked with other stuns. I don't remember it ever being used on a low health target. So I'm drawing the conclusion that the bot only understands that the skill stuns, but doesn't understand the scaling damage.

It's impressive that the AI managed to gain a rudimentary understanding of the game almost completely independently since DotA2 is a very complicated game. It's just not interesting problem because game mechanics can easily be codified. It's like being impressed at self-driving cars because the AI learned that it should stop at a red light, it should just be coded to stop at a red light. Learning to stop at a red light isn't the interesting problem that needs solving for self-driving cars.

It's definitely not the "AI is better than humans" narrative that these articles like to push.

Re: The use of embeddings in OpenAI Five

#22

Post by qwert7890 »

nindalf wrote: post #19

Some of the comments in the thread are the usual HN contrarianism - "oh, this isn't impressive because X, Y, Z". This is not the case. What OpenAI have accomplished is extraordinary, ~~especially the part where there is no explicit communication between the bots~~ (not true, see below). They managed to coordinate simply by looking at what the others were doing. Did they make elementary errors? Yes they did. Did they…

> They managed to coordinate simply by looking at what the others were doing.

This bit seems incorrect, https://medium.com/@stelmaszczykadam/do-openai-five-dota-2-b....

Re: The use of embeddings in OpenAI Five

#23

Post by Ntrails »

nindalf wrote: post #17
kenhwang wrote: post #16

Earlier quoted context omitted.

Off the top of my head: - using skills against targets where the skill would have no effect - using skills on nothing with all plausible targets not remotely nearby - using items to turn invisible and then immediately taking an action to remove invisibility - everything regarding vision/detection: placing vision where vision already exists via structures, placing multiple sources of vision on top of each other, buyin…

You're missing the forest for the trees. Yes, it is true that the bots was suboptimal and made elementary errors, notably around vision. However, overall they did really well. The mere fact that they could coordinate with each other without explicit communication was very impressive. On each of the poor utilization of skills, I could counter by saying there were other objectives in play. For example, you didn't menti…

> n each of the poor utilization of skills, I could counter by saying there were other objectives in play. For example, you didn't mention that the "skill that does more damage for every point of missing health on targets" also happens to freeze the target for 1.5 seconds. Perhaps the goal of the bot was just to hold the enemy in place.

Using scythe to hold a Bkb'd target in place is a classic use of an immunity piercing stun acknowledging you'll get zero damage. This wasn't the case in the game I watched them play, it was just "a stun". Which is fine, honestly. But they prioritised the stun over the damage and the respawn time increase (which is actually just as important as the damage). I'm not sure I believe they understood all of the aspects of the spell based on how they used it.

The reality remains that the bots are just playing micro level incredible dota, and macro level mediocre dota. There are endless examples. Giving aegis to supports, never stacking creeps for carries, ganking low importance heros, using DP ult to farm jungle.

They're still better than me and probably anyone I'll ever play with. vOv

Re: The use of embeddings in OpenAI Five

#24

Post by Ajedi32 »

qwert7890 wrote: post #22
nindalf wrote: post #19

Some of the comments in the thread are the usual HN contrarianism - "oh, this isn't impressive because X, Y, Z". This is not the case. What OpenAI have accomplished is extraordinary, ~~especially the part where there is no explicit communication between the bots~~ (not true, see below). They managed to coordinate simply by looking at what the others were doing. Did they make elementary errors? Yes they did. Did they…

> They managed to coordinate simply by looking at what the others were doing. This bit seems incorrect, https://medium.com/@stelmaszczykadam/do-openai-five-dota-2-b... .

In Dota you have perfect information of the state of all allied units, so I think it's debatable whether sharing input (observation) data between the bots really counts as "communication".

Though that same fact also means communication shouldn't really be necessary; the bots are all exact copies of each other and share a copy of the game state, so they should all have similar ideas of what actions are optimal at any given point in the game.

Re: The use of embeddings in OpenAI Five

#25

Post by backpropaganda »

abdullahkhalids wrote: post #20

Lots of OpenAI Five bashing going on in this thread. I propose a gaming-bot-realism Turing test: when a group of rank 100 or better players cannot discriminate between human teams and the bot team by watching the game, only then are the bots playing "real Dota 2".

Great test. The bots won't be able to pass. Both fast reaction times and lack of cohesive strategy will give them away in the first 5 minutes.

Re: The use of embeddings in OpenAI Five

#26

Post by backpropaganda »

kenhwang wrote: post #11
debacle wrote: post #10

Earlier quoted context omitted.

I think that smart teams are going to extract what they can out of these bots. Even top players had difficulties dealing with the inhumanness of the bots - people are generally trained to compete against something that acts to some degree on instinct, not probability. The bots lack if fear in e.g. diving a tower before 5 minutes definitely paid off to a certain degree. I also personally feel that the caster stack of…

Not even lategame, they overcommitted at all points in the game. Those tower dives for trades early game rarely lead to an objective or advantage. They were able to trade early because superior mechanical advantage matters more early game. By the time midgame rolled around, it was pretty clear how naive their strategy was. It has an element of surprise to it since it's not a very human strategy, but just because it's…

That's a great example. It would be great if you could write a blog post on OpenAI Five. There's a LOT of misinformation on this and could use a treatment like this: https://www.alexirpan.com/2018/02/14/rl-hard.html

Re: The use of embeddings in OpenAI Five

#27

Post by qwert7890 »

Ajedi32 wrote: post #24
qwert7890 wrote: post #22

Earlier quoted context omitted.

> They managed to coordinate simply by looking at what the others were doing. This bit seems incorrect, https://medium.com/@stelmaszczykadam/do-openai-five-dota-2-b... .

In Dota you have perfect information of the state of all allied units, so I think it's debatable whether sharing input (observation) data between the bots really counts as "communication". Though that same fact also means communication shouldn't really be necessary; the bots are all exact copies of each other and share a copy of the game state, so they should all have similar ideas of what actions are optimal at any…

Thank you Ajedi32, I updated the post https://medium.com/@stelmaszczykadam/do-openai-five-dota-2-b....

Re: The use of embeddings in OpenAI Five

#28

Post by prestonh »

backpropaganda wrote: post #7
prestonh wrote: post #5

Earlier quoted context omitted.

>> No complicated behavior was produced. The bots were routinely pulling off coordinated team behavior that players couldn't figure out, but that worked. This has to qualify as complicated. >> Dota 2 game was not played. A tiny subset of the game was attempted. Since people seem to be just buying whatever OpenAI propaganda sells them, let me be specific: only 18 heroes are in the game. The combinatorics explode when…

> The bots were routinely pulling off coordinated team behavior that players couldn't figure out, but that worked. This has to qualify as complicated. This is just not true. Do you know the game? Are you speaking as a player? Or are you telling us what's written in the OpenAI blog post? There is nothing a player couldn't figure out. The caster team lost because of the broken game and because few of them were rusted (…

> This is just not true. Do you know the game? Are you speaking as a player? Or are you telling us what's written in the OpenAI blog post? There is nothing a player couldn't figure out. The caster team lost because of the broken game and because few of them were rusted (Merlini hadn't played for months). The bots were garbage at the TI, and got beaten without any problem by the pro teams.

A former player. I'm not regurgitating the blog post, I'm regurgitating what the players themselves said. The AI got beaten by pro teams (top > The hero pool is still 18 heroes. Dota 2 has over 110 heroes. Can you please try to think what makes you say something so wrong with so much confidence?

Admittedly this was a mistake, the language they used in their blog post was "Removed our last major restriction from what most pros consider 'Real Dota” gameplay'", which is poorly explained and made me think the hero pool was entirely open.

DotA is basically 2 games, the drafting part and the gameplay part. The bots made huge progress in figuring out the gameplay part, which is super impressive.

>> Casters don't count as serious players.

Maybe we're using different language. The casters are definitely in the top 1% of players, or more, which I consider "serious", but not "the best". But no one was arguing that the bots are "the best", which is self-evident from their loss at the International.

Anyways, this is all beside the point. What OpenAI was able to do was really impressive and is only helping to advance the state of reinforcement learning. You argued that there's nothing impressive about what they've done, but I'd love to see you point me to an example of an ML algorithm that learned to play a team game as complex as DotA at a competent level.

Re: The use of embeddings in OpenAI Five

#29

Post by backpropaganda »

prestonh wrote: post #28
backpropaganda wrote: post #7

Earlier quoted context omitted.

> The bots were routinely pulling off coordinated team behavior that players couldn't figure out, but that worked. This has to qualify as complicated. This is just not true. Do you know the game? Are you speaking as a player? Or are you telling us what's written in the OpenAI blog post? There is nothing a player couldn't figure out. The caster team lost because of the broken game and because few of them were rusted (…

> This is just not true. Do you know the game? Are you speaking as a player? Or are you telling us what's written in the OpenAI blog post? There is nothing a player couldn't figure out. The caster team lost because of the broken game and because few of them were rusted (Merlini hadn't played for months). The bots were garbage at the TI, and got beaten without any problem by the pro teams. A former player. I'm not reg…

At a competent level? None exist. Being better than random doesn't count as competency. As others have said in this thread, it wouldn't even pass as median performance.

> is only helping to advance the state of reinforcement learning

Zero new algorithms or ideas were introduced by OpenAI Five. We just learn that model-free RL doesn't scale and we already knew that from Atari and robotics benchmarks.

Post reply on HN