The use of embeddings in OpenAI Five • Hacker News Board

Re: The use of embeddings in OpenAI Five

#11

Post by kenhwang » Wed, Sep 12, 2018, 4:27 AM UTC

debacle wrote: post #10

kenhwang wrote: post #9

Going to echo what I said last time before everyone falls for the propaganda. The bots did not play at a top tier level, the bots didn't even play at a level that could be considered median level. They did not discover new mechanics. They couldn't even understand basic mechanics. Their coordination was no more sophisticated than "wander around together and press all the buttons on the first enemy". They were showing…

I think that smart teams are going to extract what they can out of these bots. Even top players had difficulties dealing with the inhumanness of the bots - people are generally trained to compete against something that acts to some degree on instinct, not probability. The bots lack if fear in e.g. diving a tower before 5 minutes definitely paid off to a certain degree. I also personally feel that the caster stack of…

Not even lategame, they overcommitted at all points in the game. Those tower dives for trades early game rarely lead to an objective or advantage. They were able to trade early because superior mechanical advantage matters more early game.

By the time midgame rolled around, it was pretty clear how naive their strategy was. It has an element of surprise to it since it's not a very human strategy, but just because it's not human doesn't make it remotely good.

It's like watching a a car drive on a sidewalk in reverse uphill and honking to avoid pedestrians. It's very impressive that the car figured out that driving on sidewalks reduces collisions with other vehicles, and honking reduces the chance of hitting pedestrians, and it's doing that all while driving in reverse which is very hard for a human to do. But no one in their right mind would call that good driving.

Re: The use of embeddings in OpenAI Five

#12

Post by _ptgt » Wed, Sep 12, 2018, 6:15 AM UTC

backpropaganda wrote: post #8

tlb wrote: post #3

Earlier quoted context omitted.

It was a subset in early demos, but not any more: https://blog.openai.com/the-international-2018-results/

This is the kind of misinformation OpenAI is so great at. Otherwise smart people like tlb are led to believe that this was the full game even though it's not even close to the full game. Thanks for helping me make my point.

It's an Elon Musk outfit; misinformation is in its DNA.

Re: The use of embeddings in OpenAI Five

#13

Post by duskwuff » Wed, Sep 12, 2018, 6:42 AM UTC

backpropaganda wrote: post #7

prestonh wrote: post #5

Earlier quoted context omitted.

>> No complicated behavior was produced. The bots were routinely pulling off coordinated team behavior that players couldn't figure out, but that worked. This has to qualify as complicated. >> Dota 2 game was not played. A tiny subset of the game was attempted. Since people seem to be just buying whatever OpenAI propaganda sells them, let me be specific: only 18 heroes are in the game. The combinatorics explode when…

> The bots were routinely pulling off coordinated team behavior that players couldn't figure out, but that worked. This has to qualify as complicated. This is just not true. Do you know the game? Are you speaking as a player? Or are you telling us what's written in the OpenAI blog post? There is nothing a player couldn't figure out. The caster team lost because of the broken game and because few of them were rusted (…

> This is just not true. Do you know the game? Are you speaking as a player? Or are you telling us what's written in the OpenAI blog post? There is nothing a player couldn't figure out.

A lot has been made of the 1v1 Shadow Fiend OpenAI player having figured out an edge case of Magic Stick recharging. (Casting spells while outside vision doesn't give charges to the enemy hero.) This mechanic was not previously known to the OpenAI team, but was known to professional players -- it's not the epiphany it's made out to be.

Re: The use of embeddings in OpenAI Five

#14

Post by duskwuff » Wed, Sep 12, 2018, 6:44 AM UTC

debacle wrote: post #10

kenhwang wrote: post #9

Going to echo what I said last time before everyone falls for the propaganda. The bots did not play at a top tier level, the bots didn't even play at a level that could be considered median level. They did not discover new mechanics. They couldn't even understand basic mechanics. Their coordination was no more sophisticated than "wander around together and press all the buttons on the first enemy". They were showing…

I think that smart teams are going to extract what they can out of these bots. Even top players had difficulties dealing with the inhumanness of the bots - people are generally trained to compete against something that acts to some degree on instinct, not probability. The bots lack if fear in e.g. diving a tower before 5 minutes definitely paid off to a certain degree. I also personally feel that the caster stack of…

> The bots lack if fear in e.g. diving a tower before 5 minutes definitely paid off to a certain degree.

It paid off because the bots were familiar with the 5-courier format, and knew that they could ferry in consumables to recover quickly. This is not normal in DotA 2; the game was balanced around consumables being limited in the early game, and the human players OpenAI faced were not familiar with this new, unbalanced state of affairs.

Re: The use of embeddings in OpenAI Five

#15

Post by abdullahkhalids » Wed, Sep 12, 2018, 6:58 AM UTC

kenhwang wrote: post #9

Going to echo what I said last time before everyone falls for the propaganda. The bots did not play at a top tier level, the bots didn't even play at a level that could be considered median level. They did not discover new mechanics. They couldn't even understand basic mechanics. Their coordination was no more sophisticated than "wander around together and press all the buttons on the first enemy". They were showing…

> They couldn't even understand basic mechanics.

Can you expand on this?

Re: The use of embeddings in OpenAI Five

#16

Post by kenhwang » Wed, Sep 12, 2018, 7:43 AM UTC

abdullahkhalids wrote: post #15

kenhwang wrote: post #9

Going to echo what I said last time before everyone falls for the propaganda. The bots did not play at a top tier level, the bots didn't even play at a level that could be considered median level. They did not discover new mechanics. They couldn't even understand basic mechanics. Their coordination was no more sophisticated than "wander around together and press all the buttons on the first enemy". They were showing…

> They couldn't even understand basic mechanics. Can you expand on this?

Off the top of my head:

- using skills against targets where the skill would have no effect

- using skills on nothing with all plausible targets not remotely nearby

- using items to turn invisible and then immediately taking an action to remove invisibility

- everything regarding vision/detection: placing vision where vision already exists via structures, placing multiple sources of vision on top of each other, buying detection items when none of the opponents could turn invisible, placing detection in areas where detection already exists

- using a item that's canceled by close proximity to enemies in obvious close proximity of the enemy

- waiting for a neutral respawn when its impossible for it to respawn

- stacking effects that don't stack

Then there's obvious poor utilization of skills, such as:

- using a skill that does more damage for every point of missing health on targets that have no missing health

- using ults with long cooldowns on neutrals when that hero's normal skill does more damage with much much lower cooldown

- using skills that multiply it's effectiveness when there are secondary targets when there's obviously no secondary target near the primary target

- casting damage amplification on a target (target takes more damage) and then not dealing damage

Re: The use of embeddings in OpenAI Five

#17

Post by nindalf » Wed, Sep 12, 2018, 8:01 AM UTC

kenhwang wrote: post #16

abdullahkhalids wrote: post #15

Earlier quoted context omitted.

> They couldn't even understand basic mechanics. Can you expand on this?

Off the top of my head: - using skills against targets where the skill would have no effect - using skills on nothing with all plausible targets not remotely nearby - using items to turn invisible and then immediately taking an action to remove invisibility - everything regarding vision/detection: placing vision where vision already exists via structures, placing multiple sources of vision on top of each other, buyin…

You're missing the forest for the trees. Yes, it is true that the bots was suboptimal and made elementary errors, notably around vision. However, overall they did really well. The mere fact that they could coordinate with each other without explicit communication was very impressive.

On each of the poor utilization of skills, I could counter by saying there were other objectives in play. For example, you didn't mention that the "skill that does more damage for every point of missing health on targets" also happens to freeze the target for 1.5 seconds. Perhaps the goal of the bot was just to hold the enemy in place.

Honestly it's pretty easy to sit around saying "yeah nothing impressive here". But I've been playing this game for a decade and fact is, if I had seen OpenAI 5 a year ago I simply would not have believed it was possible for bots to play this well.

Re: The use of embeddings in OpenAI Five

#18

Post by abdullahkhalids » Wed, Sep 12, 2018, 8:12 AM UTC

nindalf wrote: post #17

kenhwang wrote: post #16

Earlier quoted context omitted.

Off the top of my head: - using skills against targets where the skill would have no effect - using skills on nothing with all plausible targets not remotely nearby - using items to turn invisible and then immediately taking an action to remove invisibility - everything regarding vision/detection: placing vision where vision already exists via structures, placing multiple sources of vision on top of each other, buyin…

You're missing the forest for the trees. Yes, it is true that the bots was suboptimal and made elementary errors, notably around vision. However, overall they did really well. The mere fact that they could coordinate with each other without explicit communication was very impressive. On each of the poor utilization of skills, I could counter by saying there were other objectives in play. For example, you didn't menti…

They do in fact broadcast information to teammates [1] in recent versions:

> OpenAI Five sends 512 such values every couple of milliseconds

I think one of the challenges to making the bots play realistic Dota would be to limit this passage of information. I would say only one bot can talk every time period (say 1.5 seconds) and minimum time before new information is broadcast is >0.5 seconds.

[1] https://medium.com/@stelmaszczykadam/do-openai-five-dota-2-b...

Re: The use of embeddings in OpenAI Five

#19

Post by nindalf » Wed, Sep 12, 2018, 8:15 AM UTC

Some of the comments in the thread are the usual HN contrarianism - "oh, this isn't impressive because X, Y, Z".

This is not the case. What OpenAI have accomplished is extraordinary, ~~especially the part where there is no explicit communication between the bots~~ (not true, see below). They managed to coordinate simply by looking at what the others were doing.

Did they make elementary errors? Yes they did. Did they fail to grasp the strategy of avoiding fights that the humans used? Yes they did. The fact is that even a rank amateur like me (90th percentile when I was playing) wouldn't have made those mistakes. But I still strongly believe that what has been achieved so far is nothing short of extraordinary. Judging by their rate of progress[1] I would not be surprised to see them come back in a few months with these problems figured out, capable of beating the best teams.

[1] - https://twitter.com/OpenAI/status/1037765547427954688

Re: The use of embeddings in OpenAI Five

#20

Post by abdullahkhalids » Wed, Sep 12, 2018, 8:15 AM UTC

Lots of OpenAI Five bashing going on in this thread. I propose a gaming-bot-realism Turing test: when a group of rank 100 or better players cannot discriminate between human teams and the bot team by watching the game, only then are the bots playing "real Dota 2".