Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI bots competing against Humans right now

twitch.tv

Post reply on HN
Print view View original

111–120 of 138 posts

Re: OpenAI bots competing against Humans right now

#111

Post by kolinko »

shagie wrote: post #70
moogly wrote: post #67

Earlier quoted context omitted.

There is 0 mechanical skill involved in chess. Reflexes don't matter. There is mechanical skill involved in Dota 2, and reflexes matter. And consider a game like Quake where mechanical skill is even more important (even though mechanical skill matters, Dota 2 is still primarily a game about strategy and team coordination).

I would encourage you to watch this APM demonstration https://www.youtube.com/watch?v=YbpCLqryN-Q , give https://www.engadget.com/2014/10/24/starcraft-2-and-the-ques... a read (600 APM) and consider how much of an advantage the computer has.

APM matters less in Dota than in Starcraft

Re: OpenAI bots competing against Humans right now

#112

Post by fenomas »

yvdriess wrote: post #108
fenomas wrote: post #84

Earlier quoted context omitted.

The relevant question is surely how much of the game is strategy, and how much is reaction time and twitch motor skills. I mean, "human vs AI" matchups are ostensibly about strategy - machines already win at timing and twitch, there's nothing to test. But esports games aren't pure strategy, they all involve various amounts of timing, twitch skills, the ability to monitor lots of details at once, etc. Those are all th…

Dota is a bit different from other esports such as SC2, in that it leans much more on game-sense and decision making than twitch skills[1]. Nevertheless, OpenAI dulled the reaction time artificially to be more human-like. It makes sense if the goal is to make a sophisticated strategic/gamesense AI, and not one that wins by just executing better. [1] For an entertaining case-study, check out Day[9]'s learns DotA2 seri…

Oh, that's interesting about the reaction time. Not having played DotA I have no idea how big an issue it is (and I couldn't make heads or tails of the video :D ).

It occurs to me that for a really even playing field, the humans should probably be allowed to make and install UI mods if they want to. E.g. if there's an advantage to using an ability precisely when your hit points hit 50% (or whatever), an AI can easily do that reliably so the human should probably be able to if they want to.

(Of course, for heavily twitch games like Counterstrike, being allowed to use UI mods (i.e. aimbots) would break things. But then, I suppose that the extent to which UI mods break a game is more or less the extent to which that game favors twitch over strategy.)

Re: OpenAI bots competing against Humans right now

#113

Post by king07828 »

dojomouse wrote: post #89
king07828 wrote: post #74

Earlier quoted context omitted.

Agreed that the strategy is an interesting part. Another interesting part will be creating an AI / neural network that can utilize inputs that are closer to human level inputs (e.g., using the frame buffer and audio out as input to the neural network and passing the outputs of the neural network to a keyboard and mouse driver). Just let the network train itself without having a human laboriously determine the topolog…

The main reasons they don't do this are that it's a fairly known quantity from an ML perspective (going from sequences of images to representational features), so wouldn't be proving that much to be able to do (c.f. the various Atari benchmarks which adequately learned actions to achieve rewards working with pixel inputs)... but at the same time would consume a huge fraction of the computer resource they really want…

Agreed.

One of the next significant moments could be taking the current Dota 2 algorithm and massaging it to use human style inputs and outputs. Please correct if needed, but the current Dota 2 algorithm boils down to (1) a fully connected network that generates an input state vector from the Dota 2 bot output interface, (2) an LSTM of sufficient length that generates an output state vector from the input state vector, and (3) another fully connected network that generates the Dota 2 bot interface inputs from the output state vector. This could be updated to have (1a) a convolutional network that feeds into a fully connected network, where the input to the convolutional network is the frame buffer (and perhaps the audio output) and the output of the fully connected network is the input state vector, (2) the same or similar LSTM network, and (3a) a fully connected network that outputs keyboard and mouse commands instead of DotA 2 bot interface inputs.

It is an open question as to whether current compute power is sufficient for this massage.

Re: OpenAI bots competing against Humans right now

#114

Post by douglaswlance »

asdfasgasdgasdg wrote: post #71
douglaswlance wrote: post #69

Earlier quoted context omitted.

Humans tend to progress by learning from each other. The AI will teach us.

Whether the AI teaches us or not, once it surpasses us, we will not catch up. This is the case for every game AI that has ever surpassed human performance so far, and there's no reason to expect that this will be different in the case of DotA.

Depends how you define, "surpass."

There will be a time where we learn from the AI and the AI learns from us, where we trade victories and defeats as we adapt to each other.

Don't discount the ability of humans. They figured out how to exploit the 1v1 bot in a few days and soon humans had a 100% win rate using that strategy.

Re: OpenAI bots competing against Humans right now

#115

Post by skgoa »

yvdriess wrote: post #106
skgoa wrote: post #95

Earlier quoted context omitted.

The bots also have perfect knowledge of enemy hp/mana and the relative positioning through the API. In fact the bots are gifted perfect micromanagement through the API. This enables them to do things most humeans wouldn't try, because the risk of messing up and giving the enemy a huge advantage is too high.

The API does not give the bots extra information compared to a human player. The micro- and reaction time edge is also being dulled to being more human-like. They still have superhuman team fight execution though. "We’ve increased the reaction time of OpenAI Five from 80ms to 200ms. This reaction time is much closer to human level, though we haven’t seen evidence of changes in gameplay as OpenAI Five’s strength comes…

> The API does not give the bots extra information compared to a human player.

Well, maybe not technically, but it does make it much much easier to take in all of that info and process it. You can't expect a human player to keep a perfect record of all heroes' hp, mana, all damage being dealt, all abilities being used etc. during a chaotic fight, yet the API yields this information effortlessly.

> The micro- and reaction time edge is also being dulled to being more human-like.

And yet, the bots showed superhuman near instant reaction times. 200ms is a very low amount to process very complex/confusing audiovisual data and react precisely.

Re: OpenAI bots competing against Humans right now

#116

Post by skgoa »

Super_Jambo wrote: post #110
yvdriess wrote: post #106

Earlier quoted context omitted.

The API does not give the bots extra information compared to a human player. The micro- and reaction time edge is also being dulled to being more human-like. They still have superhuman team fight execution though. "We’ve increased the reaction time of OpenAI Five from 80ms to 200ms. This reaction time is much closer to human level, though we haven’t seen evidence of changes in gameplay as OpenAI Five’s strength comes…

Part of the advantage in teamwork and coordination is presumably that they don't have a limitation on what data they can view at once? Dota & HON had people mod their client to give an optional bigger FOV resulting in bans for cheating. I'd assume the bots don't have to specify their screen position, plus no orientation response means a limitation on this wouldn't be meaningful anyway. What I'm saying is there's a bi…

Correct, the bots "see" the entire map. Well, the parts that are not hidden by 'fog of war'.

Re: OpenAI bots competing against Humans right now

#117

Post by king07828 »

rhaps0dy wrote: post #79
king07828 wrote: post #74

Earlier quoted context omitted.

Agreed that the strategy is an interesting part. Another interesting part will be creating an AI / neural network that can utilize inputs that are closer to human level inputs (e.g., using the frame buffer and audio out as input to the neural network and passing the outputs of the neural network to a keyboard and mouse driver). Just let the network train itself without having a human laboriously determine the topolog…

>Just let the network train itself without having a human laboriously determine the topology of the neural network I hope this little koan illustrates that this sentence is impossible to execute. The human always has to specify something. -- In the days when Sussman was a novice, Minsky once came to him as he sat hacking at the PDP-6. "What are you doing?", asked Minsky. "I am training a randomly wired neural net to…

Agreed that without any constraints, it could become a Sisyphean task.

The exercise then becomes one of finding the minimal constraints needed to achieve the desired results. Please correct if needed, but looking at the Dota 2 neural network [1], it boils down to generating an input State vector from the Dota 2 bot output interface, running the state Vector through an lstm (of sufficient length) to generate an output State vector, and generating the inputs for the Dota 2 bot input interface from the output State vector. Update this network (1) to have the input State Vector generated from a convolutional network that feeds a fully connected Network and uses the frame buffer as input and (2) to have the final outputs of the neural network be keyboard and mouse commands instead of dota 2 bot input interface commands, then let the network train itself. The number of elements in the state vector, the number of convolutional layers, the number of lstm layers, and the number of layers and elements in each fully connected hidden layer could each also be determined by a recurrent neural network.

[1] https://towardsdatascience.com/the-science-behind-openai-fiv... (see the image under "The Architecture")

[ random capitalization powered by Google speech dictation ]

Re: OpenAI bots competing against Humans right now

#118

Post by nopinsight »

tux3 wrote: post #58
ufo wrote: post #56

In the last game of the series the bots were forced to play an intentionally terrible hero lineup. The humans finally got to win a game but more interesting for me was that the times when the AI did seemingly crazy things were much more common. I wonder if this is an artifact of the training methodology: maybe if your team is very weak then your choices are also weaker, and reinforcement learning doesn't work as well…

It reminds me of the Go AIs going on tilt when they're far behind. When the win percentage for Go AIs gets to around 5%, every action it can take results in a losing game so it can't make the difference between normal play and super strange moves anymore. When every choice is really bad, humans tend to still go with their normal strategy and wait for their chance to turn things around, but bots assume the opponent is…

If this is true (which it might be, I haven’t studied these systems in detail), then an obvious fix is to get the AI to randomly train with ‘weaker’ versions of itself (perhaps while the strong instance is handicapped) in addition to the latest generation.

Several levels of weak opponents should be used, with varying probabilities, to tune the AI’s robustness against real-world, imperfect competitors.

Re: OpenAI bots competing against Humans right now

#119

Post by stale2002 »

andrei_says_ wrote: post #73
stale2002 wrote: post #57

Earlier quoted context omitted.

I mean, interacting with the mouse and keyboard isn't the interesting part. Like, imagine if this was a chess AI, and we were trying to determine who was better at chess, humans or AI. Would you make the AI use robotic hands to move the pieces? No, because thats not the interesting part of chess. The interesting part of chess is the strategy.

Interacting with the mouse and keyboard at this speed and precision is absolutely a very interesting part for me. It is a limitation of he human organism. What if we could pit humans and ai bots at the speed of human imagination?

It could be interesting, but playing DotA just isn't the best problem to test this problem on.

One could imagine a much better way of testing hand eye coordination, through a serious of mazes or puzzles or reaction tests.

It would be like trying to test hand key coordination by having a robot play physical chess against a person.

Re: OpenAI bots competing against Humans right now

#120

Post by komali2 »

skgoa wrote: post #115
yvdriess wrote: post #106

Earlier quoted context omitted.

The API does not give the bots extra information compared to a human player. The micro- and reaction time edge is also being dulled to being more human-like. They still have superhuman team fight execution though. "We’ve increased the reaction time of OpenAI Five from 80ms to 200ms. This reaction time is much closer to human level, though we haven’t seen evidence of changes in gameplay as OpenAI Five’s strength comes…

> The API does not give the bots extra information compared to a human player. Well, maybe not technically, but it does make it much much easier to take in all of that info and process it. You can't expect a human player to keep a perfect record of all heroes' hp, mana, all damage being dealt, all abilities being used etc. during a chaotic fight, yet the API yields this information effortlessly. > The micro- and reac…

I think you're just kind of hitting on the real difference between human and bots anyway.

Regardless of if the information comes from the machine viewing the damage count and knowing exactly how much HP a given hero has at that level/gear/just by looking at the bar, or if the information comes from an API, the machine has a perfect memory of this and all other variables, whereas humans don't.

Post reply on HN