Live data from Hacker News

OpenAI bots competing against Humans right now

twitch.tv

41–50 of 138 posts

Re: OpenAI bots competing against Humans right now

#43
post #40

If there are any OpenAI folks hanging out here, I'd be really curious to hear about the (apparent) tactical pause by the AI in game 2.

I'd speculate it's the element of surprise.

Ideally the pause would do nothing to the game. (It could be that the game has some glitches but let's assume it's solid.) Then one could assume that the pause does nothing regarding the AI players. But that is not true because the AI itself is not paused (I assume) and it has to handle this unusual input. This might throw off some versions of it and it probably learns how to handle the pause and how to use it to gain a small advantage. So it's all part of the game. Similar to when a human team uses the pause for tactical purposes.

Re: OpenAI bots competing against Humans right now

#45
My thoughts on the AI's performance so far knowing a bit about both DOTA and machine learning:

- Item-usage for things such as smoke and wards (which were recently added to their reportoire) are not well captured by the bots yet. And the buying of wards were confirmed as a scripted event. It seems hard for them to capture the long term sparse reward of these. Smoke might not be needed by a perfect agent, but wards should be. The developer interview noted that it's not very clear what the reward for an agent warding even should be.

- Some of the big advantages OpenAI gains are in team fights, where it's 200ms reaction time (upped from 80ms to resemble human reaction time more) still strikes me as something that tilts it into a solid mechanical advantage. On several occasions OpenAI's Lion managed to disable a human player performing (what I assume to be) a move which shouldn't be interruptable. (blink-->shift+ctrl ultimate ability on earthshaker) This could have tilted teamfights in the human team's favor a few times if it wasn't stopped by the "machine-like" reflexes of OpenAI.

- Positioning before team fights by OpenAI are scary. There are very few openings, and every individual agent is protected by its team.

- Likewise is the map movement scary most of the time. Being able to recall just enough agents back to defend while simultaneously taking out strategic objectives of the human team. Also Blitz(caster) has noted earlier OpenAI's ability to focus on the winnable lanes and sacrifice the others, prioritizing well. (and exploiting map mechanics unknown to most pros until a few years ago)

- When the AI takes down a tier 1 tower, they seem to be very quick to take down the remaining tier 1 towers, instantly capitalizing on their map control advantage, and expanding it.

Some interesting things / bugs:

- Sniper bot throwing multiple spells on the same location right away (even though the damage doesn't stack) effectively simply wasting his mana and cooldowns, for no gain.

- Sniper using his ultimate ability to pressure the lower hp characters of the human team continously. Usually it's more often used more as a finisher, and might be an artifact that appears from the AI having access to an unkillable courier that ferries a lot of healing/mana regenerating items.

Some additional info learned from the interview of some of the devs:

- Incentivizing killing Roshan (a boss character in the middle of the map, which yields a one-time ressurection item for one player, after being killed) is done by varying roshan's HP down to a really low amount, making sure the AI experiences the upside to this. Otherwise it would require all 5 agents gathering there, expending their magical abilities and investing a lot before actually seeing a reward. (which is unlikely to happen)

- Game length in self-play sessions are above 60 minutes around 1% of the time.

Re: OpenAI bots competing against Humans right now

#46
Expecting 3-0 slam dunk here. Consistently surprised at the ability of tried and true basic reinforcement learning at completing challenging tasks. I totally expected some kind of neccessary breakthrough in the RL field before a real time game like Dota 2 could be beaten.

Re: OpenAI bots competing against Humans right now

#48
post #47

Only one LSTM per bot so there is no "master strategy" neural network. Interesting, if I'm not wrong, bots are learning independently.

During training they were given access to each other’s reward functions, and the extent to which they weighted total team reward over their own was gradually increased.

Re: OpenAI bots competing against Humans right now

#49
post #34

Very impressed with the bots. The main strategy I see from the bots that is different from pro meta is how they spam abilities aggressively. Now the main limitation is 5 unkillable couriers which enable this. Watching, I do feel like given more practice humans would beat this version of the ai easily. It looks exploitable, and still having many rules limitations it has a ways to go. It will win in the end though, tha…

It would be really interesting to see what bots will do once humans will try to exploit this strategy, like adapting to the lane movement they always do after winning a lane etc, humans did that once today but it may be used better. Excited for the TI to see less limitation, a better shop system (lot of money throw away) and maybe better ward.

The next AI will always be much better (see AlphaGo progression).

Humans, not so much, as in all top-level competitions, human abilities improve minimally at the top, because we have millions of humans competing against each other until the plateau of human performance is reached. Then you can push that a bit more with drugs (see doping in sports). And after that, you are pretty much done.

So it's only a matter of time and effort until AIs are fully unbeatable.

Re: OpenAI bots competing against Humans right now

#50
post #49

Earlier quoted context omitted.

It would be really interesting to see what bots will do once humans will try to exploit this strategy, like adapting to the lane movement they always do after winning a lane etc, humans did that once today but it may be used better. Excited for the TI to see less limitation, a better shop system (lot of money throw away) and maybe better ward.

The next AI will always be much better (see AlphaGo progression). Humans, not so much, as in all top-level competitions, human abilities improve minimally at the top, because we have millions of humans competing against each other until the plateau of human performance is reached. Then you can push that a bit more with drugs (see doping in sports). And after that, you are pretty much done. So it's only a matter of ti…

I agree that AIs will eventually win. But I only consider human beings beaten when the AI is interfacing with the machine the same way as humans -- looking at the monitor, and inputting commands through keyboard and mouse. That is on a different level than just hooking into a log of events and calculating your next move. Given enough time humans would do that better than the AI imo
Post reply on HN