Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

More on Dota 2

blog.openai.com

Post reply on HN
Print view View original

41–50 of 106 posts

Re: More on Dota 2

#41

Post by lvoudour »

BaronSamedi wrote: post #34
lvoudour wrote: post #15

I know it has been mentioned a lot the past few days, but since the articles keep flowing about it I'll mention it again: It's a great feat and kudos to the openai team, but it is VERY unfair for the human players who rely on a sensory interface vs a direct API connection. That's unlike chess or go where the interface isn't important. The really impressive feat will be an AI that uses the same sensory information to…

I completely disagree. An API is software's natural input mechanism just like the senses are a human's natural input mechanism. Having the AI use human senses unfairly handicaps it. More importantly, however, is that this is not the key problem. The key problem is teaching the AI strategy and tactics. What heroes to pick? Where to lane them? When to rotate? What items to buy? What spells to level up? What enemies to…

>Compared to these problems, the choice of input mechanism is trivial

I don't know why you think the visual/aural problem is trivial. The biggest achievement of AIs so far in this field is classification of static images

Re: More on Dota 2

#42

Post by hobofan »

lvoudour wrote: post #37
hobofan wrote: post #32

Earlier quoted context omitted.

> sensory-based decision making You can pretty cleanly split that up into two different problems: "sensory-based data extraction" and "data-based decision making" Though I haven't worked in the field of self-driving cars, I am fairly confident that they employ a similar split: One part that takes in all the (pre-processed) data from LIDAR, cameras, etc. and maps that to a simplified model of the surroundings, and ano…

To me though as an outside observer it seems that the dota problem is harder than the car problem. There's too much action going on, too many visual/aural cues to keep track of, team cooperation/coordination, fog of war, etc. Car AI is important because there are real life-or-death consequences, but the problem (again to my limited experience) seems more tractable: path choices are limited, action is rare, there's no…

> there's no team element and there's no competition

I don't think that this is true, considering all the other humans on/near the road, that can influence the system by making actions of their own.

> There's too much action going on, too many visual/aural cues to keep track of, team cooperation/coordination, fog of war, etc.

I dont see how this is different from a car at all.

- too many visual/aural cues to keep track of -> everything you can see, car horns, etc.

- team cooperation/coordination -> as I said everyone else on/near the road

- fog of war -> blind spots & people hidden behind objects

From some experience in the AI world, I can say that I've seen systems that are good enough that they should be able to solve most of those subproblems in the limited system that is Dota. Yes, fog of war might also be interesting given that you have an agressive opponent. The real novel things lie at the strategy level, like picking a hero, buying items, etc., because, like AlphaGo, they demonstrate reasoning in systems with a large action space and delayed payoff.

Re: More on Dota 2

#43

Post by _dps »

BaronSamedi wrote: post #34
lvoudour wrote: post #15

I know it has been mentioned a lot the past few days, but since the articles keep flowing about it I'll mention it again: It's a great feat and kudos to the openai team, but it is VERY unfair for the human players who rely on a sensory interface vs a direct API connection. That's unlike chess or go where the interface isn't important. The really impressive feat will be an AI that uses the same sensory information to…

I completely disagree. An API is software's natural input mechanism just like the senses are a human's natural input mechanism. Having the AI use human senses unfairly handicaps it. More importantly, however, is that this is not the key problem. The key problem is teaching the AI strategy and tactics. What heroes to pick? Where to lane them? When to rotate? What items to buy? What spells to level up? What enemies to…

I'm almost entirely ignorant of DOTA 2, but the coverage suggests to me that it is winning mainly through what I would call "micro" excellence (firing at just the right range, careful maneuvering, and so on). It's clear that this kind of precision control is extremely advantaged by having direct access to the state of the game, rather than having to estimate it in real-time from vision data.

So, to me, based purely on the news coverage, it's not clear that it has learned anything like "superhuman" levels of strategy. We already know that computers have superhuman reaction times and precision calculation abilities, so it seems to me the interesting question is whether an advantage would remain after factoring those out.

Re: More on Dota 2

#44

Post by hobofan »

gpm wrote: post #40
hobofan wrote: post #36

Earlier quoted context omitted.

Very much depends on how you employ the frequency limits. If you say "You can only do 300 actions per minute", the bot could make all 300 actions in the first second if it sees that as optimal, breaking the latency limits. If you however say "You can only do 1 action after waiting 200ms since the last action", the bot will have to choose one optimal action every 200ms, effectively limiting the input latency.

No. Because the bot can choose the next action at the end of the 200ms, with all the information available then, not at the start. Bandwidth limits are typically of the "You can only do 1 action (send 1 packet) 200ms since you sent the last packet", not "You can only send 300 packets per minute".

How about "You have to choose an action at the end of 200ms, with the information at the start"?

At the end of the day we can be nitpicky and speculate all day long, but we won't know for sure what they achieved/didn't achieve unless they publish something more concrete than a blog post that is intentionally written in accessible language, which has the side effect eroding some of the more specific measures that were taken to ensure a level playing field.

Re: More on Dota 2

#45

Post by doikor »

_dps wrote: post #43
BaronSamedi wrote: post #34

Earlier quoted context omitted.

I completely disagree. An API is software's natural input mechanism just like the senses are a human's natural input mechanism. Having the AI use human senses unfairly handicaps it. More importantly, however, is that this is not the key problem. The key problem is teaching the AI strategy and tactics. What heroes to pick? Where to lane them? When to rotate? What items to buy? What spells to level up? What enemies to…

I'm almost entirely ignorant of DOTA 2, but the coverage suggests to me that it is winning mainly through what I would call "micro" excellence (firing at just the right range, careful maneuvering, and so on). It's clear that this kind of precision control is extremely advantaged by having direct access to the state of the game, rather than having to estimate it in real-time from vision data. So, to me, based purely o…

> it is winning mainly through what I would call "micro" excellence

This is partly it. From the interviews with the pros it mostly close to perfect micro, calculating results very well (it knows when it will win the 3 raze spam + 2 autoattacks with 2hp left and thus wins the match etc. for a human that would be hard to calculate) and due to those two it punishes every small mistake the human player makes very hard.

From the actual thinking parts it had to learn creep control (pushing too far just gives free exp/last hits to the other player etc.) and itemization (what counters what, when to get ward etc.)

Re: More on Dota 2

#46

Post by hobofan »

_dps wrote: post #43
BaronSamedi wrote: post #34

Earlier quoted context omitted.

I completely disagree. An API is software's natural input mechanism just like the senses are a human's natural input mechanism. Having the AI use human senses unfairly handicaps it. More importantly, however, is that this is not the key problem. The key problem is teaching the AI strategy and tactics. What heroes to pick? Where to lane them? When to rotate? What items to buy? What spells to level up? What enemies to…

I'm almost entirely ignorant of DOTA 2, but the coverage suggests to me that it is winning mainly through what I would call "micro" excellence (firing at just the right range, careful maneuvering, and so on). It's clear that this kind of precision control is extremely advantaged by having direct access to the state of the game, rather than having to estimate it in real-time from vision data. So, to me, based purely o…

I agree. I don't think there was anything too novel in there, but it is a necessary foundation they need before advancing on to the strategy part, or they would just be out-microed there.

I think there a little bit of strategy already in there since it learned about positioning, and not exposing itself too much (which is of course easier without fog of war). The only thing I was excited to see was the creep blocking, but the chance that this would turn out to be hand-trained was pretty high.

Re: More on Dota 2

#47

Post by bingojess »

lvoudour wrote: post #41
BaronSamedi wrote: post #34

Earlier quoted context omitted.

I completely disagree. An API is software's natural input mechanism just like the senses are a human's natural input mechanism. Having the AI use human senses unfairly handicaps it. More importantly, however, is that this is not the key problem. The key problem is teaching the AI strategy and tactics. What heroes to pick? Where to lane them? When to rotate? What items to buy? What spells to level up? What enemies to…

>Compared to these problems, the choice of input mechanism is trivial I don't know why you think the visual/aural problem is trivial. The biggest achievement of AIs so far in this field is classification of static images

Trivial may not be the best word but what they were trying to get the AI to do was learn dota strategy and tactics. It's like teaching alphaGo to play go with a camera.

Re: More on Dota 2

#48

Post by Fnoord »

lvoudour wrote: post #15

I know it has been mentioned a lot the past few days, but since the articles keep flowing about it I'll mention it again: It's a great feat and kudos to the openai team, but it is VERY unfair for the human players who rely on a sensory interface vs a direct API connection. That's unlike chess or go where the interface isn't important. The really impressive feat will be an AI that uses the same sensory information to…

The comparison goes indeed moot, for a myriad of reasons.

Interface is one of the many.

If we look at a game such as WoW we can observe the fights in group content (dungeons and raids) have become more difficult over time. The WoW development team cites a few reasons for that: 1) players have become better so the difficulty has gone up, and 2) boss mods (software) have become better. This software aids the user in observing/notifying (sensing actions) , and executing (processing information and deciding upon it) mechanics on a fight and are available thanks to the LUA engine and API. Ironically, even without such software, that game has improved majorly in communicating mechanics to the player over the years (WoW is from 2004).

Hence aspects like the UI and API are going to affect the quality of gameplay of humans as well. For AI it would only be API.

Furthermore, there is network lag, interface lag, input lag, and cognitive lag. Only the latter seems fair game to me.

Re: More on Dota 2

#49

Post by damnfine »

gcp wrote: post #11
mannykannot wrote: post #9

Earlier quoted context omitted.

It also says "We also separately trained the initial creep block using traditional RL techniques." I have no idea how significant that is, but it seems to be getting a fair amount of attention.

It's a highly specific procedure that happens before there is interaction with the opponent, so without "handing over" the understanding that having creeps on your high-ground is good, it's very hard for the learning to see through the noise and discover this.

Exactly why this is not impressive to me. The point is to be able to learn the rules, but all I see is some of not only the rules, but the actions already prespecified in many cases. Yes its hard, and thats why humans still rule the roost.
Post reply on HN