Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

More on Dota 2

blog.openai.com

Post reply on HN
Print view View original

51–60 of 106 posts

Re: More on Dota 2

#51

Post by mirceal »

bingojess wrote: post #47
lvoudour wrote: post #41

Earlier quoted context omitted.

>Compared to these problems, the choice of input mechanism is trivial I don't know why you think the visual/aural problem is trivial. The biggest achievement of AIs so far in this field is classification of static images

Trivial may not be the best word but what they were trying to get the AI to do was learn dota strategy and tactics. It's like teaching alphaGo to play go with a camera.

I disagree. The representation of state is hard for machines. Go/Chess are very simple state wise when compared to Dota. By giving it API access you have eliminated guesswork/approximations from the overall picture

Re: More on Dota 2

#52

Post by Epokhe »

noway421 wrote: post #10
NamTaf wrote: post #5

Earlier quoted context omitted.

Reinforcement learning isn't a new idea - I did a Berkeley-based edX course on it a few years ago now and it was not state-of-the-art to my knowledge. That had no deep aspect to it, we just generated a reinfrocement algorithm that utilised a good measure of performance (specifically, it was pacman and the score value is pretty good at that) and changed a few algorithm weighting variables at each iteration. My underst…

Why wouldn't algorithm reach a local maxima when playing with itself, or even degrade over time by opening up to unknown attacks?

it does. the bot can be defeated easily with tactics it didn't encounter while playing with itself.

Re: More on Dota 2

#53

Post by gpm »

hobofan wrote: post #44
gpm wrote: post #40

Earlier quoted context omitted.

No. Because the bot can choose the next action at the end of the 200ms, with all the information available then, not at the start. Bandwidth limits are typically of the "You can only do 1 action (send 1 packet) 200ms since you sent the last packet", not "You can only send 300 packets per minute".

How about "You have to choose an action at the end of 200ms, with the information at the start"? At the end of the day we can be nitpicky and speculate all day long, but we won't know for sure what they achieved/didn't achieve unless they publish something more concrete than a blog post that is intentionally written in accessible language, which has the side effect eroding some of the more specific measures that were…

If they did than then they are simulating latency, but I see nothing to suggest they did that.

Re: More on Dota 2

#54

Post by pycal »

CoffeeBob wrote: post #50

Does anyone else have a problem with the line, "the graph is surprisingly linear, meaning the team improved the bot exponentially over time"?

I think what they're pointing to is that the ELO system "true skill" that dota uses is a log normal distribution. To your point I'm not sure that means that player skill improves along the distribution, but I think it does mean their probability of winning increases exponentially.

Re: More on Dota 2

#55

Post by pycal »

omarforgotpwd wrote: post #4

I’m by no means an expert, but I’m fascinated by the idea that a neural net playing against itself can substantially outperform a supervised learning approach with a large training data set. I mean, gathering training data and making sure it’s labeled correctly and all that is a huge hassle so if you could eliminate that step or even reduce the amount or quality of training data required that should be a big win for…

Just a guess, but wouldn't this approach used in isolation result in a high potential for getting stuck at local optima? It seems to me that the success of this experiment is the result of interaction with players in different MMR brackets.

Re: More on Dota 2

#56

Post by skgoa »

_dps wrote: post #43
BaronSamedi wrote: post #34

Earlier quoted context omitted.

I completely disagree. An API is software's natural input mechanism just like the senses are a human's natural input mechanism. Having the AI use human senses unfairly handicaps it. More importantly, however, is that this is not the key problem. The key problem is teaching the AI strategy and tactics. What heroes to pick? Where to lane them? When to rotate? What items to buy? What spells to level up? What enemies to…

I'm almost entirely ignorant of DOTA 2, but the coverage suggests to me that it is winning mainly through what I would call "micro" excellence (firing at just the right range, careful maneuvering, and so on). It's clear that this kind of precision control is extremely advantaged by having direct access to the state of the game, rather than having to estimate it in real-time from vision data. So, to me, based purely o…

As someone who plays dota, this bot really isn't that impressive. The hero they chose is seen as a very difficult one to master for humans, precisely because judging distances, current life, current mana, damage etc. is so difficult to do on the fly and even if you can keep track of all of that in your head, you need extremely precise inputs to outplay your opponent. Yet by going through the API, they handed all of that to their bot on a silver platter. They pretty much let the bot sidestep the core challenge of this hero, while it was kept in place for the human players.

On top of that they also reduced the complexity of the game quite significantly by limiting items etc., which further reduced what humans could do against the bot. Even then, the bot utterly failed once humans were allowed to use a tiny bit of creativity.

And that's not even taking into account that this was not even close to the complexity of a real dota match. The big challenge in dota is in the decision making with incomplete information and in coordinating 5 people with only voice and the ability to ping the mini map, in a giant "search space" created by hundreds of different heroes, items and game mechanic interactions.

Re: More on Dota 2

#58

Post by grimskin »

_dps wrote: post #43
BaronSamedi wrote: post #34

Earlier quoted context omitted.

I completely disagree. An API is software's natural input mechanism just like the senses are a human's natural input mechanism. Having the AI use human senses unfairly handicaps it. More importantly, however, is that this is not the key problem. The key problem is teaching the AI strategy and tactics. What heroes to pick? Where to lane them? When to rotate? What items to buy? What spells to level up? What enemies to…

I'm almost entirely ignorant of DOTA 2, but the coverage suggests to me that it is winning mainly through what I would call "micro" excellence (firing at just the right range, careful maneuvering, and so on). It's clear that this kind of precision control is extremely advantaged by having direct access to the state of the game, rather than having to estimate it in real-time from vision data. So, to me, based purely o…

> it's not clear that it has learned anything like "superhuman" levels of strategy

But still, it's learned "human" levels of strategy, and that actually amazing. And let's not forget, that this is only beginning, more stuff to come. IIRC - some third-party StarCraft AI actually invented a trick that was later used by human players on pro-level tournaments.

Re: More on Dota 2

#59

Post by dumbmatter »

lvoudour wrote: post #15

I know it has been mentioned a lot the past few days, but since the articles keep flowing about it I'll mention it again: It's a great feat and kudos to the openai team, but it is VERY unfair for the human players who rely on a sensory interface vs a direct API connection. That's unlike chess or go where the interface isn't important. The really impressive feat will be an AI that uses the same sensory information to…

Yeah, it's like Watson on Jeopardy. It could buzz in faster than a human, so it got to answer every question that it thought it knew, which was most of them. Problem is, the human competitors surely also knew most of them (maybe even more than Watson) but they couldn't buzz in fast enough.

Re: More on Dota 2

#60

Post by [deleted] »

BaronSamedi wrote: post #34
lvoudour wrote: post #15

I know it has been mentioned a lot the past few days, but since the articles keep flowing about it I'll mention it again: It's a great feat and kudos to the openai team, but it is VERY unfair for the human players who rely on a sensory interface vs a direct API connection. That's unlike chess or go where the interface isn't important. The really impressive feat will be an AI that uses the same sensory information to…

I completely disagree. An API is software's natural input mechanism just like the senses are a human's natural input mechanism. Having the AI use human senses unfairly handicaps it. More importantly, however, is that this is not the key problem. The key problem is teaching the AI strategy and tactics. What heroes to pick? Where to lane them? When to rotate? What items to buy? What spells to level up? What enemies to…

[deleted]
Post reply on HN