Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI bots competing against Humans right now

twitch.tv

Post reply on HN
Print view View original

121–130 of 138 posts

Re: OpenAI bots competing against Humans right now

#121

Post by QML »

nopinsight wrote: post #82

What the cases so far tell us: Once an AI beats humans who have spent 10,000 hours practicing a skill, it is a matter of time before it beats the best professionals in the field. Cases it already happened: board games such as Chess and Go, Poker, diagnostics of certain diseases using medical images Cases where AI is still clearly inferior: video understanding, natural language understanding , motor control esp of han…

Modern AI may "beat" humans at certain tasks, but I doubt they are as efficient as humans are at those tasks: it makes sense that throwing more compute at problem will yield better results. But is it possible to get better results while constraining training time?

Edit: I looked up how much time it takes to train: "OpenAI Five plays 180 years worth of games against itself every day, learning via self-play." [1]

[1] https://blog.openai.com/openai-five/

Re: OpenAI bots competing against Humans right now

#122

Post by wwhitlow »

zaarn wrote: post #90
ufo wrote: post #56

In the last game of the series the bots were forced to play an intentionally terrible hero lineup. The humans finally got to win a game but more interesting for me was that the times when the AI did seemingly crazy things were much more common. I wonder if this is an artifact of the training methodology: maybe if your team is very weak then your choices are also weaker, and reinforcement learning doesn't work as well…

I think this might be a result of "the only winning move is not to play", so to speak. If the game is, in the mind of the AI agents, unwinnable, not playing is not an option, therefore it begins to pick random actions instead. I'm not sure if the AI can surrender (I only managed to watch the first two games as it was rather late at night) but it might be a path to explore; having the AI give up if the game cannot be…

At what percentage would you allow an AI to consider a game unwinnable? While an AI that behaves erratically when the odds are low might be worth allowing it to be considered forfeit worthy, but the thing about humans is we make mistakes. Therefore an ideal AI that can continue to execute reasonable moves should have a lower percentage threshold where it decides to forfeit. See this match[0] for an example of a spectacular comeback that I feel an AI might have considered forfeit worthy if not well defined.

[0]: https://youtu.be/LwSQv_sNZBI

Re: OpenAI bots competing against Humans right now

#123

Post by ionforce »

wwhitlow wrote: post #122
zaarn wrote: post #90

Earlier quoted context omitted.

I think this might be a result of "the only winning move is not to play", so to speak. If the game is, in the mind of the AI agents, unwinnable, not playing is not an option, therefore it begins to pick random actions instead. I'm not sure if the AI can surrender (I only managed to watch the first two games as it was rather late at night) but it might be a path to explore; having the AI give up if the game cannot be…

At what percentage would you allow an AI to consider a game unwinnable? While an AI that behaves erratically when the odds are low might be worth allowing it to be considered forfeit worthy, but the thing about humans is we make mistakes. Therefore an ideal AI that can continue to execute reasonable moves should have a lower percentage threshold where it decides to forfeit. See this match[0] for an example of a spect…

Maybe this is a limitation of self-play. If the opponent an AI faces during training is always optimal, then there's no surface area of mistakes. The losing AI, in its model/mind, knows that the game is over after a specific threshold. So it hasn't learned how to optimize for capitalizing on mistakes.

I wonder if this situation can be fixed by adding more randomness. For example, force AI'1 to be in a losing position to AI'2, but then suddenly switch the power level of AI'2 to be much weaker (where mistakes happen) so that AI'1 learns how to fight its way out of tough situations.

Re: OpenAI bots competing against Humans right now

#124

Post by drusepth »

Rychard wrote: post #68
hackandtrip wrote: post #64

Earlier quoted context omitted.

I'd say that those were crazy things, but not too much. AI understood that teamfight were impossibile to win, and so the really weird choices (sven dying for the tower lol) . I'd say that matches with such lineup are not too much and so they didn't really know, but they did an awesome job and kept things balanced for so long

> really weird choices (sven dying for the tower lol) Taking a tier 2 tower nets everyone on the team 120 gold (a further 150+ gold goes to the hero who gets the last hit), and losing Sven probably gave the opposing team less than was gained. Perhaps the AI simply placed more value on increasing the total net worth of the team than it valued saving the life of one of its core heroes. Additionally, there was no guaran…

Dota player here: other metrics that could have been involved in the trading decision are:

- Potential chance the enemy team would deny the tower before another friendly hero could take it (netting Sven's team 0 gold for the time spent whacking away at it)

- Map vision (removing a T2 often cuts a significant section of map awareness away, since the tower is no longer providing vision or protection)

- XP gains (Sven won't gain any XP while dead, nor from killing the tower)

- Creep equilibrium (this is less important, or at least thought about less often, later on in the game and past T1 towers, but might've been a factor in drawing the creep clash point to a particular location)

- Dictating team net worth averages (to some extent, if they predicted a loss in opponents forcing a teamfight or predicted a likely pickoff, gold lost could be minimized now by taking a death early, lowering the average net worth on the team).

Obviously, there are others and these can also be mixed and matched in various ways (e.g. cutting off map vision so they can more safely farm additional jungle creeps).

Not saying any of these aspects _were_ a part of the decision to trade Sven for a tower, but.. just wanted to include a few more subsurface aspects that _could_ be used in such a decision.

Re: OpenAI bots competing against Humans right now

#125

Post by ionforce »

lawrenceyan wrote: post #46

Expecting 3-0 slam dunk here. Consistently surprised at the ability of tried and true basic reinforcement learning at completing challenging tasks. I totally expected some kind of neccessary breakthrough in the RL field before a real time game like Dota 2 could be beaten.

I would love to know why the third match turned sour. I suspect (as an amateur with no ML background) that that matchup was under-trained.

Like I could imagine OpenAI getting stuck in a subset of the draft pool for which it trained against, like maybe the top 10 of 18 champs. And then picking outside of that meta causes it to fall back on much less robust training/strategy.

Re: OpenAI bots competing against Humans right now

#126

Post by drusepth »

sqrt17 wrote: post #105
akerro wrote: post #104

Earlier quoted context omitted.

>given more practice humans would beat this version of the ai given more practice bots would beat humans. that's the point, train bots, which are faster to train than humans to beat humans.

It's important to keep in mind the exact quantity of "more practice". Current mechanisms of reinforcement learning are not very data-efficient, which means that often humans will learn faster than bots. It will still allow bots to discover any unrealistic advantage they have over humans (e.g. faster micromanagement), but if the game is fair and experience to learn from is limited, humans may still prevail. To "beat"…

To follow this thought, it's also worth pointing out that AIs also have an advantage of parallelization over humans in many cases. What might take an AI 100k hours can often be achieved in ~10k hours in parallel across 10 machines. This is what enables the current system to train over 180 years worth of games every day.

Re: OpenAI bots competing against Humans right now

#127

Post by thomasahle »

yters wrote: post #63

AI winning due to super human abilities is about as impressive as a Counter Strike bot with perfect aim making headshots every time.

Check the DeepMind Quake bot: https://deepmind.com/blog/capture-the-flag/

Even when reducing tagging accuracy to below human level, they still performed better.

Re: OpenAI bots competing against Humans right now

#128

Post by zackaman »

ionforce wrote: post #125
lawrenceyan wrote: post #46

Expecting 3-0 slam dunk here. Consistently surprised at the ability of tried and true basic reinforcement learning at completing challenging tasks. I totally expected some kind of neccessary breakthrough in the RL field before a real time game like Dota 2 could be beaten.

I would love to know why the third match turned sour. I suspect (as an amateur with no ML background) that that matchup was under-trained. Like I could imagine OpenAI getting stuck in a subset of the draft pool for which it trained against, like maybe the top 10 of 18 champs. And then picking outside of that meta causes it to fall back on much less robust training/strategy.

Because the first two matches were so lopsided, the bot lineup was selected by twitch chat + audience members. We drafted them a pretty terrible lineup, and from the start the bots estimated their chance of winning to be about 2.9%.

Re: OpenAI bots competing against Humans right now

#129

Post by mellinoe »

zackaman wrote: post #128
ionforce wrote: post #125

Earlier quoted context omitted.

I would love to know why the third match turned sour. I suspect (as an amateur with no ML background) that that matchup was under-trained. Like I could imagine OpenAI getting stuck in a subset of the draft pool for which it trained against, like maybe the top 10 of 18 champs. And then picking outside of that meta causes it to fall back on much less robust training/strategy.

Because the first two matches were so lopsided, the bot lineup was selected by twitch chat + audience members. We drafted them a pretty terrible lineup, and from the start the bots estimated their chance of winning to be about 2.9%.

Indeed. And to explain further: not all hero combinations are equal. Meaning: you cannot select any arbitrary set of 5 heroes and expect them to perform well. Different heroes have different strengths and synergies that make them stronger or weaker depending on the specific teammates and opponents that are present. This is why drafting is considered such an important (and difficult) portion of the game. In match 3, a purposefully-bad team was selected. It would have been VERY impressive if it was able to win.

Re: OpenAI bots competing against Humans right now

#130

Post by red75prime »

QML wrote: post #121
nopinsight wrote: post #82

What the cases so far tell us: Once an AI beats humans who have spent 10,000 hours practicing a skill, it is a matter of time before it beats the best professionals in the field. Cases it already happened: board games such as Chess and Go, Poker, diagnostics of certain diseases using medical images Cases where AI is still clearly inferior: video understanding, natural language understanding , motor control esp of han…

Modern AI may "beat" humans at certain tasks, but I doubt they are as efficient as humans are at those tasks: it makes sense that throwing more compute at problem will yield better results. But is it possible to get better results while constraining training time? Edit: I looked up how much time it takes to train: "OpenAI Five plays 180 years worth of games against itself every day, learning via self-play." [1] [1] h…

If you can get hundred years experience in 24 hours by wall clock, why not use it? Maybe it's better to teach AIs to create simplified, but usable models, they can use to simulate and train on.
Post reply on HN