Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI bots competing against Humans right now

twitch.tv

Post reply on HN
Print view View original

101–110 of 138 posts

Re: OpenAI bots competing against Humans right now

#101

Post by nopinsight »

badpun wrote: post #93
nopinsight wrote: post #82

What the cases so far tell us: Once an AI beats humans who have spent 10,000 hours practicing a skill, it is a matter of time before it beats the best professionals in the field. Cases it already happened: board games such as Chess and Go, Poker, diagnostics of certain diseases using medical images Cases where AI is still clearly inferior: video understanding, natural language understanding , motor control esp of han…

> Poker This is a misnomer. The only variant of Poker where AI beats humans is heads-up (two-player) variant, which is simplest form of poker and also rarely played. The AI was (marginally) beating humans there by playing a game-theory-optimal strategy. For poker games with 3+ players, the GTO strategy (Nash equilibrium) no longer exists, so AIs need to use more standard techniques (search-based, reinforcement learni…

Thanks for the clarification. I do not know much about poker.

The observation applies to a given variant of poker (or any other domain). So if an AI beats humans with 10000-hour experience in that variant, the best experts in that specific variant are not far-off targets.

Superficially similar problems might in fact require very different techniques to solve as your example illustrates.

Re: OpenAI bots competing against Humans right now

#102

Post by hsikka »

hsikka wrote: post #88
nopinsight wrote: post #82

What the cases so far tell us: Once an AI beats humans who have spent 10,000 hours practicing a skill, it is a matter of time before it beats the best professionals in the field. Cases it already happened: board games such as Chess and Go, Poker, diagnostics of certain diseases using medical images Cases where AI is still clearly inferior: video understanding, natural language understanding , motor control esp of han…

I actually have an intuition around using modular neural networks with dynamic topology to tackle more complex disease cases, general medicine, and other complex, hierarchical problems. I'm working towards trying to use them in my thesis for school

If anyone has any reading suggestions or research I should look into, I'm all ears

Re: OpenAI bots competing against Humans right now

#103

Post by sethgecko »

lma21 wrote: post #87
drexlspivey wrote: post #24

Earlier quoted context omitted.

You don't need to update any weights, the model is already trained.

Ignorant here. How do you take an already trained model and execute it elsewhere? Isn’t the training phase part of the whole (ongoing) simulation?

The training phase is when you are trying to minimise your loss function by trying different weights each iteration.

Once that is finished, you have a trained model which you can use by providing input and getting an output (with the weights frozen).

The training phase is very expensive computationally because you have to calculate the gradient of your loss function on potentially huge tensors.

The execution phase is not that expensive and commercial laptops will be likely able to run the model without any problems.

Re: OpenAI bots competing against Humans right now

#104

Post by akerro »

kmnc wrote: post #34

Very impressed with the bots. The main strategy I see from the bots that is different from pro meta is how they spam abilities aggressively. Now the main limitation is 5 unkillable couriers which enable this. Watching, I do feel like given more practice humans would beat this version of the ai easily. It looks exploitable, and still having many rules limitations it has a ways to go. It will win in the end though, tha…

>given more practice humans would beat this version of the ai

given more practice bots would beat humans. that's the point, train bots, which are faster to train than humans to beat humans.

Re: OpenAI bots competing against Humans right now

#105

Post by sqrt17 »

akerro wrote: post #104
kmnc wrote: post #34

Very impressed with the bots. The main strategy I see from the bots that is different from pro meta is how they spam abilities aggressively. Now the main limitation is 5 unkillable couriers which enable this. Watching, I do feel like given more practice humans would beat this version of the ai easily. It looks exploitable, and still having many rules limitations it has a ways to go. It will win in the end though, tha…

>given more practice humans would beat this version of the ai given more practice bots would beat humans. that's the point, train bots, which are faster to train than humans to beat humans.

It's important to keep in mind the exact quantity of "more practice". Current mechanisms of reinforcement learning are not very data-efficient, which means that often humans will learn faster than bots. It will still allow bots to discover any unrealistic advantage they have over humans (e.g. faster micromanagement), but if the game is fair and experience to learn from is limited, humans may still prevail.

To "beat" Atari games, AIs trained using reinforcement learning had to put in significantly more than the 10k hours one would expect a human to put in in order to become expert. So, AIs won't beat humans in tasks where training data is costly; however these cases are not interesting to researchers and hence you won't hear from the respective results.

Re: OpenAI bots competing against Humans right now

#106

Post by yvdriess »

skgoa wrote: post #95
kmnc wrote: post #34

Very impressed with the bots. The main strategy I see from the bots that is different from pro meta is how they spam abilities aggressively. Now the main limitation is 5 unkillable couriers which enable this. Watching, I do feel like given more practice humans would beat this version of the ai easily. It looks exploitable, and still having many rules limitations it has a ways to go. It will win in the end though, tha…

The bots also have perfect knowledge of enemy hp/mana and the relative positioning through the API. In fact the bots are gifted perfect micromanagement through the API. This enables them to do things most humeans wouldn't try, because the risk of messing up and giving the enemy a huge advantage is too high.

The API does not give the bots extra information compared to a human player. The micro- and reaction time edge is also being dulled to being more human-like. They still have superhuman team fight execution though.

"We’ve increased the reaction time of OpenAI Five from 80ms to 200ms. This reaction time is much closer to human level, though we haven’t seen evidence of changes in gameplay as OpenAI Five’s strength comes more from teamwork and coordination than reflexes."

Re: OpenAI bots competing against Humans right now

#107

Post by richardknop »

21 wrote: post #51
darepublic wrote: post #50

Earlier quoted context omitted.

I agree that AIs will eventually win. But I only consider human beings beaten when the AI is interfacing with the machine the same way as humans -- looking at the monitor, and inputting commands through keyboard and mouse. That is on a different level than just hooking into a log of events and calculating your next move. Given enough time humans would do that better than the AI imo

Come on, do you think hooking a robot to the keyboard is the hard part? Do you want 5 fingers too? And if watching the screen, do you want it to have bad eyes like we do too (good resolution only in the center)?

Yes it is difficult. Under current conditions AI has access to much more data than humans. So it is unfair battle, it would be fair comparison if AI was programmed in such a way to have more of an analogue input of data similar to human eyes looking at a monitor and also analogue input of actions similar to human pressing keyboard keys which is much more inefficient as AI just calling internal functions to execute actions.

Re: OpenAI bots competing against Humans right now

#108

Post by yvdriess »

fenomas wrote: post #84
21 wrote: post #51

Earlier quoted context omitted.

Come on, do you think hooking a robot to the keyboard is the hard part? Do you want 5 fingers too? And if watching the screen, do you want it to have bad eyes like we do too (good resolution only in the center)?

The relevant question is surely how much of the game is strategy, and how much is reaction time and twitch motor skills. I mean, "human vs AI" matchups are ostensibly about strategy - machines already win at timing and twitch, there's nothing to test. But esports games aren't pure strategy, they all involve various amounts of timing, twitch skills, the ability to monitor lots of details at once, etc. Those are all th…

Dota is a bit different from other esports such as SC2, in that it leans much more on game-sense and decision making than twitch skills[1]. Nevertheless, OpenAI dulled the reaction time artificially to be more human-like. It makes sense if the goal is to make a sophisticated strategic/gamesense AI, and not one that wins by just executing better.

[1] For an entertaining case-study, check out Day[9]'s learns DotA2 series.

Re: OpenAI bots competing against Humans right now

#109

Post by jjjjjjjjjjjjjjj »

NegatioN wrote: post #45

My thoughts on the AI's performance so far knowing a bit about both DOTA and machine learning: - Item-usage for things such as smoke and wards (which were recently added to their reportoire) are not well captured by the bots yet. And the buying of wards were confirmed as a scripted event. It seems hard for them to capture the long term sparse reward of these. Smoke might not be needed by a perfect agent, but wards sh…

> Lion managed to disable a human player performing (what I assume to be) a move which shouldn't be interruptable

Fogged (the human player) commented on this and said he messed up. If he had shift-queued the spell, or just used it immediately after blink, it would've landed [0].

Cancelling an initiation with instant spells (Lion hex, Rubick lift, etc) does happen frequently in high level human play as well, where you continuously pre-cast the spell, cancel, walk back slightly, repeat, on the out-of-range initiating enemy, to have the spell interrupt the initiation as soon as the initiating enemy blinks in to range. I do agree that the bots have a solid mechanical advantage, just pointing out that this specific scenario does frequently happen in human play as well (albeit not on every single initiation).

[0] https://www.reddit.com/r/DotA2/comments/94vdpm/openai_hex_wa...

Re: OpenAI bots competing against Humans right now

#110

Post by Super_Jambo »

yvdriess wrote: post #106
skgoa wrote: post #95

Earlier quoted context omitted.

The bots also have perfect knowledge of enemy hp/mana and the relative positioning through the API. In fact the bots are gifted perfect micromanagement through the API. This enables them to do things most humeans wouldn't try, because the risk of messing up and giving the enemy a huge advantage is too high.

The API does not give the bots extra information compared to a human player. The micro- and reaction time edge is also being dulled to being more human-like. They still have superhuman team fight execution though. "We’ve increased the reaction time of OpenAI Five from 80ms to 200ms. This reaction time is much closer to human level, though we haven’t seen evidence of changes in gameplay as OpenAI Five’s strength comes…

Part of the advantage in teamwork and coordination is presumably that they don't have a limitation on what data they can view at once?

Dota & HON had people mod their client to give an optional bigger FOV resulting in bans for cheating.

I'd assume the bots don't have to specify their screen position, plus no orientation response means a limitation on this wouldn't be meaningful anyway. What I'm saying is there's a big difference between 'I see lion on the mini-map down there' and 'lion showed for 1 frame on the other side of the map, his HP is 324, he has a TP scroll, no boots and a health pot.'

Something I noticed is the bots seem to like range and AoE far more than the normal human meta. The humans being limited in the distance they can see to one screen were frequently just failing to appreciate how dangerous 2-3 bots half a screen away were to them.

Quite a few teamfight wins came from the bots inevitably causing far more damage to the entire enemy team via heros like DP & Gyro. But this isn't really perfect teamfight execution. I'd have really liked to see a mirror match.

Post reply on HN