OpenAI bots competing against Humans right now • Hacker News Board

Re: OpenAI bots competing against Humans right now

#131

Post by ethbro » Mon, Aug 06, 2018, 9:23 PM UTC

TangoTrotFox wrote: post #85

ethbro wrote: post #77

Earlier quoted context omitted.

> game AI that has ever surpassed human performance so far Am I missing something, or does that set consist of Checkers, Chess, and Go so far? (presumably with analogous misc games of comparable complexity) Discounting the reaction time wins, I'd say the sample size is too limited to generalize to eventual AI behavior in more complex / open-ended games. Extrapolation was the cause of the last AI winter.

I think you're looking at things with a sort of hindsight bias. Victory at chess was at one time considered to be the indicator of the emergence of true 'intelligence' in computing. The reason is that it's an extremely open, creative, and strategic game spattered with a minefield of tactical nuance. Nobody, human or computer, is getting even remotely close to scratching the depth of the game from a numeric point of v…

> Victory at chess was at one time considered to be the indicator of the emergence of true 'intelligence' in computing.

I would disagree with this characterization. I believe at the time, it was (a) a problem that a machine had not yet conquered, (b) a problem that it seemed feasible that a machine might conquer, and (c) a problem that, once conquered, would point the way to general artificial intelligence.

I would point at (c) as the assumption that proved to be erroneous. Deep Blue was clever algorithmic and hardware engineering (with a healthy budget) but led to... what?

AlphaGo is a fundamentally different approach, which shows signs of being more adaptable.

Point being, that winning a game is not sufficient evidence that a given approach will scale to winning all games, much less generalized intelligence.

To put it in terms of the fallacy I read in an article linked on HN (paraphrased), 'The public assumes that if a machine can perform a task that humans can perform, the machine must be human-like, and therefore able to perform all tasks that humans can perform.'

But in the same way that we use rendering tricks to go beyond-state-of-hardware-art in graphics rendering (by abusing hidden limitations), so do we often build ml systems.

I believe the most optimistic point against me was the slide in this year's GTC keynote pointing to the "Cambrian explosion" in the diversity of ml approaches this time around.

Re: OpenAI bots competing against Humans right now

#132

Post by ethbro » Mon, Aug 06, 2018, 9:27 PM UTC

red75prime wrote: post #92

ethbro wrote: post #77

Earlier quoted context omitted.

> game AI that has ever surpassed human performance so far Am I missing something, or does that set consist of Checkers, Chess, and Go so far? (presumably with analogous misc games of comparable complexity) Discounting the reaction time wins, I'd say the sample size is too limited to generalize to eventual AI behavior in more complex / open-ended games. Extrapolation was the cause of the last AI winter.

Statistics is not everything. Which hypothesis can lead to the outcome "AIs will not be able to beat humans in every game"? I can see following hypotheses (in no particular order): 1. Human brain is the optimal solution in the space of all computational devices capable of playing games, and we can only approach it. 2. To do computation human brain employs some physical processes, we will not be able to replicate in t…

I'd point to a combination of (2) and (5) as the maximally likely reason we'd fail to build a generalized game playing system.

I believe there exists a combination of hardware and software capable of beating humans in all games. However, I also believe victory in a single game gives us minimal information on whether or not the system generalizes to many games (to say nothing of non-game, e.g. more complex, ruleless systems).

Re: OpenAI bots competing against Humans right now

#133

Post by ufo » Mon, Aug 06, 2018, 10:09 PM UTC

ionforce wrote: post #123

wwhitlow wrote: post #122

Earlier quoted context omitted.

At what percentage would you allow an AI to consider a game unwinnable? While an AI that behaves erratically when the odds are low might be worth allowing it to be considered forfeit worthy, but the thing about humans is we make mistakes. Therefore an ideal AI that can continue to execute reasonable moves should have a lower percentage threshold where it decides to forfeit. See this match[0] for an example of a spect…

Maybe this is a limitation of self-play. If the opponent an AI faces during training is always optimal, then there's no surface area of mistakes. The losing AI, in its model/mind, knows that the game is over after a specific threshold. So it hasn't learned how to optimize for capitalizing on mistakes. I wonder if this situation can be fixed by adding more randomness. For example, force AI'1 to be in a losing position…

One of the most interesting takeaways from the post game interview for me was that the AI can be very stupid if you just blindly throw it in a self-play setting but with clever use of randomization (modifying power levels) and action restrictions (for example, only allowing the agent to spend an anti-invis item when a nearby enemy goes out of sight) it is possible to provide better learning opportunities for the AI.

Re: OpenAI bots competing against Humans right now

#134

Post by ionforce » Mon, Aug 06, 2018, 10:13 PM UTC

ufo wrote: post #133

ionforce wrote: post #123

Earlier quoted context omitted.

Maybe this is a limitation of self-play. If the opponent an AI faces during training is always optimal, then there's no surface area of mistakes. The losing AI, in its model/mind, knows that the game is over after a specific threshold. So it hasn't learned how to optimize for capitalizing on mistakes. I wonder if this situation can be fixed by adding more randomness. For example, force AI'1 to be in a losing position…

One of the most interesting takeaways from the post game interview for me was that the AI can be very stupid if you just blindly throw it in a self-play setting but with clever use of randomization (modifying power levels) and action restrictions (for example, only allowing the agent to spend an anti-invis item when a nearby enemy goes out of sight) it is possible to provide better learning opportunities for the AI.

> for example, only allowing anti-invis items when an enemy goes out of sight

These are the kind of actions you specifically don't want to code in because you're throwing in human knowledge. You want the AI to learn by itself that using anti-invis when everyone is visible is a low-value move.

The purist in me was even mad that they had a hand-crafted evaluation function. (e.g. prefer gold, prefer taking towers, each given some arbitrary value)

Re: OpenAI bots competing against Humans right now

#135

Post by QML » Tue, Aug 07, 2018, 6:09 AM UTC

red75prime wrote: post #130

QML wrote: post #121

Earlier quoted context omitted.

Modern AI may "beat" humans at certain tasks, but I doubt they are as efficient as humans are at those tasks: it makes sense that throwing more compute at problem will yield better results. But is it possible to get better results while constraining training time? Edit: I looked up how much time it takes to train: "OpenAI Five plays 180 years worth of games against itself every day, learning via self-play." [1] [1] h…

If you can get hundred years experience in 24 hours by wall clock, why not use it? Maybe it's better to teach AIs to create simplified, but usable models, they can use to simulate and train on.

I view the problem in terms of "computational complexity": if the AI of today requires O(2^n) time to perform at a human level, it is possible to get that down to O(n^2)?

I believe there's a connotation with problems, that if the best algorithm to solve it is exponential (brute-force search), then we truly don't understand the (underlying structure of the) problem.

Re: OpenAI bots competing against Humans right now

#136

Post by ionforce » Tue, Aug 07, 2018, 1:51 PM UTC

mellinoe wrote: post #129

zackaman wrote: post #128

Earlier quoted context omitted.

Because the first two matches were so lopsided, the bot lineup was selected by twitch chat + audience members. We drafted them a pretty terrible lineup, and from the start the bots estimated their chance of winning to be about 2.9%.

Indeed. And to explain further: not all hero combinations are equal. Meaning: you cannot select any arbitrary set of 5 heroes and expect them to perform well. Different heroes have different strengths and synergies that make them stronger or weaker depending on the specific teammates and opponents that are present. This is why drafting is considered such an important (and difficult) portion of the game. In match 3, a…

> Meaning: you cannot select any arbitrary set of 5 heroes and expect them to perform well.

When I think of AI, I think of something crawling its way out of purposefully adversarial situations such as this one. I would have loved to see optimal play from 5 wacky heroes.

I just have this suspicion that that wasn't optimal for that team comp.

But of course the matchup itself is a thing.

Re: OpenAI bots competing against Humans right now

#137

Post by red75prime » Tue, Aug 07, 2018, 2:05 PM UTC

QML wrote: post #135

red75prime wrote: post #130

Earlier quoted context omitted.

If you can get hundred years experience in 24 hours by wall clock, why not use it? Maybe it's better to teach AIs to create simplified, but usable models, they can use to simulate and train on.

I view the problem in terms of "computational complexity": if the AI of today requires O(2^n) time to perform at a human level, it is possible to get that down to O(n^2)? I believe there's a connotation with problems, that if the best algorithm to solve it is exponential (brute-force search), then we truly don't understand the (underlying structure of the) problem.

I don't think it is a fair comparison. We have the ability to build somewhat working strategy from known parts, and then we still need thousands of hours (or man-years in the case of Go) to improve it and to teach the brain to do some parts without conscious attention. Alpha Zero, OpenAI Five and others build strategy (mostly) from scratch.

I have a feeling that it is not possible to reduce algorithmic complexity of finding optimal solutions in most of the intellectual tasks (those that are in NP complexity class and above).

Most likely it is a trade-off. Quickly cobble up suboptimal strategy / build better strategy from scratch avoiding all time-saving benefits of using known parts, and avoiding all the pitfalls of not reevaluating utility of those parts in the current situation.

AIs surely will need to use all the spectrum to compete with humans.

Re: OpenAI bots competing against Humans right now

#138

Post by yvdriess » Wed, Aug 08, 2018, 11:34 AM UTC

fenomas wrote: post #112

yvdriess wrote: post #108

Earlier quoted context omitted.

Dota is a bit different from other esports such as SC2, in that it leans much more on game-sense and decision making than twitch skills[1]. Nevertheless, OpenAI dulled the reaction time artificially to be more human-like. It makes sense if the goal is to make a sophisticated strategic/gamesense AI, and not one that wins by just executing better. [1] For an entertaining case-study, check out Day[9]'s learns DotA2 seri…

Oh, that's interesting about the reaction time. Not having played DotA I have no idea how big an issue it is (and I couldn't make heads or tails of the video :D ). It occurs to me that for a really even playing field, the humans should probably be allowed to make and install UI mods if they want to. E.g. if there's an advantage to using an ability precisely when your hit points hit 50% (or whatever), an AI can easily…

Good point, and a hot topic over the years in the DotA community. The developer introduced some UI changes over the years that do exactly that: show every bit of information that an expert human or AI could realistically figure out and exploit. For some, it's dumbing down the game for a new audience, nullifying their hard work. I think the community consensus now is in favor: the game is complex enough that it does not need arbitrary skill-differentiation mechanics. For instance, high-level players used to memorize and practice the duration of some abilities (stuns), because you have an advantage if you can chain these perfectly. Valve introduced a visual progress bar that showed how much the disabled effect on a hero lasts, making it much easier to chain disables. They now also show how far 'towers' can shoot, but also which entity it is currently targeting (tower 'aggro' mechanics has its own special logic). They go so far as to show spawn boxes for neutral monsters.