Hi! I'm one of the authors on the paper. We'd be happy to answer any questions. Ask us anything!
Damn, I haven’t gotten around to fully reading Pluribus yet, and now there’s more? Congrats on the results! What’s next?
Building AI that can master complex cooperative games with hidden information
21–29 of 29 posts
Re: Building AI that can master complex cooperative games with hidden information
#22Hi! I'm one of the authors on the paper. We'd be happy to answer any questions. Ask us anything!
Isn't this not the same thing as AI that can beat humans at things like Bridge where the bidding game matters quite a lot? IIRC in Hanabi the fact that there is imperfect information does not really matter that much for strategy, where as in things like League of Legends or Bridge or many of those types of games it really does matter quite a lot.
The imperfect information in Hanabi absolutely matters a ton. It's not an interesting game without it.
Re: Building AI that can master complex cooperative games with hidden information
#23If you're wondering why this is interesting, games that AIs excel at like chess/checkers/go are all two-player, zero-sum, perfect-information (everyone knows everything), deterministic games, so you can exactly predict your opponents behavior by simulating "what would I do if I were them, and trying to make me lose". The only real hard problem in this space is extreme branching factors. Everything gets vastly more co…
That being said:
> hidden information forces you to manage how much you reveal to your opponent
It is hard to see how that could be harder for a learning AI to pick up than any other game action with long-term benefits. It might reveal that AI are so much better at humans in games like Go and Chess that they aren't even exploiting long-term links between cause and effects, but I suspect from the example we've seen with Go that this is a solved problem.
> and requires you to simulate multiple "alternate futures" based on things you learn after making a decision,
That is describing a tree search. Why is that theoretically difficult? I can see it is a real engineering challenge to simulate a complex environment.
> and randomness is equivalent to an extra player that makes irrational unpredictable moves.
Computers handle randomness much better than humans at all levels because they have an actual statistical grounding. They still aren't very good at it, but they sure thrash humans.
I suppose basically I can see why a hidden-information game could be intractable because simulating the environment is so hard that it can't be done and breakthroughs in simulation must be found to train the AI. But I don't see why that is being linked to hidden information and cooperative gameplay. We know humans do fine without utilising hidden information, because they can play these games and they don't know any hidden information.
Re: Building AI that can master complex cooperative games with hidden information
#24If you're wondering why this is interesting, games that AIs excel at like chess/checkers/go are all two-player, zero-sum, perfect-information (everyone knows everything), deterministic games, so you can exactly predict your opponents behavior by simulating "what would I do if I were them, and trying to make me lose". The only real hard problem in this space is extreme branching factors. Everything gets vastly more co…
In particular, communication is in some ways a complex multiplayer game requiring multi-level modeling of other participants. In the ideal case, it can be modeled as a cooperative game. In suboptimal cases that sadly occur often in the real world, it's a game where you're fractionally cooperating and fractionally competing with every other player, and the degree to which you're cooperating or competing with any given…
Re: Building AI that can master complex cooperative games with hidden information
#25Earlier quoted context omitted.
What's FB going to do with this?
Open source it, learn from it, and build upon it to continue to push forward the frontier of AI.
Facebook probably pays through the nose for AI research and probably wants a ROI. Facebook makes money by building better user models and spamming targetted ads. Some of them are getting scarily good.
Re: Building AI that can master complex cooperative games with hidden information
#26they (basically) applied the ideas from a bot that plays poker to another game. it's interesting work, though perhaps not groundbreaking. This idea of selfplay + counterfactual regret minimization does seem to be the superior way to solve game theoretic problems. Identifying valuable game theoretic problems remains a challenge...
The search algorithm shares a lot in common with our Pluribus poker AI ( https://ai.facebook.com/blog/pluribus-first-ai-to-beat-pros-... ), but we added "retrospective belief updates" which makes it way more scalable. We also didn't use counterfactual regret minimization (CFR) because in cooperative games you want to be as predictable as possible, whereas CFR helps make you unpredictable in a balanced way (useful in…
I see what you're saying about the real time search (which took me quite some time to understand). I came up with a way to do that from disk due to memory limitations. It limits the number of search iterations but doesn't seem to have a huge negative impact on quality so far.
Anyway, thanks again!
Re: Building AI that can master complex cooperative games with hidden information
#27They should tackle Starcraft II next like DeepMind has with AlphaStar, or at least a similar real time RTS like Starcraft with a fog of war and a partially observable state.
Re: Building AI that can master complex cooperative games with hidden information
#28Earlier quoted context omitted.
Hanabi definitely has traits involving theory of mind that I've not seen present in other games. For example, I've played Hanabi with 3+ players where the person before me deliberately gave a misleading hint to the person after me. For example, "this is your only blue card" indicating a blue 5 even though only blue 1-3 have been played. They were counting on me to anticipate that the mislead person was now very likel…
> For example, I've played Hanabi with 3+ players where the person before me deliberately gave a misleading hint to the person after me. For example, "this is your only blue card" indicating a blue 5 even though only blue 1-3 have been played. They were counting on me to anticipate that the mislead person was now very likely to waste that valuable card & realize that I could only reasonably avert that misplay by play…
Re: Building AI that can master complex cooperative games with hidden information
#29Earlier quoted context omitted.
In particular, communication is in some ways a complex multiplayer game requiring multi-level modeling of other participants. In the ideal case, it can be modeled as a cooperative game. In suboptimal cases that sadly occur often in the real world, it's a game where you're fractionally cooperating and fractionally competing with every other player, and the degree to which you're cooperating or competing with any given…
Unfortunately Facebook’s approach sidesteps this complexity by ensuring each player uses the same random seed and searches policy based on information they can all see. It’s not really solving the problem as intended in my opinion.