Poker Tournament for LLMs

> with five copies of Pluribus playing against one professional

Although this configuration is designed to water down the difficulty in multi-player setting.

Pluribus against 2 professionals and 3 randos would better test. Two pros would take turns taking money from the 3 randos and Pluribus would be left behind and confused if it could not read the table.

Yeah I can see that for sure. That's also a holy grail of a poker enthusiast "can we please have non-mixed solution that is close enough". The problem is that 2% or even 1% equilibrium utility is huge. Professional players are often not happy seeing solutions that are 0.5% or less from equilibrium (measured by how much the solution can be exploited).

>>Continual resolving done in DeepStack [1]

Right, thank you. I am very used to the term resolving but not "online search". The idea here is to first approximate the solution using betting abstraction (for example solving with 3 bet sizes) and then hope this gets closer to the real thing if we resolve parts of the tree with more sizes (those parts that become relevant for the current play).

>>Gadget game introduced in [3], used in continual resolving.

I don't see "strategy consistency" in the paper nor a gadget game. Did you mean a different one?

>>Being imprecise like this would arguably not result in a super-human play.

Well, you have noticed that we can get somewhat close with a deterministic strategy and that is one step closer. There is nothing stopping LLMs from giving more precise answers like 70-30 or 90-10 or whatever.

>>But this is in token space. I'd be curious to see a demonstration of sampling of a distribution (i.e. some uniform) in the "token space", not via external tool calling. Can you make an LLM sample an integer from 1 to 10, or from any other interval, e.g. 223 to 566, without an external tool?

It doesn't have to sample it. It just needs to approximate the function that takes a game state and outputs the best move. That move is a distribution, not a single action. It's purely about pattern recognition (like chess). It can even learn to output colors or w/e (yellow for 100-0, red for 90-10, blue for 80-20 etc.). It doesn't need to do any sampling itself, just recognize patterns.

>>You don't need an LLM under such scheme -- you can do a k-NN or some other simple approximation. But any strategy/value approximation would encounter the very same problem DeepStack had to solve with gadget games about strategy inconsistency [5]. During play, you will enter a subgame which is not covered by your training data very quickly, as poker has ~10^160 states.

Ok, thank you I see what you mean by strategy consistency now. It's true that generating data if you need resolving (for example for no-limit poker) is also computationally expensive.

However your point:

>You don't need an LLM under such scheme -- you can do a k-NN or some other simple approximation.

Is not clear to me. You can say that about any other game then, no? The point of LLMs is that they are good at recognizing patterns in a huge space and may be able to approximate games like chess or poker pretty efficiently unlike traditional techniques.

>>How you define "precision" ?

I mean that there are patterns that seem very similar but result in completely different correct answers. In chess a miniscule difference in positions may result in a the same move being a winning one in one but a losing one in another. In poker if you call 25% more or 35% more if the bet size is 20% smaller is unlikely to result in a huge blunder. Chess is more volatile and thus you need more "precision" telling patterns apart.

I realize it's nota technical term but it's the one that comes to mind when you think about things LLMs are good and bad at. They are very good at seeing general patterns but weak when they need to be precise.

I am not sure that is true. Yes it will likely give a 3 or 7 but that is because it is trying to represent that distribution from the training data. It's not trying for a random digit there, it's trying for what the data set does.

It would certainly be possible to give an AI the notion of a random digit, and rather than training on fixed output examples give it additional training to make it to produce an embedding that was exactly equidistant from the tokens 0..9 when it wanted a random digit.

You could then fine tune it to use that ability to generate sequences of random digits to provide samples in reasoning steps.

That’s true. The original goal was to see which model performs statistically better than the others, but I quickly realized that would be neither practical nor particularly entertaining.

A proper benchmark would require things like: - Tens of thousands of hands played - Strict heads-up format (only two models compared at a time) - Each hand played twice with positions swapped

The current setup is mainly useful for observing common reasoning failure modes and how often they occur.