Poker Tournament for LLMs

I agree, but they should come up with the distribution as well.

If you directly give the distribution to the LLM, it is not doing anything interesting. It is just sampling from the strategy you tell it to play.

In the context of gambling, random numbers or prngs can't have any unknown possible frequencies or tendencies. There can't be any doubt as to whether the number could be distorted or hallucinated. A pseudo random number that might or might not be from some algorithm picked by GPT is wayyyy worse than a mersenne twister, because it's open to distortion. Worse, there's no paper trail. MT is not the way to run a casino, or at least not sufficient, but at least you know it's pseudorandom based on a seed. With GPT you cannot know that, which means it doesn't fit the definition of "random" in any way. And if you find yourself watching a player getting blackjack 10 times in a row for $2k per bet, you will ask yourself where those numbers came from.

I did this for Risk. Was good fun (in a token hungry kind of way).

https://andreasthinks.me/posts/ai-at-play/

>>1) There are currently no algorithms that can compute deterministic equilibrium strategies [0]. Therefore, mixed (randomized) strategies must be used for professional-level play or stronger.

It's not that the algorithm is currently not known but it's the nature of the game that deterministic equilibrium strategies don't exist for anything but most trivial games. It's very easy to prove as well (think Rock-Paper-Scissors).

>>2) In practice, strong play has been achieved with: i) online search and ii) a mechanism to ensure strategy consistency. Without ii) an adaptive opponent can learn to exploit inconsistency weaknesses in a repeated play.

In practice strong play was achieved by computing approximate equilibria using various algorithms. I have no idea what you mean by "online search" or "mechanism to ensure strategy consistency". Those are not terms used by people who solve/approximate poker games.

>>3) LLMs do not have a mechanism for sampling from given probability distributions. E.g. if you ask LLM to sample a random number from 1 to 10, it will likely give you 3 or 7, as those are overrepresented in the training data.

This is not a big limitation imo. LLM can give an answer like "it's likely mixed between call and a fold" and then you can do the last step yourself. Adding some form of RNG to LLM is trivial as well and already often done (temperature etc.)

>>Based on these points, it’s not technically feasible for current LLMs to play poker strongly

Strong disagree on this one.

>>This is in contrast with Chess, where there is lots more of training data, there exists a deterministic optimal strategy and you do not need to ensure strategy consistency.

You can have as much training data for poker as you have for chess. Just use a very strong program that approximates the equilibrium and generate it. In fact it's even easier to generate the data. Generating chess games is very expensive computationally while generating poker hands from an already calculated semi-optimal solution is trivial and very fast.

The reason both games are hard for LLMs is that they require precision and LLMs are very bad at precision. I am not sure which game is easier to teach an LLM to play well. I would guess poker. They will get better at chess quicker though as it's more prestigious target, there is way longer tradition of chess programming and people understand it way better (things like game representation, move representation etc.).

Imo poker is easier because it's easier to avoid huge blunders. In chess a miniscule difference in state can turn a good move into a losing blunder. Poker is much more stable so general not-so-precise pattern recognition should do better.

I am really puzzled by "strategy consistency" term. You are a PhD but you use a term that is not really used in either poker nor chess programming. There really isn't anything special about poker in comparison to chess. Both games come down to: "here is the current state of the game - tell me what the best move is".

It's just in poker the best/optimal move can be "split it to 70% call and 30% fold" or similar. LLMs in theory should be able to learn those patterns pretty well once they are exposed to a lot of data.

It's true that multiway poker doesn't have "optimal" solution. It has equilibrium one but that's not guaranteed to do well. I don't think your point is about that though.

>>Agreed. I don't know how fast it would get into the money, but an equilibrium strategy is guaranteed to not lose, in expectation.

That's only true for heads-up play. It doesn't apply to poker tournaments.