Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Poker Tournament for LLMs

pokerbattle.ai

Post reply on HN
Print view View original

131–140 of 212 posts

Re: Poker Tournament for LLMs

#131

Post by bluecalm »

LPisGood wrote: post #116
bluecalm wrote: post #48

Earlier quoted context omitted.

>>1) There are currently no algorithms that can compute deterministic equilibrium strategies [0]. Therefore, mixed (randomized) strategies must be used for professional-level play or stronger. It's not that the algorithm is currently not known but it's the nature of the game that deterministic equilibrium strategies don't exist for anything but most trivial games. It's very easy to prove as well (think Rock-Paper-Sci…

> There really isn't anything special about poker in comparison to chess They are dramatically different. There is no hidden information in chess, there are only two players in chess, the number of moves you can make is far smaller in chess, and there is no randomness in chess. This is why you never hear about EV in chess theory, but it’s central to poker.

>>There is no hidden information in chess

Hidden information doesn't make a game more complicated. Rock Paper Scissors have hidden information but it's a very simple game for example. You can argue there is no hidden information in poker either if you think in terms of ranges. Your inputs are the public cards on the board and betting history - nothing hidden there. Your move requires a probability distribution across the whole range (all possible hands). Framed like that hidden information in poker disappears. The task is to just find the best distributions so the strategy is unexploitable - same as in chess (you need to play moves that won't lose and preferably win if the opponent makes a mistake).

Re: Poker Tournament for LLMs

#132

Post by eclark »

Cool_Caribou wrote: post #64
bluecalm wrote: post #48

Earlier quoted context omitted.

>>1) There are currently no algorithms that can compute deterministic equilibrium strategies [0]. Therefore, mixed (randomized) strategies must be used for professional-level play or stronger. It's not that the algorithm is currently not known but it's the nature of the game that deterministic equilibrium strategies don't exist for anything but most trivial games. It's very easy to prove as well (think Rock-Paper-Sci…

Is limit poker a trivial game? I believe it's been solved for a long time already.

No it's far from trivial for three reasons.

First being the hidden information, you don't know your opponents hand holdings; that is to say everyone in the game has a different information set.

The second is that there's a variable number of players in the game at any time. Heads up games are closer to solved. Mid ring games have had some decent attempts made. Full ring with 9 players is hard, and academic papers on it are sparse.

The third is the potential number of actions. For no limit games there's a lot of potential actions, as you can bet in small decimal increments of a big blind. Betting 4.4 big blinds could be correct and profitable, while betting 4.9 big blinds could be losing, so there's a lot to explore.

Re: Poker Tournament for LLMs

#133

Post by nialv7 »

michalsustr wrote: post #31

I have PhD in algorithmic game theory and worked on poker. 1) There are currently no algorithms that can compute deterministic equilibrium strategies [0]. Therefore, mixed (randomized) strategies must be used for professional-level play or stronger. 2) In practice, strong play has been achieved with: i) online search and ii) a mechanism to ensure strategy consistency. Without ii) an adaptive opponent can learn to exp…

That's fascinating. Are there any introductory literature you would recommend to someone curious about poker AI?

Re: Poker Tournament for LLMs

#134

Post by sceptic123 »

tarruda wrote: post #98
michalsustr wrote: post #31

I have PhD in algorithmic game theory and worked on poker. 1) There are currently no algorithms that can compute deterministic equilibrium strategies [0]. Therefore, mixed (randomized) strategies must be used for professional-level play or stronger. 2) In practice, strong play has been achieved with: i) online search and ii) a mechanism to ensure strategy consistency. Without ii) an adaptive opponent can learn to exp…

> LLMs do not have a mechanism for sampling from given probability distributions Would a LLM with tool calls be able to do this?

Then it's not the LLM doing the work

Re: Poker Tournament for LLMs

#135

Post by eclark »

brrrrrm wrote: post #127
eclark wrote: post #123

I am the author/maintainer of rs-poker ( https://github.com/elliottneilclark/rs-poker ). I've been working on algorithmic poker for quite a while. This isn't the way to do it. LLMs would need to be able to do math, lie, and be random. None of which are they currently capable. We know how to compute the best moves in poker (it's computationally challenging; the more choices and players are present, the more likely it…

> None of which are they currently capable what makes you say this? modern LLMs (the top players in this leaderboard) are typically equipped with the ability to execute arbitrary Python and regularly do math + random generations. I agree it's not an efficient mechanism by any means, but I think a fine-tuned LLM could play near GTO for almost all hands in a small ring setting

To play GTO currently you need to play hand ranges. (For example when looking at a hand I would think: I could have AKs-ATs, QQ-99, and she/he could have JT-98s, 99-44, so my next move will act like I have strength and they don't because the board doesn't contain any low cards). We have do this since you can't always bet 4x pot when you have aces, the opponents will always know your hand strength directly.

LLM's aren't capable of this deception. They can't be told that they have some thing, pretend like they have something else, and then revert to gound truth. Their egar nature with large context leads to them getting confused.

On top of that there's a lot of precise math. In no limit the bets are not capped, so you can bet 9.2 big blinds in a spot. That could be profitable because your opponents will call and lose (eg the players willing to pay that sometimes have hands that you can beat). However betting 9.8 big blinds might be enough to scare off the good hands. So there's a lot of probiblity math with multiplication.

Deep math with multiplication and accuracy are not the forte of llm's.

Re: Poker Tournament for LLMs

#136

Post by d-moon »

nialv7 wrote: post #133
michalsustr wrote: post #31

I have PhD in algorithmic game theory and worked on poker. 1) There are currently no algorithms that can compute deterministic equilibrium strategies [0]. Therefore, mixed (randomized) strategies must be used for professional-level play or stronger. 2) In practice, strong play has been achieved with: i) online search and ii) a mechanism to ensure strategy consistency. Without ii) an adaptive opponent can learn to exp…

That's fascinating. Are there any introductory literature you would recommend to someone curious about poker AI?

MIT’s IAP Pokerbts class https://github.com/mitpokerbots

Re: Poker Tournament for LLMs

#137

Post by eclark »

Tostino wrote: post #126
eclark wrote: post #123

I am the author/maintainer of rs-poker ( https://github.com/elliottneilclark/rs-poker ). I've been working on algorithmic poker for quite a while. This isn't the way to do it. LLMs would need to be able to do math, lie, and be random. None of which are they currently capable. We know how to compute the best moves in poker (it's computationally challenging; the more choices and players are present, the more likely it…

Why wouldn't something like an RL environment allow them to specialize in poker playing, gaining those skills as necessary to increase score in that environment? E.g. given a small code execution environment, it could use some secure random generator to pick between options, it could use a calculator for whatever math it decides it can't do 'mentally', and they are very capable of deception already, even more so when…

> Why wouldn't something like an RL environment allow them to specialize in poker playing, gaining those skills as necessary to increase score in that environment?

I think an RL environment is needed to solve poker with an ML model. I also think that like chess, you need the model to do some approximate work. General-purpose LLMs trained on text corpus are bad at math, bad at accuracy, and struggle to stay on task while exploring.

So a purpose built model with a purpose built exploring harness is likely needed. I've built the basis of an RL like environment, and the basis of learning agents in rust for poker. Next steps to come.

Re: Poker Tournament for LLMs

#138

Post by JoeAltmaier »

eclark wrote: post #135
brrrrrm wrote: post #127

Earlier quoted context omitted.

> None of which are they currently capable what makes you say this? modern LLMs (the top players in this leaderboard) are typically equipped with the ability to execute arbitrary Python and regularly do math + random generations. I agree it's not an efficient mechanism by any means, but I think a fine-tuned LLM could play near GTO for almost all hands in a small ring setting

To play GTO currently you need to play hand ranges. (For example when looking at a hand I would think: I could have AKs-ATs, QQ-99, and she/he could have JT-98s, 99-44, so my next move will act like I have strength and they don't because the board doesn't contain any low cards). We have do this since you can't always bet 4x pot when you have aces, the opponents will always know your hand strength directly. LLM's aren…

Agreed. I tried it on a simple game of exchanging colored tokens from a small set of recipes. Challenged it to start with two red and end up with four white, for instance. I failed. It would make one or two correct moves, then either hallucinate a recipe, hallucinate the resulting set of tiles after a move, or just declare itself done!

Re: Poker Tournament for LLMs

#139

Post by meep_morp »

camillomiller wrote: post #2

As a Texas Hold'em enthusiast, some of the hands are moronic. Just checked one where grok wins with A3s because Gemini folds K10 with an Ace and a King on the board, without Grok betting anything. Gemini just folds instead of checking. It's not even GTO, it's just pure hallucination. Meaning: I wouldn't read anything into the fact that Grok leads. These machines are not made to play games like online poker determinis…

I play PLO and sometimes share hand histories with ChatGPT for fun. It can never successfully parse a starting hand let alone how it interacts with the board.

Re: Poker Tournament for LLMs

#140

Post by jwatte »

michalsustr wrote: post #31

I have PhD in algorithmic game theory and worked on poker. 1) There are currently no algorithms that can compute deterministic equilibrium strategies [0]. Therefore, mixed (randomized) strategies must be used for professional-level play or stronger. 2) In practice, strong play has been achieved with: i) online search and ii) a mechanism to ensure strategy consistency. Without ii) an adaptive opponent can learn to exp…

Tool using LLMs can easily be given a tool to sample whatever distribution you want. The trick is to proompt them when to invoke the tool, and correctly use its output.
Post reply on HN