Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Poker Tournament for LLMs

pokerbattle.ai

Post reply on HN
Print view View original

161–170 of 212 posts

Re: Poker Tournament for LLMs

#161

Post by eclark »

jwatte wrote: post #141
eclark wrote: post #124

Earlier quoted context omitted.

They would need to lie, which they can't currently do. To play at our current best, our approximation of optimal play involves ranges. Thinking about your hand as being any one of a number of cards. Then imagine that you have combinations of those hands, and decide what you would do. That process of exploration by imagination doesn't work with an eager LLM using huge encoded context.

I don't think this analysis matches the underlying implementation. The width of the models is typically wide enough to "explore" many possible actions, score them, and let the sampler pick the next action based on the weights. (Whether a given trained parameter set will be any good at it, is a different question.) The number of attention heads for the context is similarly quite high. And, as a matter of mechanics, th…

No the widths are not wide enough to explore. The number of possible game states can explode beyond the number of atoms in the universe pretty easily, especially if you use deep stacks with small big blinds.

For example when computing the counterfactual tree for 9 way preflop. 9 players have up to 6 different times that they can be asked to perform an action (seat 0 can bet 1, seat 1 raises min, seat 2 calls, back to seat 0 raises min, with seat 1 calling, and seat 2 raising min, etc). Each of those actions has check, fold, bet min, raise the min (starting blinds of 100 are pretty high all ready), raise one more than the min, raise two more than the min, ... raise all in (with up to a million chips).

(1,000,000.00 - 999,900.00) ^ 6 times per round ^ 9 players That's just for pre flop. Postflop, River, Turn, Showdown. Now imagine that we have to simulate which cards they have and which order they come in the streets (that greatly changes the value of the pot).

As for LLMs being great at range stats, I would point you to the latest research by UChicago. Text trained LLMs are horrible at multiplication. Try getting any of them to multiply any non-regular number by e or pi. https://computerscience.uchicago.edu/news/why-cant-powerful-...

Don't get what I'm saying wrong though. Masked attention and sequence-based context models are going to be critical to machines solving hidden information problems like this. Large Language Models trained on the web crawl and the stack with text input will not be those models though.

Re: Poker Tournament for LLMs

#162

Post by Nicook »

furyofantares wrote: post #158
michalsustr wrote: post #31

I have PhD in algorithmic game theory and worked on poker. 1) There are currently no algorithms that can compute deterministic equilibrium strategies [0]. Therefore, mixed (randomized) strategies must be used for professional-level play or stronger. 2) In practice, strong play has been achieved with: i) online search and ii) a mechanism to ensure strategy consistency. Without ii) an adaptive opponent can learn to exp…

> 3) LLMs do not have a mechanism for sampling from given probability distributions. E.g. if you ask LLM to sample a random number from 1 to 10, it will likely give you 3 or 7, as those are overrepresented in the training data. You can have them output a probability distribution and then have normal code pick the action. There's other ways to do this, you don't need to make the LLM pick a random number.

so you're confirming that what he said is correct

Re: Poker Tournament for LLMs

#163

Post by VerminOctopus1 »

noduerme wrote: post #39
michalsustr wrote: post #31

I have PhD in algorithmic game theory and worked on poker. 1) There are currently no algorithms that can compute deterministic equilibrium strategies [0]. Therefore, mixed (randomized) strategies must be used for professional-level play or stronger. 2) In practice, strong play has been achieved with: i) online search and ii) a mechanism to ensure strategy consistency. Without ii) an adaptive opponent can learn to exp…

I ran a casino and wrote a bot framework that, with a user's permission, attempted to clone their betting strategy based on their hand history (mainly how they bet as a ratio to the pot in a similar blind odds situation relative to the aggressiveness of players before and after), and I let the players play against their own bots. It was fun to watch. Oftentimes the players would lose against their bot versions for aw…

How did you collect their hand history?

Re: Poker Tournament for LLMs

#164

Post by furyofantares »

Nicook wrote: post #162
furyofantares wrote: post #158

Earlier quoted context omitted.

> 3) LLMs do not have a mechanism for sampling from given probability distributions. E.g. if you ask LLM to sample a random number from 1 to 10, it will likely give you 3 or 7, as those are overrepresented in the training data. You can have them output a probability distribution and then have normal code pick the action. There's other ways to do this, you don't need to make the LLM pick a random number.

so you're confirming that what he said is correct

No.

It's not like an LLM can play poker without some shim around it. You're gonna have to interpret its results and take actions. And you want the LLM to produce a distribution either way before picking an explicit action from that distribution. Having the shim pick the random number instead of the LLM does not take anything away from it.

Re: Poker Tournament for LLMs

#165

Post by CGMthrowaway »

michalsustr wrote: post #31

I have PhD in algorithmic game theory and worked on poker. 1) There are currently no algorithms that can compute deterministic equilibrium strategies [0]. Therefore, mixed (randomized) strategies must be used for professional-level play or stronger. 2) In practice, strong play has been achieved with: i) online search and ii) a mechanism to ensure strategy consistency. Without ii) an adaptive opponent can learn to exp…

>if you ask LLM to sample a random number from 1 to 10, it will likely give you 3 or 7, as those are overrepresented in the training data.

I just tried this on GPT-4 ("give me 100 random numbers from 1 to 10") and it gave me exactly 10 of each number 1-10, but in no particular order. Heh

Re: Poker Tournament for LLMs

#166

Post by KalMann »

CGMthrowaway wrote: post #165
michalsustr wrote: post #31

I have PhD in algorithmic game theory and worked on poker. 1) There are currently no algorithms that can compute deterministic equilibrium strategies [0]. Therefore, mixed (randomized) strategies must be used for professional-level play or stronger. 2) In practice, strong play has been achieved with: i) online search and ii) a mechanism to ensure strategy consistency. Without ii) an adaptive opponent can learn to exp…

>if you ask LLM to sample a random number from 1 to 10, it will likely give you 3 or 7, as those are overrepresented in the training data. I just tried this on GPT-4 ("give me 100 random numbers from 1 to 10") and it gave me exactly 10 of each number 1-10, but in no particular order. Heh

I think the way you phrase it is important. If you want to test what he said you should try and create 100 independent prompts in which you ask for a number between 1 and 10.

Re: Poker Tournament for LLMs

#167

Post by Eckter2 »

_ink_ wrote: post #53
michalsustr wrote: post #31

I have PhD in algorithmic game theory and worked on poker. 1) There are currently no algorithms that can compute deterministic equilibrium strategies [0]. Therefore, mixed (randomized) strategies must be used for professional-level play or stronger. 2) In practice, strong play has been achieved with: i) online search and ii) a mechanism to ensure strategy consistency. Without ii) an adaptive opponent can learn to exp…

> LLMs do not have a mechanism for sampling from given probability distributions. They could have a tool for that, tho.

They already have the tool, it's python interpreter with `random`.

I just tested with a mistral's chat: I asked it to answer either "foo" or "bar" and that I need either option to have the same probability. I did not mention the code interpreter or any other instruction. It did generate and execute a basic `random.choice(["foo", "bar"])` snippet.

I'm assuming more mainstream models would do the same. And I'm assuming that a model would figure out that randomness is important when playing poker.

Re: Poker Tournament for LLMs

#168

Post by LPisGood »

bluecalm wrote: post #131
LPisGood wrote: post #116

Earlier quoted context omitted.

> There really isn't anything special about poker in comparison to chess They are dramatically different. There is no hidden information in chess, there are only two players in chess, the number of moves you can make is far smaller in chess, and there is no randomness in chess. This is why you never hear about EV in chess theory, but it’s central to poker.

>>There is no hidden information in chess Hidden information doesn't make a game more complicated. Rock Paper Scissors have hidden information but it's a very simple game for example. You can argue there is no hidden information in poker either if you think in terms of ranges. Your inputs are the public cards on the board and betting history - nothing hidden there. Your move requires a probability distribution across…

More complicated? That’s ambiguous. It certainly makes it different.

If you apply probabilistic methods it doesn’t remove hidden information from the problem. These are just quite literally the techniques used to deal with hidden information.

Re: Poker Tournament for LLMs

#169

Post by Workaccount2 »

sceptic123 wrote: post #152
catketch wrote: post #151

Earlier quoted context omitted.

this is is a distinction without a difference in many instances. I can easily ask an llm to write a python tool to produce random numbers for a given distribution and then use that tool as needed. The LLM writes the code, and uses the executable result. Then end black box result is the LLM doing the work

But why limit it to generating random numbers, isn't the logical conclusion that the LLM writes a poker bot instead of playing the game? How would that demonstrate the poker skills of an LLM?

There is a distinction, but for all intents and purposes, it's superficial.

Re: Poker Tournament for LLMs

#170

Post by tasuki »

VerminOctopus1 wrote: post #163
noduerme wrote: post #39

Earlier quoted context omitted.

I ran a casino and wrote a bot framework that, with a user's permission, attempted to clone their betting strategy based on their hand history (mainly how they bet as a ratio to the pot in a similar blind odds situation relative to the aggressiveness of players before and after), and I let the players play against their own bots. It was fun to watch. Oftentimes the players would lose against their bot versions for aw…

How did you collect their hand history?

> I ran a casino

It's in the first four words! Which parts have you read?

Post reply on HN