201–210 of 212 posts
Earlier quoted context omitted.
Do you have more info on deterministic equilibrium strategies for us (total beginners in the field) to learn about?
This is the citation for [0]: Sparsified Linear Programming for Zero-Sum Equilibrium Finding https://arxiv.org/pdf/2006.03451
> Tournament format > Texas Hold'em cash game, $10/$20
So, not a tournament at all, but a cash game.
Earlier quoted context omitted.
Set the temperature to zero and that's exactly what you get. In some NN implementations, randomness is actually pretty important to keep the gradients from getting stuck at local minima/maxima. Is that true for LLMs, or is it not something that applies at all?
Are you talking about training?
Earlier quoted context omitted.
They would need to lie, which they can't currently do. To play at our current best, our approximation of optimal play involves ranges. Thinking about your hand as being any one of a number of cards. Then imagine that you have combinations of those hands, and decide what you would do. That process of exploration by imagination doesn't work with an eager LLM using huge encoded context.
Why would they need to lie? Where's the lying in Poker? (Ignore for a moment that LLMs can lie just fine.) What you are describing is exploring a range of counterfactuals. That's not lying.
To see that LLMs aren't capable of this, I present all of the prompt jailbreaks that rely on repeated admonitions. And that makes sense if you think about the training data. There's not a lot of human writing that takes a fact and then confidently asserts the opposite as data mounts.
LLMs produce the most likely response from the input embeddings. Almost always, the easiest is that the next token is in agreement of the other tokens in the sequence. The problem in poker is that a good amount of the tokens in the sequence are masked and/or controlled by a villain who is actively trying to deceive.
Also, notice that I'm careful to say LLM's and not generalize to all attention head + MLP models. As attention with softmax and dot product is a good universal function. Instead, it's the large language model part that makes the models not great fits for poker. Human text doesn't have a latent space that's written about enough and thoroughly enough to have poker solved in there.
Earlier quoted context omitted.
Why would they need to lie? Where's the lying in Poker? (Ignore for a moment that LLMs can lie just fine.) What you are describing is exploring a range of counterfactuals. That's not lying.
Early game bluffs are essentially lies that you tell through the rest of the streets. In order to keep your opponents from knowing when you have premium starting hands, it's required to play some ranges, sometimes as if they were a different range. E.g., 10% of the time, I will bluff and act like I have AK, KK, AA, QQ. On the next street, I will need to continue that; otherwise, it becomes not profitable (opponents o…
In game theory, the point of bluffing is not so much to make money from your bluff directly, but to mask when you are playing a genuinely good hand.
> [...] it's required to play some ranges, sometimes as if they were a different range; [...]
Why the mental gymnastics? Just say what the optimal play for 'some ranges' is, and then play that. The extra indirection in explanation might be useful for human intuition, but I'm not sure the machine needs that dressing up.
> LLMs produce the most likely response from the input embeddings. [...]
If I wanted to have my LLM play poker, I would ask it suggest me probabilities for what to play next, and then sample from there, instead of using the next-token sampler in the LLM to directly tell you the action you should take.
(But I'm not sure that's what the original article is doing.)
> The problem in poker is that a good amount of the tokens in the sequence are masked and/or controlled by a villain who is actively trying to deceive.
> Human text doesn't have a latent space that's written about enough and thoroughly enough to have poker solved in there.
I agree with both. Though it's still a fun exercise to pit contemporary off-the-shelf LLMs against each other here.
And perhaps add a purpose built poker bot to the mix as a benchmark. And also try with and without access to an external random sampler (like I suggested above). Or with and without access to eg being able to run freshly written Python code.
Earlier quoted context omitted.
Are you talking about training?
I'm not sure, hence the question. AFAIK temperature only comes into play at inference time once the distribution is known, but I don't know if there are other places where random numbers are involved.
Eg you tend to randomly shuffle your corpus to train on. If you use drop-out (https://en.wikipedia.org/wiki/Dilution_(neural_networks)) you use randomness. You might also randomly perturb your training data. Lots of other sources of randomness that you might want to try.
This is my area of expertise. I love the experiment. In general games of imperfect information such as Poker, Diplomacy, etc are much much harder than perfect information games such as Chess. Multiplayer (3+) poker in particular is interesting because you cannot achieve a nash equilibrium (e.g. it is not zero sum). That is part of the reason they are a fantastic venue for exploration of the capabilities of LLMs. They…
Cool app! I think I found a broken problem (or it’s worded strangely?) and I’m unable to progress beyond it “Pick 2 that beat villain on board” “QQ552” In submitting Queen and 2 to make a full house but it just says “Incorrect. Two pair on board. Win with a full house or ace kicker”
If you wanna shoot me a note I'd love to hear any thoughts you have! Amir@pokerskill.com
This is my area of expertise. I love the experiment. In general games of imperfect information such as Poker, Diplomacy, etc are much much harder than perfect information games such as Chess. Multiplayer (3+) poker in particular is interesting because you cannot achieve a nash equilibrium (e.g. it is not zero sum). That is part of the reason they are a fantastic venue for exploration of the capabilities of LLMs. They…
> pokerskill.com Cool app, love the concept! Played poker a lot 20 years ago and very little since. Ran into some minor UX snags (iPhone) - feel free to hit me up if you're looking for feedback.
If you wanna shoot me a note I'd love to hear any thoughts you have!
Amir@pokerskill.com
Earlier quoted context omitted.
Cool app! I think I found a broken problem (or it’s worded strangely?) and I’m unable to progress beyond it “Pick 2 that beat villain on board” “QQ552” In submitting Queen and 2 to make a full house but it just says “Incorrect. Two pair on board. Win with a full house or ace kicker”
I know this is driving me nuts! I've had the fix in the approval queue just waiting on Apple :) If you wanna shoot me a note I'd love to hear any thoughts you have! Amir@pokerskill.com