51–60 of 101 posts
I used to play professionally, and I still play in the casinos. These LLMs are playing better than most human players I encounter (low limits). They're kinda bad, but not as criminally bad as the humans.
Post-flop on the other hand is all over the place...
Honest question, but this seems like an expensive project to host given the number of tokens per second. How is this being paid for?
Earlier quoted context omitted.
How are these differebt in your mind? The history is the history. Or do you mean - each agent has a chance to think after every turn?
Well they can be watching all the action and thinking the whole time as the action leads up them, just like we do in poker. To me it's different, subtly perhaps.
Your idea of having it being passed in real time and having the LLM create a chain of thoughts even if action is not on them is interesting. I'd be curious to see if it would result in improved play.
I'm not an expert, but as I understand it there are existing solvers for poker/holdem? Perhaps one of the players could be a traditional solver to see how the LLMs fare against those?
This also wouldn't even be a close contest, I think Pluribus demonstrated a solid win rate against professional players in a test.
As I was developing this project, a main thought came to mind as to the comparison between cost and performance between a "purpose" built AI such as Pluribus versus a general LLM model. I think Pluribus training costs ~$144 in cloud computing credits.
Curious if you used pokerkit for this, or some other engine or custom engine?
Why are there 2 Claude Players ?
Earlier quoted context omitted.
Well they can be watching all the action and thinking the whole time as the action leads up them, just like we do in poker. To me it's different, subtly perhaps.
For my implementation, I'm passing in the current hand's action history (e.g. Player 1 raises to $X preflop, Player 2 calls, Player 3 calls. Flop is A B C, Player 2 checks, etc) whenever the action is on the player. Your idea of having it being passed in real time and having the LLM create a chain of thoughts even if action is not on them is interesting. I'd be curious to see if it would result in improved play.
I used to play professionally, and I still play in the casinos. These LLMs are playing better than most human players I encounter (low limits). They're kinda bad, but not as criminally bad as the humans.
I'm actually surprised at how well they play pre-flop (mostly). Did some initial analysis on VPIP/PFR across positions, and somewhat decent. Post-flop on the other hand is all over the place...
Earlier quoted context omitted.
To expand on this - an LLM will try to play (and reason) like a person would, while a solver simply crunches the possibility space for the mathematically optimal move. It’s similar to how an LLM can sometimes play chess on a reasonably high (but not world-class) level, while Stockfish (the chess solver) can easily crush even the best human player in the world.
How does a poker solver select bet size? Doesn't this depend on posteriors on the opponent's 'policy' + hand estimation?