I wonder how NovaSolver would fair here.
(author of PokerBattle here)
I think it would've completely crush them (like any other solver-based solution). Poker is safe for now :)
Poker Tournament for LLMs
91–100 of 212 posts
Re: Poker Tournament for LLMs
#92check out House of TEN - https://houseof.ten.xyz - it's a blockchain based (fully on-chain) Texas Hold'em played by AI Agents
(author of PokerBattle here)
Haven't seen it before, thanks Are you affiliated with them?
Re: Poker Tournament for LLMs
#93Cool idea and interesting that Grok is winning and has “bad” stats. I wonder if Grok is exploiting Minstral and Meta who vpip too much and the don’t c-bet. Seems to win a lot of showdowns and folds to a lot of three bets. Punishes the nits because it’s able to get away from bad hands. Goes to showdown very little so not showing its hands much - winning smaller pots earlier on.
The results/numbers aren't interesting because the number of samples is woefully insufficient to draw any conclusions beyond "that's a nice looking dashboard" or maybe "this is a cool idea"
(author of PokerBattle here)
You right, results and numbers are mainly for entertainment purposes. This sample size would allow to analyze main reasoning failure modes and how often they occur.
Re: Poker Tournament for LLMs
#94Whis was built on Vercel and its shitting the bed right now
Re: Poker Tournament for LLMs
#95Cool idea and interesting that Grok is winning and has “bad” stats. I wonder if Grok is exploiting Minstral and Meta who vpip too much and the don’t c-bet. Seems to win a lot of showdowns and folds to a lot of three bets. Punishes the nits because it’s able to get away from bad hands. Goes to showdown very little so not showing its hands much - winning smaller pots earlier on.
The results/numbers aren't interesting because the number of samples is woefully insufficient to draw any conclusions beyond "that's a nice looking dashboard" or maybe "this is a cool idea"
Anti-grok cope right here
Re: Poker Tournament for LLMs
#96We (TEN Protocol) did this a few months ago, using blockchain to make the LLMs’ actions publicly visible and TEEs for verifiable randomness in shuffling and other processes. We used a mix of LLMs across five players and ran multiple tournaments over several months. The longest game we observed lasted over 50 hours straight. Screenshot of the gameplay: https://pbs.twimg.com/media/GpywKpDXMAApYap?format=png&name=... Po…
Why use blockchain here? I don't see how this would make the list of actions any more trustworthy. No one else was involved and no one can disprove anything.
Re: Poker Tournament for LLMs
#97It doesn't seem like the design of this experiment allows AIs to evolve novel strategy over time. I wonder if poker-as-text is similar to maths -- LLMs are unable to reason about the underlying reality.
You mean that they don’t have access to whole opponent behavior? It would be hilaroius to allow table talk and see them trying to bluff and sway each other :D
I mean, LLMs have the same sorts of problem with
"Which poker hand is better: 7S8C or 2SJH"
as
"What is 77 + 19"?
Re: Poker Tournament for LLMs
#98I have PhD in algorithmic game theory and worked on poker. 1) There are currently no algorithms that can compute deterministic equilibrium strategies [0]. Therefore, mixed (randomized) strategies must be used for professional-level play or stronger. 2) In practice, strong play has been achieved with: i) online search and ii) a mechanism to ensure strategy consistency. Without ii) an adaptive opponent can learn to exp…
> LLMs do not have a mechanism for sampling from given probability distributions
Would a LLM with tool calls be able to do this?
Re: Poker Tournament for LLMs
#99This is the STEM version of dog fighting.
Re: Poker Tournament for LLMs
#100I have PhD in algorithmic game theory and worked on poker. 1) There are currently no algorithms that can compute deterministic equilibrium strategies [0]. Therefore, mixed (randomized) strategies must be used for professional-level play or stronger. 2) In practice, strong play has been achieved with: i) online search and ii) a mechanism to ensure strategy consistency. Without ii) an adaptive opponent can learn to exp…
I ran a casino and wrote a bot framework that, with a user's permission, attempted to clone their betting strategy based on their hand history (mainly how they bet as a ratio to the pot in a similar blind odds situation relative to the aggressiveness of players before and after), and I let the players play against their own bots. It was fun to watch. Oftentimes the players would lose against their bot versions for aw…
> LLMs would have no hope at conceptualizing any of that.
Counter argument - generating probabilistic tokens (degree of randomness) is core concept for an LLM.