Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How A.I. Conquered Poker

nytimes.com

Post reply on HN
Print view View original

51–60 of 188 posts

Re: How A.I. Conquered Poker

#51

Post by dwohnitmok »

isaacg wrote: post #20
marcodiego wrote: post #6

Is there any remaining logic game which AI hasn't become superhuman? IIRC Monte Carlo tree search has done wonders in this area.

As far as I know, in Magic: the Gathering, the best bots are far worse than most players. Part of the difficulty is that the rules are so complicated that there are only a couple of complete rules implementations. Beyond that, it's an imperfect information game with far more actions per game than poker, so optimal-solver techniques haven't seen success.

I think this is purely a resource issue, e.g. if Google Brain decided to make an MtG bot I would be fairly confident it would be superhuman. Even real time strategy games like Starcraft are looking like they're on the cusp of superhuman bots (Alphastar was competitive as Protoss against elite players, but did not consistently beat them).

Re: How A.I. Conquered Poker

#52

Post by splonk »

(Former pro and high stakes player, occasional solver developer)

This is actually one of the best poker articles I've ever seen in generalist media. Not too clickbaity, reasonable high level overview of game theory, a (very accurate IMO) quote from old pro Erik Seidel about the state of the game just 15 years ago, a discussion on variance vs. EV and results, and most importantly, an emphasis on math, randomization techniques and emotional control rather than the TV image of staring in someone's eyes and reading their soul. Probably the one biggest misconception people have is that pros have sick reading abilities since TV likes to emphasize staredowns, when the actual single biggest skill long term pros have is the ability to lose hand after hand for hours and still play their best game.

Incidentally, the anecdote at the top of the article is pretty intuitive game theoretically. Basically on the river you need to bluff with some portion of your hands, or else nobody will ever call when you have a good hand. The natural portion of your hands to bluff with is the absolute worst ones - you don't want to bluff with your middling hands because you have some small chance of just winning a showdown when it checks around. On a board of Kc4c5c2d2c, 7d6d is quite likely the absolute worst hand you can hold given the action that's taken place, therefore it's the one you bluff with. (For some pot/stack sizes it's possible that you bluff so rarely that you have to choose between 7d6d/7h6h/7s6s, but that's getting into details.)

Re: How A.I. Conquered Poker

#53

Post by splonk »

tromp wrote: post #48

I like how the article gives examples of the sort of entropy players draw on to make their random choices: > Koon will often randomly select which of the solver’s tactics to employ in a given hand. He’ll glance down at the second hand on his watch, or at a poker chip to note the orientation of the casino logo as if it were a clock face, in order to generate a percentage between 1 and 100.

The watch face is the classic one, the chip orientation is new to me. I like it, it's less obvious that you're randomizing something, and given how much poker players like to fidget with their chips the orientation is probably reasonably random.

Another classic is to use the suit of your cards, but that has problems with being correlated with the state of the board. It works fine preflop.

Re: How A.I. Conquered Poker

#54

Post by splonk »

marcodiego wrote: post #6

Is there any remaining logic game which AI hasn't become superhuman? IIRC Monte Carlo tree search has done wonders in this area.

Here's a recent paper from DeepMind about their recent efforts towards generalized game solving (caveat: I haven't actually read it yet).

https://arxiv.org/pdf/2112.03178.pdf

Abstract:

"Games have a long history of serving as a benchmark for progress in artificial intelligence. Recently, approaches using search and learning have shown strong performance across a set of perfect information games, and approaches using game-theoretic reasoning and learning have shown strong performance for specific imperfect information poker variants. We introduce Player of Games, a general-purpose algorithm that unifies previous approaches, combining guided search, self-play learning, and game-theoretic reasoning. Player of Games is the first algorithm to achieve strong empirical performance in large perfect and imperfect information games — an important step towards truly general algorithms for arbitrary environments. We prove that Player of Games is sound, converging to perfect play as available computation time and approximation capacity increases. Player of Games reaches strong performance in chess and Go, beats the strongest openly available agent in heads-up no-limit Texas hold’em poker (Slumbot), and defeats the state-of-the-art agent in Scotland Yard, an imperfect information game that illustrates the value of guided search, learning, and game-theoretic reasoning"

Re: How A.I. Conquered Poker

#55

Post by jdmoreira »

dwohnitmok wrote: post #51
isaacg wrote: post #20

Earlier quoted context omitted.

As far as I know, in Magic: the Gathering, the best bots are far worse than most players. Part of the difficulty is that the rules are so complicated that there are only a couple of complete rules implementations. Beyond that, it's an imperfect information game with far more actions per game than poker, so optimal-solver techniques haven't seen success.

I think this is purely a resource issue, e.g. if Google Brain decided to make an MtG bot I would be fairly confident it would be superhuman. Even real time strategy games like Starcraft are looking like they're on the cusp of superhuman bots (Alphastar was competitive as Protoss against elite players, but did not consistently beat them).

The search tree is huge in mtg. It has to be the largest of any game. You can take actions all the time. There are triggers all the time, you can stack your actions on top of your opponent actions. Huge space really.

And then of course it's also imperfect information both in the sense of your opponent hand but also his deck. The cardpool is also very large for some formats.

I actually don't think it's solvable just by throwing MCTS at it with todays hardware but would love to know more about this, if someone else has more insight please reply.

EDIT: Oh and there is also the meta-game / deck building aspect. If you are going to win a tournament you have to have favorable matchups against most players in the room.

Re: How A.I. Conquered Poker

#56

Post by smallerfish »

conductr wrote: post #32
smallerfish wrote: post #21

I think there's room for a "battlebots meets online poker" product. Imagine buying your bot a real money budget and setting it loose in a tournament to combat other bots.

I'm actually working on this as a side project. I've been dragging my feet as I wasn't sure anyone would want to "code to play" and had no real marketing plan. It's a fun project though, especially trying to design in a way that proves I'm not manipulating the deck behind the scenes. Email in profile if anyone is up for being an alpha player.

I think the real money aspect would obviously get you some traction, but that's difficult to deal with (since you might run into online gambling laws, although you could very credibly argue that this isn't really any different from algorithmic trading). Maybe you could get in on the hype train and launch it as a crypto product?

Re: How A.I. Conquered Poker

#57

Post by smallerfish »

bloodyplonker22 wrote: post #30
smallerfish wrote: post #21

I think there's room for a "battlebots meets online poker" product. Imagine buying your bot a real money budget and setting it loose in a tournament to combat other bots.

Online poker has already been evolving in this direction. Any serious player who plays online will use player assistance tools (such as PIOsolver). Top level poker AI such as Pluribus has already proven that it can hold its own and beat the best online players such as linuslove. It is well known that online poker sites are full of poker bots now. There are programs to help users try to tell if they are playing agains…

Yeah, that was the evolution of my thought process. Online poker is already full of bots, which will get more and more sophisticated over time and defeat bot detection hurdles -- so why not embrace it and have bot-only zones? You could have entire tournaments play out in less than a minute if all participants were software.

Re: How A.I. Conquered Poker

#58

Post by michalsustr »

bluecalm wrote: post #50
michalsustr wrote: post #47

Earlier quoted context omitted.

Congrats on putting this together, it's not easy! I'm curious, what papers have you used for your implementation? I know there are a couple of reimplementations of DeepStack out there. Disclaimer: I'm doing phd in this area, generalizing to harder games than poker.

The solver was purely based on our (mine and my partner's) ideas. We tried reading some papers to find ideas to improve our solver but we had found that they are not very helpful - written in opaque language, using wrong tree representation and not focusing on practical implementation aspect. Maybe things changed since then but I haven't found anything useful at all in published papers about poker with the exception…

Interesting! We have very recently wrote a paper on the various tree representations [1]. Maybe it is the same distinction you found on your own? I judge this based on the screenshots on your website. Public state trees are amenable to nice factorization over the hands, and indeed most implementations use it, but formally they write they use the history tree notation.

[1] https://arxiv.org/abs/2112.10890

Re: How A.I. Conquered Poker

#59

Post by Reflecticon »

jstx1 wrote: post #44
soferio wrote: post #39

What teaching or training tools are out there for a very average player at no limit Texas hold’em who just wants to get at bit better to a respectable level at a modest time commitment , and does not need to be a pro-level player?

Books (there's very few good ones) Training videos 1-on-1 coaching PioSolver (and other similar software) For any of these to stick, you need to spend some amount of time studying by yourself; just consuming learning material and playing isn't enough.

Could you recommend a couple of books for a start?

Re: How A.I. Conquered Poker

#60

Post by abaldwin99 »

paxys wrote: post #22
torginus wrote: post #18

Does this mean that online poker is effectively over?

Chess has been "solved" by computers for a long time now, yet the online Chess community is bigger than ever. I don't see why it would be any different for Poker.

The difference for poker is it takes a lot more play time to sort players by skill. There's no ELO to climb and thus you don't get to play against more challenging opponents.

e.g. I'll lose 100% of my matches against a Chess GM but against the best poker player in the world I could go all in preflop every hand and still win the match ~20% of the time.

Only way to counter this variance is to play lots and lots of hands (Law of large numbers) It could take weeks/months in a single matchup to determine the better poker player.

In terms of playing for real money, in general you'll find more challenging competition at higher stakes. But then there's an incentive for bots.

Post reply on HN