No limit: AI poker bot is first to beat professionals at multiplayer game • Hacker News Board

Re: No limit: AI poker bot is first to beat professionals at multiplayer game

#141

Post by snarf21 » Thu, Jul 11, 2019, 10:26 PM UTC

MFLoon wrote: post #65

snarf21 wrote: post #53

Earlier quoted context omitted.

How do you think these same pros would do in a follow-up match? As described in the article, the bot put players off their game with much more varied betting and with donks. Do you think the margin would decrease as players are exposed to these strategies? Players face mental fatigue and have so over-learned their existing strategies that it takes time to adapt new strategies and even more time for those new strategi…

I also suspect it would not be able to maintain a ~40bb/100 hand win rate. The thing about human players is, while the best are capable of learning and employing truly balanced GTO strategies, in practice they rarely adhere to these because other humans (even good pros) will still have exploitable flaws in their strategies, and attempting to exploit these will be more profitable than sticking to the unexploitable str…

Yeah, that is kinda what I was trying to tease out. These 10K hands are nothing compared to the XM of hands these pros have already played. It would be interesting to see how well they did after 1M hands. I'm sure the bot would likely still have an edge but I'd assume the players would adjust their strategy and but less confused by the random sized bets.

I was also confused by the sample videos where everyone had $10K at the start of each of the demo hands. It was unclear to me if that just the simulation of the hands or actual game play. If everyone starts every hand with $10K, then the feat seems less strong as going all-in has less risk.

Re: No limit: AI poker bot is first to beat professionals at multiplayer game

#142

Post by traderjane » Thu, Jul 11, 2019, 10:29 PM UTC

amelius wrote: post #130

noambrown wrote: post #113

Earlier quoted context omitted.

I think it took the community a while to come up with the right algorithms. So much of early AI research was focused on beating humans at chess and later Go. But those techniques don't directly carry over to an imperfect-information game like poker. The challenge of hidden information was kind of neglected by the AI community. This line of research really has its origins in the game theory community actually (which i…

So will this be the end of online poker?

Even with a magical human test, you couldn't know whether it was human + robot performance.

Re: No limit: AI poker bot is first to beat professionals at multiplayer game

#143

Post by csa » Thu, Jul 11, 2019, 10:30 PM UTC

friendlybus wrote: post #109

ehsankia wrote: post #86

Earlier quoted context omitted.

This is something I've always wondered, how come bots haven't taken over online poker considering how much money there is to be made, and all you need is to be slightly better than average right? Is high level poker really that hard to achieve?

Post blizerian's time in poker, the human players in online games use statistics to make insane bets on odds. They play many many games at once and just look for the opening and make the insane bets when the openings come up. They've done the math that it's worth it to do those kinds of bets.

I did this with great success.

For reference, I once folded top boat to quads (he showed) to a river all in raise in PLO to a dude who had a 100% win showdown when raise river stat over several thousand hands. Other stats confirmed he was a nit, so it was an easy fold. Iirc, this was PLO 200 or PLO 400 — I never saw anyone that nitty at the PLO 1000 or PLO 2000 tables.

FWIW, I did a lot more than “look for an opening”, although I did a lot of that. I tried to play GTO as much as possible, but I would adjust to people who were exploitable when they called too much, folded too much, or were too aggressive into weakness.

I spent a lot of time away from the table analyzing stats of the regulars to find leaks to exploit. It was worth the time, and it made it much easier to play 8-12 tables of PLO.

Re: No limit: AI poker bot is first to beat professionals at multiplayer game

#144

Post by ryandrake » Thu, Jul 11, 2019, 10:32 PM UTC

noambrown wrote: post #6

I'm one of the authors of the bot, AMA

Interesting that the conventional wisdom of never open limping emerged as confirmed through self-play. What other general poker “best practices” were either confirmed or upended through this research?

Re: No limit: AI poker bot is first to beat professionals at multiplayer game

#145

Post by russelldavis » Thu, Jul 11, 2019, 10:32 PM UTC

MFLoon wrote: post #106

newfangle wrote: post #101

Earlier quoted context omitted.

Donk bets exist in the meta, ie when the turn is extremely good for your range but is horrible for your opponent. ( if you have a fd on the flop and it hits on the turn you can overbet the pot on the turn with your bluffs and foushes then just go all in on the river) if they have top pair its pretty hard to play against that

Oh, sure. I more meant flop donk bets; I guess it doesn't specify which street the donking was happening.

The same logic can apply to flop donk bets. Some flops favor the donking player's range more than their opponent.

Re: No limit: AI poker bot is first to beat professionals at multiplayer game

#146

Post by nickpsecurity » Thu, Jul 11, 2019, 10:34 PM UTC

nradov wrote: post #85

noambrown wrote: post #51

Earlier quoted context omitted.

I don't think the poker world would be happy with us if we did that. Heads-up limit hold'em isn't really played professionally anymore, but six-player no-limit hold'em is very popular.

Why are you concerned about the happiness of the poker world?

It could just be for ethical reasons. I think anbop has a good reason even for unethical folks: hitting the best players hard in their wallets will definitely make it harder to recruit them for comparisons that validate these experiments. My prediction is that releasing this software will lead to profitable cheating like what people do with Blackjack at casinos.

Re: No limit: AI poker bot is first to beat professionals at multiplayer game

#147

Post by vagab0nd » Thu, Jul 11, 2019, 10:34 PM UTC

noambrown wrote: post #6

I'm one of the authors of the bot, AMA

Very interesting results. From the paper it sounds like the algorithms you used are very similar to Libratus (pre-solved blueprint + subgame solving). What change made it so that the computation requirement is much lower now?

Re: No limit: AI poker bot is first to beat professionals at multiplayer game

#148

Post by tc » Thu, Jul 11, 2019, 10:40 PM UTC

noambrown wrote: post #6

I'm one of the authors of the bot, AMA

Could you perhaps speak to some of the engineering details that the paper glosses over. E.g.:

- Are the action and information abstraction procedures hand-engineered or learned in some manner?

- How does it decide how many bets to consider in a particular situation?

- Is there anything interesting going on with how the strategy is compressed in memory?

- How do you decide in the first betting round if a bet is far enough off-tree that online search is needed?

- When searching beyond leaf nodes, how did you choose how far to bias the strategies toward calling, raising, and folding?

- After it calculates how it would act with every possible hand, how does it use that to balance its strategy while taking into account the hand it is actually holding?

- In general, how much do these kind of engineering details and hyperparameters matter to your results and to the efficiency of training? How much time did you spend on this? Roughly how many lines of code are important for making this work?

- Why does this training method work so well on CPUs vs GPUs? Do you think there are any lessons here that might improve training efficiency for 2-player perfect-information systems such as AlphaZero?

Re: No limit: AI poker bot is first to beat professionals at multiplayer game

#149

Post by RivieraKid » Thu, Jul 11, 2019, 10:46 PM UTC

noambrown wrote: post #6

I'm one of the authors of the bot, AMA

Can you share more details about the abstraction? The paper is kind of vague on it. How does it decide if it should use 1 or 14 bet values? Is it a perfect recall abstraction? How many information sets are there?

Re: No limit: AI poker bot is first to beat professionals at multiplayer game

#150

Post by MFLoon » Thu, Jul 11, 2019, 10:51 PM UTC

russelldavis wrote: post #145

MFLoon wrote: post #106

Earlier quoted context omitted.

Oh, sure. I more meant flop donk bets; I guess it doesn't specify which street the donking was happening.

The same logic can apply to flop donk bets. Some flops favor the donking player's range more than their opponent.

Yea I'm not saying it's impossible to devise an unexploitable flop donking strategy. I think the reason thinking players generally don't is because of the complexity of adding significantly more branches early in the game tree - basically going from 3 (check-{fold,call,raise}) to 6 (those 3 plus donk-{fold,call,raise}).