Live data from Hacker News

No limit: AI poker bot is first to beat professionals at multiplayer game

nature.com

61–70 of 407 posts

Re: No limit: AI poker bot is first to beat professionals at multiplayer game

#61
post #54

Curious, why was 100BB used for six max? If I recall right, the head ups experiment was 200BB?

Guessing it's because it's most similar to a regular 6max game. Also it should limit lower the number of possible ways to play a hand, less chips means the correct choices are easier, so maybe it's computationally easier

Yeah, my question for the author is if stack depth was relevant for this experiment. In headups, they exhausted the entire tree, in six max, they went to a fixed depth.

Re: No limit: AI poker bot is first to beat professionals at multiplayer game

#62

Earlier quoted context omitted.

Any chance of the code being released or a cepheus style answer key being provided? http://poker.srv.ualberta.ca/strategy

I don't think the poker world would be happy with us if we did that. Heads-up limit hold'em isn't really played professionally anymore, but six-player no-limit hold'em is very popular.

It depends who you ask. I think it's inevitable that it's released one day. By not releasing you're just delaying it.

All the top high stakes players already have solvers that they've spent a lot of money developing and studying privately. They would definitely be upset with you, but by releasing the code you are democratizing the information to all the midstakes pros who want to study but don't have the resources to pay developers and solve the game privately.

Re: No limit: AI poker bot is first to beat professionals at multiplayer game

#63

Earlier quoted context omitted.

Any chance of the code being released or a cepheus style answer key being provided? http://poker.srv.ualberta.ca/strategy

I don't think the poker world would be happy with us if we did that. Heads-up limit hold'em isn't really played professionally anymore, but six-player no-limit hold'em is very popular.

Have you thought about open sourcing the non-AI pieces? It would be great for other researchers so they wouldn’t have to build the poker pieces from scratch

Re: No limit: AI poker bot is first to beat professionals at multiplayer game

#64
post #53

I'm one of the authors of the bot, AMA

How do you think these same pros would do in a follow-up match? As described in the article, the bot put players off their game with much more varied betting and with donks. Do you think the margin would decrease as players are exposed to these strategies? Players face mental fatigue and have so over-learned their existing strategies that it takes time to adapt new strategies and even more time for those new strategi…

In the paper we include a graph of performance over the course of the 10,000-hand 5 humans + 1 AI experiment that was played over 12 days. There's no indication that the bot's performance decreased over time (there is a temporary downward blip in the middle, but that's likely just variance). Based on discussions with pros, it sounds like they didn't find any weaknesses and they didn't seem to think they'd find any given more time.

Re: No limit: AI poker bot is first to beat professionals at multiplayer game

#65
post #53

I'm one of the authors of the bot, AMA

How do you think these same pros would do in a follow-up match? As described in the article, the bot put players off their game with much more varied betting and with donks. Do you think the margin would decrease as players are exposed to these strategies? Players face mental fatigue and have so over-learned their existing strategies that it takes time to adapt new strategies and even more time for those new strategi…

I also suspect it would not be able to maintain a ~40bb/100 hand win rate. The thing about human players is, while the best are capable of learning and employing truly balanced GTO strategies, in practice they rarely adhere to these because other humans (even good pros) will still have exploitable flaws in their strategies, and attempting to exploit these will be more profitable than sticking to the unexploitable strategy; of course it also opens the exploiter to counter-exploitation, creating a fluctuating cycle of players trying to exploit, getting exploited, then moving back towards playing unexploitably. That's the normal state of a pro's strategy in a given game - so to switch to a steady state of always playing unexploitably would be a fairly big adjustment even to top tier pros who are capable of it.

Re: No limit: AI poker bot is first to beat professionals at multiplayer game

#66
post #8

Earlier quoted context omitted.

Can you share some about what strategies the bot prefers and how these compare with common professional human strategies?

We talk about this a bit in the paper. Based on the feedback from the pros, the bot seems to "donk bet" (call and then bet on the next round) much more than human pros do. It also randomizes between multiple bet sizes, including very large bet sizes, while humans stick to just one or two sizes depending on the situation.

Is there a way to see the EV the bot is calculating when it's deciding between checking and donk betting? When you place these spots in solvers, they actually advocate for a significant amount of donk betting on certain boards, but pros don't do it because the EV is marginal and it's better for pros to simplify their strategy so they make less mistakes. If you have a flop donk bet strategy, you also have to develop a corresponding turn and river strategy, which makes it extremely difficult.

Re: No limit: AI poker bot is first to beat professionals at multiplayer game

#67

I'm one of the authors of the bot, AMA

Knowing when to bluff often depends on the psychology of the opponent, but since it trained playing itself it doesn't seem that knowing when to bluff would be learned. Did it bluff very often?

At the highest levels of play psychological factors are pretty minimal. Before a showdown which cards you actually hold aren't particularly material, as the only information you convey is through your bids. This means if you predict that you're more likely to win a hand by bidding (and inducing a fold) than by calling and going to a showdown it makes mathematical sense to "bluff". I'm sure AIs have no trouble learning that fact.

Re: No limit: AI poker bot is first to beat professionals at multiplayer game

#68

I'm one of the authors of the bot, AMA

Neal - super interesting stuff. Couple of questions: 1) What were the reasons for choosing 6-handed play (assuming logistical and costs)? It would be interesting to see how the bot’s strategy would differ in a full ring game. 2) Are there any plans to commercialize the bot as a tool for training human players?

6-handed is a very common format online.

Re: No limit: AI poker bot is first to beat professionals at multiplayer game

#69
post #58
post #45

Earlier quoted context omitted.

I couldn't find it confirmed in the primary or secondary article, but I would bet the bot is just playing cash at a fixed stack depth rather than a tournament; just like in the wild, bots are much more of a problem in online cash than online tournaments. Dynamically adjusting strategies by stack depth, number of players, and pay jumps, would probably be several orders of magnitude more complex.

> There were two formats for the experiment: five humans playing with one AI at the table, and one human playing with five copies of the AI at the table. In each case, there were six players at the table with 10,000 chips at the start of each hand. The small blind was 50 chips, and the big blind was 100 chips. In the fb article linked above.

Ah thanks. As I suspected, cash game with fixed 100bb stacks.

Re: No limit: AI poker bot is first to beat professionals at multiplayer game

#70
post #30

How long until a slightly worse version of this model is reverse engineered and appears at every table in online poker?

Slightly worse versions are already out in the wild. Bot using the published technique will be live in a couple of months tops.

Colluding bots are the main worry if you play online though.
Post reply on HN