Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

No limit: AI poker bot is first to beat professionals at multiplayer game

nature.com

Post reply on HN
Print view View original

71–80 of 407 posts

Re: No limit: AI poker bot is first to beat professionals at multiplayer game

#71

Post by noambrown »

ewhauser421 wrote: post #57
noambrown wrote: post #6

I'm one of the authors of the bot, AMA

Neal - super interesting stuff. Couple of questions: 1) What were the reasons for choosing 6-handed play (assuming logistical and costs)? It would be interesting to see how the bot’s strategy would differ in a full ring game. 2) Are there any plans to commercialize the bot as a tool for training human players?

1) The goal was to show convincingly that we could handle multi-player poker. The exact number of players was kind of arbitrary. We chose six-player because that's the most common/popular format. Considering training the 6-player bot would cost less than $150 on a cloud computing service, I think it's safe to say these techniques would all work fine in other formats.

2) I'm quite happy working on fundamental AI research and plan to continue in that direction.

Re: No limit: AI poker bot is first to beat professionals at multiplayer game

#72

Post by TheChosenZygote »

noambrown wrote: post #64
snarf21 wrote: post #53

Earlier quoted context omitted.

How do you think these same pros would do in a follow-up match? As described in the article, the bot put players off their game with much more varied betting and with donks. Do you think the margin would decrease as players are exposed to these strategies? Players face mental fatigue and have so over-learned their existing strategies that it takes time to adapt new strategies and even more time for those new strategi…

In the paper we include a graph of performance over the course of the 10,000-hand 5 humans + 1 AI experiment that was played over 12 days. There's no indication that the bot's performance decreased over time (there is a temporary downward blip in the middle, but that's likely just variance). Based on discussions with pros, it sounds like they didn't find any weaknesses and they didn't seem to think they'd find any gi…

I think it would be hard for the pros to find exploits against the bot, but they could definitely lose less. When using solvers, pros generally only input a couple of sizings for bets, and avoid 2x+ pot sizings, which from the video it seemed like the bot used at much higher frequencies than other pros.

Re: No limit: AI poker bot is first to beat professionals at multiplayer game

#73

Post by elcomet »

femto113 wrote: post #67
waynecochran wrote: post #52

Earlier quoted context omitted.

Knowing when to bluff often depends on the psychology of the opponent, but since it trained playing itself it doesn't seem that knowing when to bluff would be learned. Did it bluff very often?

At the highest levels of play psychological factors are pretty minimal. Before a showdown which cards you actually hold aren't particularly material, as the only information you convey is through your bids. This means if you predict that you're more likely to win a hand by bidding (and inducing a fold) than by calling and going to a showdown it makes mathematical sense to "bluff". I'm sure AIs have no trouble learnin…

The issue is that you don't know exactly the probability of your opponent folding.

This is psychology.

Re: No limit: AI poker bot is first to beat professionals at multiplayer game

#74

Post by [deleted] »

DevX101 wrote: post #30
neural_thing wrote: post #5

How long until a slightly worse version of this model is reverse engineered and appears at every table in online poker?

Slightly worse versions are already out in the wild. Bot using the published technique will be live in a couple of months tops.

[deleted]

Re: No limit: AI poker bot is first to beat professionals at multiplayer game

#75

Post by Bartweiss »

pezo1919 wrote: post #44
noambrown wrote: post #23

Earlier quoted context omitted.

The paper is here: https://science.sciencemag.org/content/early/2019/07/10/scie... It's going for game-theory-optimal play. It doesn't adapt to its opponents' observed weaknesses. But I think it's cool to show that you don't need to adapt to opponent weaknesses to win at poker at the highest levels. You just need to not have any weaknesses yourself.

I thought myself the same. However if players do expose each others weaknesses fast enough it could lead to a chip gain which might be hard to overcome right? Just in theory ofc. :)

This is a great question. I wonder how this bot would do in a game with a couple of pros and a couple of reasonably skilled amateurs?

Still well, I suspect, since straightforward theoretically-correct poker will take money off the amateurs efficiently. But it seems possible that playing to wipe out weaker or less consistent players could provide enough margin to bully the more stable AI player.

Re: No limit: AI poker bot is first to beat professionals at multiplayer game

#77

Post by noambrown »

ewhauser421 wrote: post #63
noambrown wrote: post #51

Earlier quoted context omitted.

I don't think the poker world would be happy with us if we did that. Heads-up limit hold'em isn't really played professionally anymore, but six-player no-limit hold'em is very popular.

Have you thought about open sourcing the non-AI pieces? It would be great for other researchers so they wouldn’t have to build the poker pieces from scratch

There is some open-source code in this area, and hopefully there will be more going forward. Here's one example: https://github.com/EricSteinberger/Deep-CFR

Re: No limit: AI poker bot is first to beat professionals at multiplayer game

#79

Post by noambrown »

waynecochran wrote: post #52
noambrown wrote: post #6

I'm one of the authors of the bot, AMA

Knowing when to bluff often depends on the psychology of the opponent, but since it trained playing itself it doesn't seem that knowing when to bluff would be learned. Did it bluff very often?

The bot does bluff, and in fact it learns from self-play that bluffing is (sometimes) the optimal thing to do. At the end of the day, bluffing is simply betting when you have a weak hand. The bot learns from experience that when it bets with a weak hand, the opponent (another copy of itself) sometimes folds and it makes more money than if it hadn't bet. The bot doesn't view it as deceptive or dishonest. It just views it as the action that makes it the most money.

Of course, a key part of bluffing is getting the probabilities right. You can't always bluff and you can't never bluff, because that would make you too predictable. But our self-play and search algorithms are designed to get those probabilities right.

Re: No limit: AI poker bot is first to beat professionals at multiplayer game

#80

Post by bcassedy »

pezo1919 wrote: post #44
noambrown wrote: post #23

Earlier quoted context omitted.

The paper is here: https://science.sciencemag.org/content/early/2019/07/10/scie... It's going for game-theory-optimal play. It doesn't adapt to its opponents' observed weaknesses. But I think it's cool to show that you don't need to adapt to opponent weaknesses to win at poker at the highest levels. You just need to not have any weaknesses yourself.

I thought myself the same. However if players do expose each others weaknesses fast enough it could lead to a chip gain which might be hard to overcome right? Just in theory ofc. :)

This is true in tournament play. In a cash game it doesn't matter since there is no elimination, you can always rebuy.

And it's not like in the movies where if you don't have the money to call a bet, you lose. You simply are considered all in for the main pot and then sidepots that you aren't eligible to win will be created for any bets you can't cover.

Post reply on HN