Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

No limit: AI poker bot is first to beat professionals at multiplayer game

nature.com

Post reply on HN
Print view View original

101–110 of 407 posts

Re: No limit: AI poker bot is first to beat professionals at multiplayer game

#101

Post by newfangle »

MFLoon wrote: post #50
noambrown wrote: post #28

Earlier quoted context omitted.

We talk about this a bit in the paper. Based on the feedback from the pros, the bot seems to "donk bet" (call and then bet on the next round) much more than human pros do. It also randomizes between multiple bet sizes, including very large bet sizes, while humans stick to just one or two sizes depending on the situation.

When human players donk bet it's almost always a weak player employing an extremely exploitable strategy, whereas pros almost never do it because the metagame has evolved around the presumption that nobody ever donk bets. I'd love to see what the bot's balanced GTO donking strategy looks like.

Donk bets exist in the meta, ie when the turn is extremely good for your range but is horrible for your opponent. ( if you have a fd on the flop and it hits on the turn you can overbet the pot on the turn with your bluffs and foushes then just go all in on the river) if they have top pair its pretty hard to play against that

Re: No limit: AI poker bot is first to beat professionals at multiplayer game

#102

Post by wallawe »

bluetwo wrote: post #82
noambrown wrote: post #6

I'm one of the authors of the bot, AMA

First of all, I laughed at the 20-second average per game in self-play, since I ran into the same thing and have been trying to speed up the algorithm but haven't been able to get it faster (without throwing more hardware at it). Second, I haven't read everything, but I believe you are playing a cash-game and not tournament-style. Is that correct? If that is the case, any chance you will be doing a tourney-style vers…

> In Tourney play, the top 2 or 3 players get paid out

Or top 2 or 3 thousand... depends on the tournament but it's usually the top 15% ish.

Re: No limit: AI poker bot is first to beat professionals at multiplayer game

#104

Post by n3k5 »

noambrown wrote: post #6

I'm one of the authors of the bot, AMA

What took you so long? I mean not the Pluribus team specifically, but Poker AI researchers in general.

The desire to master this sort of game has inspired the development of entire branches of mathematics. Computers are better at maths than humans. They're less prone to hazardous cognitive biases (gambler's fallacy etc.) and can put on an excellent poker face.

As a layperson who's rather ignorant about both no-limit Texas hold 'em and applicable AI techniques, my intuition would tell me that super-human hold 'em should have been achieved before super-human Go. Apparently your software requires way less CPU power than AlphaGo/AlphaZero, which seems to support my hypothesis. What am I missing?

Bonus questions in case you have the time and inclination to oblige:

What does this mean for people who like to play on-line Poker for real money?

Could you recommend some literature (white papers/books/lecture series/whatever) to someone interested in writing an AI (running on potato-grade hardware) for a niche "draft and pass" card game (e.g. Sushi Go!) as a recreational programming exercise?

Re: No limit: AI poker bot is first to beat professionals at multiplayer game

#106

Post by MFLoon »

newfangle wrote: post #101
MFLoon wrote: post #50

Earlier quoted context omitted.

When human players donk bet it's almost always a weak player employing an extremely exploitable strategy, whereas pros almost never do it because the metagame has evolved around the presumption that nobody ever donk bets. I'd love to see what the bot's balanced GTO donking strategy looks like.

Donk bets exist in the meta, ie when the turn is extremely good for your range but is horrible for your opponent. ( if you have a fd on the flop and it hits on the turn you can overbet the pot on the turn with your bluffs and foushes then just go all in on the river) if they have top pair its pretty hard to play against that

Oh, sure. I more meant flop donk bets; I guess it doesn't specify which street the donking was happening.

Re: No limit: AI poker bot is first to beat professionals at multiplayer game

#107

Post by junar »

rightbyte wrote: post #59
JaRail wrote: post #15

So let me see if I understand this. I don't believe it's hard to write a probabilistic program to play poker. That's enough to win against humans in 2-player. With one AI and multiple professional human players sitting at a physical table, the humans outperform the probabilistic model because they take advantage of each other's mistakes/styles. Some players crash out faster but the winner gets ahead of the safe proba…

Interesting article. Too bad a don't have a subscription to read the paper. The bot played like 10 000 hands. There is no way that is enough to prove it's better or worse than the opponents. More so in no-limit where some key all-ins can turn the game up side down. The variance is higher than limit or fixed, right? I did a heads up Texas holdem fixed bot with "counter factual regret minimization" like 8 years ago fro…

They mention that they use AIVAT to reduce variance.

> Although poker is a game of skill, there is an extremely large luck component as well. It is common for top professionals to lose money even over the course of 10,000 hands of poker simply because of bad luck. To reduce the role of luck, we used a version of the AIVAT[1] variance reduction algorithm, which applies a baseline estimate of the value of each situation to reduce variance while still keeping the samples unbiased. For example, if the bot is dealt a really strong hand, AIVAT will subtract a baseline value from its winnings to counter the good luck. This adjustment allowed us to achieve statistically significant results with roughly 10x fewer hands than would normally be needed.

[1] https://arxiv.org/abs/1612.06915

Re: No limit: AI poker bot is first to beat professionals at multiplayer game

#108

Post by noambrown »

bluetwo wrote: post #82
noambrown wrote: post #6

I'm one of the authors of the bot, AMA

First of all, I laughed at the 20-second average per game in self-play, since I ran into the same thing and have been trying to speed up the algorithm but haven't been able to get it faster (without throwing more hardware at it). Second, I haven't read everything, but I believe you are playing a cash-game and not tournament-style. Is that correct? If that is the case, any chance you will be doing a tourney-style vers…

We're doing cash games in this experiment. At the end of the day, this is about advancing AI, not about making a poker bot. Going from two-player to multi-player has important implications for AI beyond just poker. I don't think the same is true for cash game vs tournament.

There's a cash game almost every night at the FBNY office! I don't usually play though -- I'm not nearly as good as the bot.

Re: No limit: AI poker bot is first to beat professionals at multiplayer game

#109

Post by friendlybus »

ehsankia wrote: post #86
neural_thing wrote: post #5

How long until a slightly worse version of this model is reverse engineered and appears at every table in online poker?

This is something I've always wondered, how come bots haven't taken over online poker considering how much money there is to be made, and all you need is to be slightly better than average right? Is high level poker really that hard to achieve?

Post blizerian's time in poker, the human players in online games use statistics to make insane bets on odds. They play many many games at once and just look for the opening and make the insane bets when the openings come up. They've done the math that it's worth it to do those kinds of bets.

Re: No limit: AI poker bot is first to beat professionals at multiplayer game

#110

Post by trishume »

asdfman123 wrote: post #103

So, is this the end of online poker? Will it just become increasingly sophisticated bots playing each other online?

I'm really confused about why stock for the company that makes PokerStars hasn't moved at all today: https://www.google.com/search?tbm=fin&q=TSE:+TSGI#scso=_wqsn...

The fact that there's a published recipe for a superhuman bot that can be trained for $150 and run on any desktop computer sounds like an existential threat to their business.

The main mitigating factor I can think of is that you'd need to also adversarially train it so it isn't distinguishable from a skilled human. But that doesn't seem like it would be too difficult.

Post reply on HN