Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

No limit: AI poker bot is first to beat professionals at multiplayer game

nature.com

Post reply on HN
Print view View original

41–50 of 407 posts

Re: No limit: AI poker bot is first to beat professionals at multiplayer game

#41

Post by alexashka »

The title is misleading - bots have been beating no limit pros in 1v1 matches for quite some time.

This is for 6-man games. The article mentions 10,000 hands - this is a very small sample size to draw any real conclusions, as anyone who has dabbled in online poker for more than a few thousand dollars can attest to. Regardless - it's trivial to write a bot that'll beat 90% of the players, as site runners can all attest to (bots are a serious problem that is not new). What does it matter that a bot can beat 'the best' or 'professionals'? It's enough that it can do better than the vast majority, outside of dystopian woes about robots taking over or being 'superior' to human beings.

Glossing over all that - I am curious if this can be used for something other than ruining online poker, which has largely already been ruined by allowing multi-tabling professionals with custom software that gathers statistics on players (data mining), existing bots, US government and irresponsible (criminal) site runners (looking at you ultimate bet)

Re: No limit: AI poker bot is first to beat professionals at multiplayer game

#42

Post by cortesoft »

r00fus wrote: post #3

I was really hoping the article would go into more detail on how the AI engaged with the human players. Was it online? the picture on the article seems to imply IRL. If IRL, what inputs did it have, simply cards shown or could it read tells? Did those players know they were playing an AI?

Tells aren't really a thing for top level poker players.

Re: No limit: AI poker bot is first to beat professionals at multiplayer game

#44

Post by pezo1919 »

noambrown wrote: post #23
DennisP wrote: post #21

Earlier quoted context omitted.

Are any papers available yet? Is the bot going for game-theory-optimal play, or trying to exploit weaknesses in other players?

The paper is here: https://science.sciencemag.org/content/early/2019/07/10/scie... It's going for game-theory-optimal play. It doesn't adapt to its opponents' observed weaknesses. But I think it's cool to show that you don't need to adapt to opponent weaknesses to win at poker at the highest levels. You just need to not have any weaknesses yourself.

I thought myself the same. However if players do expose each others weaknesses fast enough it could lead to a chip gain which might be hard to overcome right? Just in theory ofc. :)

Re: No limit: AI poker bot is first to beat professionals at multiplayer game

#45

Post by MFLoon »

gfody wrote: post #38
noambrown wrote: post #23

Earlier quoted context omitted.

The paper is here: https://science.sciencemag.org/content/early/2019/07/10/scie... It's going for game-theory-optimal play. It doesn't adapt to its opponents' observed weaknesses. But I think it's cool to show that you don't need to adapt to opponent weaknesses to win at poker at the highest levels. You just need to not have any weaknesses yourself.

> you don't need to adapt to opponent weaknesses to win at poker at the highest levels that may be true for limit poker, but in a no-limit tournament the best this bot could do is not lose. as the pressure increases with the blinds and the players are forced to bluff and call bluffs how does this bot avoid folding itself to death from a run of bad cards? I could see this bot doing well at cashing but I don't see how…

I couldn't find it confirmed in the primary or secondary article, but I would bet the bot is just playing cash at a fixed stack depth rather than a tournament; just like in the wild, bots are much more of a problem in online cash than online tournaments. Dynamically adjusting strategies by stack depth, number of players, and pay jumps, would probably be several orders of magnitude more complex.

Re: No limit: AI poker bot is first to beat professionals at multiplayer game

#46

Post by DennisP »

gfody wrote: post #38
noambrown wrote: post #23

Earlier quoted context omitted.

The paper is here: https://science.sciencemag.org/content/early/2019/07/10/scie... It's going for game-theory-optimal play. It doesn't adapt to its opponents' observed weaknesses. But I think it's cool to show that you don't need to adapt to opponent weaknesses to win at poker at the highest levels. You just need to not have any weaknesses yourself.

> you don't need to adapt to opponent weaknesses to win at poker at the highest levels that may be true for limit poker, but in a no-limit tournament the best this bot could do is not lose. as the pressure increases with the blinds and the players are forced to bluff and call bluffs how does this bot avoid folding itself to death from a run of bad cards? I could see this bot doing well at cashing but I don't see how…

Optimal play includes bluffing. It's "optimal" according to game theory.

For example, game theory may tell you that in a particular situation, you can't be exploited if you bluff 10% of the time. If the opponent bluffs less than that, you can come out ahead by more often folding when he bets. If the opponent bluffs more than 10%, you can call or reraise when he bets. But if he bluffs the optimal amount, it doesn't matter either way, you can't take advantage of him.

So this bot would bluff at 10% to avoid getting exploited, but wouldn't try to detect whether the opponent is exploitable. (The latter is risky since a crafty opponent can switch up strategies, manipulating you into playing an exploitable strategy.)

Re: No limit: AI poker bot is first to beat professionals at multiplayer game

#47

Post by WA »

gfody wrote: post #38
noambrown wrote: post #23

Earlier quoted context omitted.

The paper is here: https://science.sciencemag.org/content/early/2019/07/10/scie... It's going for game-theory-optimal play. It doesn't adapt to its opponents' observed weaknesses. But I think it's cool to show that you don't need to adapt to opponent weaknesses to win at poker at the highest levels. You just need to not have any weaknesses yourself.

> you don't need to adapt to opponent weaknesses to win at poker at the highest levels that may be true for limit poker, but in a no-limit tournament the best this bot could do is not lose. as the pressure increases with the blinds and the players are forced to bluff and call bluffs how does this bot avoid folding itself to death from a run of bad cards? I could see this bot doing well at cashing but I don't see how…

Isn’t this survivorship bias, or do you know which player repeatedly will place 1st beforehand? Granted that poker is pretty popular, there must be quite a few people who always become first place.

Or to turn this around: given enough bots, some bots will place 1st a lot more than others. It’s just unclear which one.

Re: No limit: AI poker bot is first to beat professionals at multiplayer game

#48

Post by noambrown »

auggierose wrote: post #32

This is fascinating stuff. So do I understand this right, Liberatus worked using computing the Nash equilibrium, while the new multiplayer version works using self-play like AlphaGo Zero? Did you run the multiplayer version against the two-player version? If yes, how did it go? Could you recommend a series of books / papers that can take me from zero to being able to reprogram this (I know programming and mathematics…

Training was super cheap. It would cost under $150 on cloud computing services.

The training aspect has some improvements but is at its core similar to Libratus. The search algorithm is the biggest difference.

There aren't that many great resources out there for helping new people get caught up to speed on this area. That's something we hope to fix in the future. Maybe this would be a good place to start? http://modelai.gettysburg.edu/2013/cfr/cfr.pdf

Re: No limit: AI poker bot is first to beat professionals at multiplayer game

#49

Post by baq »

pennaMan wrote: post #25
neural_thing wrote: post #5

How long until a slightly worse version of this model is reverse engineered and appears at every table in online poker?

I'm wondering how long until poker games will require a captcha on every round

Captchas are easier than the game

Re: No limit: AI poker bot is first to beat professionals at multiplayer game

#50

Post by MFLoon »

noambrown wrote: post #28
eries wrote: post #8

Earlier quoted context omitted.

Can you share some about what strategies the bot prefers and how these compare with common professional human strategies?

We talk about this a bit in the paper. Based on the feedback from the pros, the bot seems to "donk bet" (call and then bet on the next round) much more than human pros do. It also randomizes between multiple bet sizes, including very large bet sizes, while humans stick to just one or two sizes depending on the situation.

When human players donk bet it's almost always a weak player employing an extremely exploitable strategy, whereas pros almost never do it because the metagame has evolved around the presumption that nobody ever donk bets. I'd love to see what the bot's balanced GTO donking strategy looks like.
Post reply on HN