Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI Beats Four Top Poker Players

bbc.co.uk

Post reply on HN
Print view View original

221–230 of 234 posts

Re: AI Beats Four Top Poker Players

#221

Post by xapata »

6nf wrote: post #215
xapata wrote: post #151

I suspect Libratus' overbet frequency is overfit to this particular reduced-variance game format. In a normal game, the opponent doesn't take chips off the table after winning a hand and might stand up at any moment. It's hard to know how much that affected the strategy, but in the Reddit thread, the human players said the overbet frequency was what they were most surprised by.

Maybe overbetting is better EV and humans just don't know it yet. It will be interesting to see if this changes the game at the higher stakes Heads Up tables

Yes, but expected value isn't everything. A good investor considers the ratio of expected return to variance of return. In a normal game, the variance on an overbet is enormous and therefore the denominator of its Sharpe ratio [0] is large and drives the value (or should I say "score" so as not to get confused with "expected value"?) of that action down.

You'd happily go all-in pre-flop AA vs KK. On the other hand if you got 4-bet pre-flop by a 22 and you're holding AK, you might ask, "Check it down?" This is assuming the opponent really likes small pocket pairs and will call an all-in, etc.

[0] https://en.wikipedia.org/wiki/Sharpe_ratio

Re: AI Beats Four Top Poker Players

#222

Post by hanasu »

6nf wrote: post #215
xapata wrote: post #151

I suspect Libratus' overbet frequency is overfit to this particular reduced-variance game format. In a normal game, the opponent doesn't take chips off the table after winning a hand and might stand up at any moment. It's hard to know how much that affected the strategy, but in the Reddit thread, the human players said the overbet frequency was what they were most surprised by.

Maybe overbetting is better EV and humans just don't know it yet. It will be interesting to see if this changes the game at the higher stakes Heads Up tables

Theoretically the best way to maximize an edge is to just wager larger amounts at all times. This has a beneficial side effect of deviating from 'standard' human play which can be further exploited, although I doubt this the intention of the AI.

Re: AI Beats Four Top Poker Players

#223

Post by [deleted] »

afastow wrote: post #111

Sounds like the end of online poker is very near. This version required a supercomputer and took a long time to decide its actions but those type of things tend to be quickly improved given enough motivation. Even if poker sites could somehow perfectly detect automated players(which they can't of course), highly skilled poker is profitable enough that some people would be willing to manually execute the actions thems…

[deleted]

Re: AI Beats Four Top Poker Players

#224

Post by 6nf »

xapata wrote: post #221
6nf wrote: post #215

Earlier quoted context omitted.

Maybe overbetting is better EV and humans just don't know it yet. It will be interesting to see if this changes the game at the higher stakes Heads Up tables

Yes, but expected value isn't everything. A good investor considers the ratio of expected return to variance of return. In a normal game, the variance on an overbet is enormous and therefore the denominator of its Sharpe ratio [0] is large and drives the value (or should I say "score" so as not to get confused with "expected value"?) of that action down. You'd happily go all-in pre-flop AA vs KK. On the other hand if…

> you might ask, "Check it down?"

In a heads up game? Really? "Check it down?" ???

Re: AI Beats Four Top Poker Players

#225

Post by xapata »

hanasu wrote: post #222
6nf wrote: post #215

Earlier quoted context omitted.

Maybe overbetting is better EV and humans just don't know it yet. It will be interesting to see if this changes the game at the higher stakes Heads Up tables

Theoretically the best way to maximize an edge is to just wager larger amounts at all times. This has a beneficial side effect of deviating from 'standard' human play which can be further exploited, although I doubt this the intention of the AI.

Maximum expected value is not always optimal. Bets are an investment with uncertain returns. In normal circumstances, portfolio theory helps us optimize our investments. This reduced-variance format removed the budget constraint and made portfolio theory irrelevant. Overbets made sense to the bot who had only trained in this particular format, but not to the humans who have trained for years with a budget.

Re: AI Beats Four Top Poker Players

#226

Post by xapata »

6nf wrote: post #224
xapata wrote: post #221

Earlier quoted context omitted.

Yes, but expected value isn't everything. A good investor considers the ratio of expected return to variance of return. In a normal game, the variance on an overbet is enormous and therefore the denominator of its Sharpe ratio [0] is large and drives the value (or should I say "score" so as not to get confused with "expected value"?) of that action down. You'd happily go all-in pre-flop AA vs KK. On the other hand if…

> you might ask, "Check it down?" In a heads up game? Really? "Check it down?" ???

The example applies more to a ring game, where you might be going after a particular fish and don't want to get involved with a better player. Still, I think it illustrates the principle of trying to keep the variance down.

Also, if you're first to act on the next round, it might be worth asking, even if you don't think they'll agree.

Re: AI Beats Four Top Poker Players

#227

Post by xapata »

brilliantcode wrote: post #205
xapata wrote: post #160

Earlier quoted context omitted.

Ha! That's an amusing interpretation of how supply and demand interact to find an equilibrium price/quantity point. A typical Econ 101 textbook says: higher demand --> higher price higher price --> higher supply higher supply --> lower price lower price --> higher demand In Econ 101 we pretend that cycle eventually reaches an equilibrium. In grad school we analyze the dynamics. But even if we believed your demand cau…

Yes and if you've actually paid attention you will see that a high salary signals other workers to flood the market with a lower salary thus starting the race to the bottom :) Programmers have largely driven themselves to zero, just take a look at the workers on freelance websites and how much difficulty a native English speaking freelancer has against an army of commoditized labor. There are still six digit earning…

Another nonsense implication of your theory: it's better to have a low wage, so you stay under the radar and don't signal other workers to enter the market.

Re: AI Beats Four Top Poker Players

#228

Post by 6nf »

xapata wrote: post #226
6nf wrote: post #224

Earlier quoted context omitted.

> you might ask, "Check it down?" In a heads up game? Really? "Check it down?" ???

The example applies more to a ring game, where you might be going after a particular fish and don't want to get involved with a better player. Still, I think it illustrates the principle of trying to keep the variance down. Also, if you're first to act on the next round, it might be worth asking, even if you don't think they'll agree.

That's collusion and it's not cool. No top pro would ever seriously ask to 'check it down'

Kelly's criterion and the sharpe ratio is not relevant for the format played in these human-AI heads up games. They play with an unlimited bankroll.

Re: AI Beats Four Top Poker Players

#229

Post by xapata »

6nf wrote: post #228
xapata wrote: post #226

Earlier quoted context omitted.

The example applies more to a ring game, where you might be going after a particular fish and don't want to get involved with a better player. Still, I think it illustrates the principle of trying to keep the variance down. Also, if you're first to act on the next round, it might be worth asking, even if you don't think they'll agree.

That's collusion and it's not cool. No top pro would ever seriously ask to 'check it down' Kelly's criterion and the sharpe ratio is not relevant for the format played in these human-AI heads up games. They play with an unlimited bankroll.

I'm not talking about a tournament. And yes, it could indicate collusion. But most of the time, it's just a friendly low- or mid-stakes game and the players are tired of thinking.

My point exactly about the unlimited bankroll. The experimenters may not have realized that an unlimited bankroll would significantly affect the strategy.

Re: AI Beats Four Top Poker Players

#230

Post by tripzilch »

__s wrote: post #140
xapata wrote: post #117

Earlier quoted context omitted.

> Poker is solved using a very large game tree You mean Libratus' strategy used a very large game tree. That is not the only strategy. Take a look at research from the University of Alberta [0]. Also, I'm not certain Libratus' strategy can be simplified to "very large game tree" as I haven't seen the paper, yet. While finding a Nash equilibrium means no other player can beat you, it doesn't mean you're going to make…

Key example of that Nash point is Rock Paper Scissors. You can't exploit anybody playing 100% random, yet that's the Nash Equilibrium. RPSAI competitions tend to have 100% random players perform quite poorly

> RPSAI competitions tend to have 100% random players perform quite poorly

Not as much "quite poorly" but more specifically, they will provably land at exactly the median position in the ranking (let's assume there's only one pure random bot in the tournament, no reason to have more than one, but the argument also works with multiple).

While it's impossible to win more than 50% of the time from a pure random RPS bot, it's also impossible to lose more than 50% of the time.

So all the other AIs will on average score exactly 50% against the random RPS bot. Whether they end up in the final ranking above or below this median line depends on how well they do against each other.

Post reply on HN