Live data from Hacker News

DeepStack: Expert-Level Artificial Intelligence in No-Limit Poker

arxiv.org

11–20 of 50 posts

Re: DeepStack: Expert-Level Artificial Intelligence in No-Limit Poker

#11
post #3
post #2

I think I can wrap my head around neural nets being superior at games with perfect information like chess or go. But how would you teach bluffing and randomness to a neural net?

I imagine it's mostly just playing the percentages. Bet when it has a high percentage of winning, fold when it doesn't. It doesn't need to read its opponents if it can play the percentages perfectly.

This is true for Limit Hold'em, but very much not true for No Limit. Limit Hold'em is a solved game, because as long as you're playing the odds, you can play perfectly. No Limit changes things because the bets can vary wildly. If you play a tight game (just play the odds), and opponent will get out whenever you're in, and will bluff just to see if you call or fold.

Bluffing is a major component in No Limit, and there are very different profitable playing strategies.

Re: DeepStack: Expert-Level Artificial Intelligence in No-Limit Poker

#12
post #2

I think I can wrap my head around neural nets being superior at games with perfect information like chess or go. But how would you teach bluffing and randomness to a neural net?

At higher levels poker is about game theory, for example, the player bluffs at an optimal frequency in a certain situation so as to be indifferent to whether the opponent calls or folds.

Exploitative strategies, based on understanding opponent weaknesses and tendencies will win $ at a higher rate, but are themselves exploitable.

For example, almost never bluffing and playing only strong cards crushes beginners who play too many hands and call too much.

This strategy is easily beaten though by stealing most pots and then not paying off the infrequent big bets (strong hands don't come often enough).

A "perfect" game theory strategy is like armor, slowly bleeding the opponent every time they deviate from perfection themselves.

not sure if that helps but maybe some seeds to google at least

Re: DeepStack: Expert-Level Artificial Intelligence in No-Limit Poker

#13
To be fair, none of the so called pros are considered big names in today's no limit heads-up games. They should probably challenge ppl like WCGRider, Jungleman etc. next.

On another point, CMU just can't seem to catch a break, their thunder continuously being stolen by UofAlberta in poker research, first in limit, now no limit. UofA clearly tried to publish this before the CMU poker challenge that's supposed to begin soon.

To read more about the CMU challenge http://www.cmu.edu/news/stories/archives/2017/january/poker-...

Re: DeepStack: Expert-Level Artificial Intelligence in No-Limit Poker

#14
> DeepStack becomes the first computer program to beat professional poker players in heads-up no-limit Texas hold'em

Whether any others have been made before now is anyone's guess. Botting is a known problem in online poker. If there's a golden goose out there, I'm sure it's being kept under wraps.

Re: DeepStack: Expert-Level Artificial Intelligence in No-Limit Poker

#15

most people (including here on HN) are complete n00bs when it comes to understanding how poker is played and how computers can play it, so just to straighten y'all out at the git-go here: computers are better at bluffing and randomness than humans are. Bluffing is an important optimizing strategy in playing poker well, and it entails tracking the expected value of a pot (which includes cost expectations, don't forget…

Does this imply that a pro may well do better in a multiplayer game with mixed humans and machines (by using "tells" to build up a bigger stack from the humans' inaccuracies), than in heads up against a machine?

Re: DeepStack: Expert-Level Artificial Intelligence in No-Limit Poker

#16

most people (including here on HN) are complete n00bs when it comes to understanding how poker is played and how computers can play it, so just to straighten y'all out at the git-go here: computers are better at bluffing and randomness than humans are. Bluffing is an important optimizing strategy in playing poker well, and it entails tracking the expected value of a pot (which includes cost expectations, don't forget…

No - If the only thing computers couldn't beat humans at was reading tells, they'd win online poker.

But they don't yet do that: this paper is about beating humans at heads up, which is a much more limited domain than a full table.

If you want to learn about why to bluff I'd recommend reading about using game theory to solve Kuhn poker.

Re: DeepStack: Expert-Level Artificial Intelligence in No-Limit Poker

#17
post #2

I think I can wrap my head around neural nets being superior at games with perfect information like chess or go. But how would you teach bluffing and randomness to a neural net?

Former poker pro here.

Top professionals are building their strategy around game theory. They'll attempt to play in such a way that they aren't exploitable and look to deviate when they've spotted a weakness in their opponent's play.

Basically, the game theory optimal strategy is unexploitable. In every situation, the best you can do is break even by also playing the optimal strategy. If you deviate from optimal strategy, the optimal strategy will beat you, but it's possible that a strategy tailored to taking advantage of your specific deviations would beat you more quickly.

Unexploitable play typically means that you bet a size with a range of holdings that would make your opponent indifferent to all of his options (And the converse is true when facing a bet). For humans, this means that they gravitate to a few standard bet sizes, while a computer could, in theory, balance their range with much more granularity.

Last I read, for training the neural net it'll play billions+ hands against versions of itself designed to exploit various weaknesses. It'll start out by performing random actions, for example, say it'll have a 33% chance to call your bet, raise, or fold. It then starts to see that it does better when it raises your bet with the nuts and also as a bluff. Eventually, it arrives at an equilibrium strategy.

Since computers are much better at randomness than humans are, they're able to more effectively play these types of strategies and with more complexity of bet sizing. There is what's called a mixed strategy, a strategy where given a situation with the same hole cards you will call, raise, or fold to a bet with some non-zero probability. Doing that as a human is very difficult, but it's something computers manage to do quite easily.

Re: DeepStack: Expert-Level Artificial Intelligence in No-Limit Poker

#18
post #2

I think I can wrap my head around neural nets being superior at games with perfect information like chess or go. But how would you teach bluffing and randomness to a neural net?

Former poker pro here. Top professionals are building their strategy around game theory. They'll attempt to play in such a way that they aren't exploitable and look to deviate when they've spotted a weakness in their opponent's play. Basically, the game theory optimal strategy is unexploitable. In every situation, the best you can do is break even by also playing the optimal strategy. If you deviate from optimal stra…

Since you are here, I have a few questions about your former job.

First, how does one become a pro poker player?

Second, does it work like a sport where you get paid from sponsorships, or do you just directly take home what you win? Or a combination of both?

Third, is this something that you can do part-time, or does it require full time attention?

Fourth, why did you quit?

Re: DeepStack: Expert-Level Artificial Intelligence in No-Limit Poker

#19

most people (including here on HN) are complete n00bs when it comes to understanding how poker is played and how computers can play it, so just to straighten y'all out at the git-go here: computers are better at bluffing and randomness than humans are. Bluffing is an important optimizing strategy in playing poker well, and it entails tracking the expected value of a pot (which includes cost expectations, don't forget…

As an actual complete n00b, in online poker, how are tells communicated when you can't see someone's facial expressions or body language? My guess is the dollar value of bets, and the timing side channel?

Re: DeepStack: Expert-Level Artificial Intelligence in No-Limit Poker

#20
post #15

most people (including here on HN) are complete n00bs when it comes to understanding how poker is played and how computers can play it, so just to straighten y'all out at the git-go here: computers are better at bluffing and randomness than humans are. Bluffing is an important optimizing strategy in playing poker well, and it entails tracking the expected value of a pot (which includes cost expectations, don't forget…

Does this imply that a pro may well do better in a multiplayer game with mixed humans and machines (by using "tells" to build up a bigger stack from the humans' inaccuracies), than in heads up against a machine?

Players can and will target others as "easier" and selectively get in fights with them. If nothing else, you'll certainly avoid getting into fights with a player that continuously beats you.
Post reply on HN