Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

DeepStack: Expert-Level Artificial Intelligence in No-Limit Poker

arxiv.org

Post reply on HN
Print view View original

21–30 of 50 posts

Re: DeepStack: Expert-Level Artificial Intelligence in No-Limit Poker

#21

Post by jdmichal »

geofft wrote: post #19
philosopheer wrote: post #9

most people (including here on HN) are complete n00bs when it comes to understanding how poker is played and how computers can play it, so just to straighten y'all out at the git-go here: computers are better at bluffing and randomness than humans are. Bluffing is an important optimizing strategy in playing poker well, and it entails tracking the expected value of a pot (which includes cost expectations, don't forget…

As an actual complete n00b, in online poker, how are tells communicated when you can't see someone's facial expressions or body language? My guess is the dollar value of bets, and the timing side channel?

Bet amount certainly plays a big part. This is what philosopheer means when he talks about "the mathematics of the game". Bet amount is part of that mathematical part; it's basically a signal of your confidence against the current size of the pot. This is why, when you do reading, many strategies will talk about bet amounts as multipliers of the current pot.

Timing can be informative, but it's actually weaker online than in person. In person, you know whether the person is physically present, and can generally gauge when they're paying attention also. Online, taking a long time could simply mean that they're not paying attention. (I've watched streams on Twitch of pro players working multiple tables online.)

Re: DeepStack: Expert-Level Artificial Intelligence in No-Limit Poker

#22

Post by ska »

falcolas wrote: post #3
lawn wrote: post #2

I think I can wrap my head around neural nets being superior at games with perfect information like chess or go. But how would you teach bluffing and randomness to a neural net?

I imagine it's mostly just playing the percentages. Bet when it has a high percentage of winning, fold when it doesn't. It doesn't need to read its opponents if it can play the percentages perfectly.

No, this is a terrible idea. If you play like this consistently, you are basically telling your opponents when to play against you and when to get out of the way. They can even pick bet sizes (assuming no limit) to refine your possible hands very accurately.

Re: DeepStack: Expert-Level Artificial Intelligence in No-Limit Poker

#23

Post by jdmichal »

jspiral wrote: post #12
lawn wrote: post #2

I think I can wrap my head around neural nets being superior at games with perfect information like chess or go. But how would you teach bluffing and randomness to a neural net?

At higher levels poker is about game theory, for example, the player bluffs at an optimal frequency in a certain situation so as to be indifferent to whether the opponent calls or folds. Exploitative strategies, based on understanding opponent weaknesses and tendencies will win $ at a higher rate, but are themselves exploitable. For example, almost never bluffing and playing only strong cards crushes beginners who pl…

> This strategy is easily beaten though by stealing most pots and then not paying off the infrequent big bets (strong hands don't come often enough).

To dig deeper:

Or you can try actively punishing the big hands by folding out early. Of course, that strategy opens you up to being bled by your opponent bluffing strong hands. Attempting to actively punish the big hands here is a deviation. This is what jspiral means by "deviations from perfection".

Re: DeepStack: Expert-Level Artificial Intelligence in No-Limit Poker

#24

Post by jdmichal »

ska wrote: post #22
falcolas wrote: post #3

Earlier quoted context omitted.

I imagine it's mostly just playing the percentages. Bet when it has a high percentage of winning, fold when it doesn't. It doesn't need to read its opponents if it can play the percentages perfectly.

No, this is a terrible idea. If you play like this consistently, you are basically telling your opponents when to play against you and when to get out of the way. They can even pick bet sizes (assuming no limit) to refine your possible hands very accurately.

I'm usually surprised in tournaments by the number of people willing to play with me after I've sat quietly folding for the first three rounds... By all means, everyone should fold and give me the blinds, but there always seems to be a player or two who bite!

Re: DeepStack: Expert-Level Artificial Intelligence in No-Limit Poker

#25

Post by jdmichal »

reverend_gonzo wrote: post #11
falcolas wrote: post #3

Earlier quoted context omitted.

I imagine it's mostly just playing the percentages. Bet when it has a high percentage of winning, fold when it doesn't. It doesn't need to read its opponents if it can play the percentages perfectly.

This is true for Limit Hold'em, but very much not true for No Limit. Limit Hold'em is a solved game, because as long as you're playing the odds, you can play perfectly. No Limit changes things because the bets can vary wildly. If you play a tight game (just play the odds), and opponent will get out whenever you're in, and will bluff just to see if you call or fold. Bluffing is a major component in No Limit, and there…

I fell to this when I randomly decided to play some limit hold'em one night. Kept losing to a guy that chased every chance at odds he could, because I couldn't make bets big enough to scare him out. Lesson learned!

Re: DeepStack: Expert-Level Artificial Intelligence in No-Limit Poker

#26

Post by tgb »

bcassedy wrote: post #17
lawn wrote: post #2

I think I can wrap my head around neural nets being superior at games with perfect information like chess or go. But how would you teach bluffing and randomness to a neural net?

Former poker pro here. Top professionals are building their strategy around game theory. They'll attempt to play in such a way that they aren't exploitable and look to deviate when they've spotted a weakness in their opponent's play. Basically, the game theory optimal strategy is unexploitable. In every situation, the best you can do is break even by also playing the optimal strategy. If you deviate from optimal stra…

Is it known that there is an optimal strategy?

Re: DeepStack: Expert-Level Artificial Intelligence in No-Limit Poker

#27

Post by osti »

tgb wrote: post #26
bcassedy wrote: post #17

Earlier quoted context omitted.

Former poker pro here. Top professionals are building their strategy around game theory. They'll attempt to play in such a way that they aren't exploitable and look to deviate when they've spotted a weakness in their opponent's play. Basically, the game theory optimal strategy is unexploitable. In every situation, the best you can do is break even by also playing the optimal strategy. If you deviate from optimal stra…

Is it known that there is an optimal strategy?

In heads up ie. 1 on 1, there is Nash equilibrium, but in multiplayer game there isn't because the other players can coolude against you.

Re: DeepStack: Expert-Level Artificial Intelligence in No-Limit Poker

#28

Post by bcassedy »

tgb wrote: post #26
bcassedy wrote: post #17

Earlier quoted context omitted.

Former poker pro here. Top professionals are building their strategy around game theory. They'll attempt to play in such a way that they aren't exploitable and look to deviate when they've spotted a weakness in their opponent's play. Basically, the game theory optimal strategy is unexploitable. In every situation, the best you can do is break even by also playing the optimal strategy. If you deviate from optimal stra…

Is it known that there is an optimal strategy?

We know that there exists an optimal strategy, but that we still aren't close to achieving it. It's a zero sum game where both parties have the same lack of information and the betting order rotates. I think it has to have an optimal strategy.

Re: DeepStack: Expert-Level Artificial Intelligence in No-Limit Poker

#29

Post by zitterbewegung »

MikeTV wrote: post #14

> DeepStack becomes the first computer program to beat professional poker players in heads-up no-limit Texas hold'em Whether any others have been made before now is anyone's guess. Botting is a known problem in online poker. If there's a golden goose out there, I'm sure it's being kept under wraps.

You can collude multiple bots or perform other tasks which could make the botting problem in Texas Holdem not equivalent to the same achievement that they present in the paper.

Re: DeepStack: Expert-Level Artificial Intelligence in No-Limit Poker

#30

Post by jat850 »

IgorPartola wrote: post #18
bcassedy wrote: post #17

Earlier quoted context omitted.

Former poker pro here. Top professionals are building their strategy around game theory. They'll attempt to play in such a way that they aren't exploitable and look to deviate when they've spotted a weakness in their opponent's play. Basically, the game theory optimal strategy is unexploitable. In every situation, the best you can do is break even by also playing the optimal strategy. If you deviate from optimal stra…

Since you are here, I have a few questions about your former job. First, how does one become a pro poker player? Second, does it work like a sport where you get paid from sponsorships, or do you just directly take home what you win? Or a combination of both? Third, is this something that you can do part-time, or does it require full time attention? Fourth, why did you quit?

Not who you're replying to, but I can give a few answers, having also played professionally for some time.

1. Lots of means this can happen. "Professional poker player" is generally taken to mean "derives primary source of income from playing poker" or sometimes "spends the majority of their time playing poker". There's no exact qualification.

2. Not generally sponsorship driven, although there are some modes of sponsorship that do factor in in some ways - the primary means of income is basically by winning money from other players. A secondary component is often rakeback, in online play. In cash games, you can join and leave at any time and your winnings or losses are simply the amount you are up or down in that particular session. In tourmanent play there is a payout structure based on your placement in the tournament (often something like 25% of the total prize pool to 1st, 15% to second, etc.)

3. It can be done part time. There is nothing to say that your bankroll can't be seeded or supplemented by external means, and you can play cash games for short or long periods. Tournaments are typically long (at least if you remain in them a long time, though you can be eliminated at any time, basically). I used to play approximately 40 hours a week (since I was treating it like a job), but now I play 10-15 and it represents about 30-40% of my yearly income.

4. I personally quit because I found the stress associated with having it as the sole means of providing myself too overwhelming. One can have a stretch of negative earnings that can last hours, days, weeks, even months - and it can be psychologically damaging in some ways. I also found that I preferred to keep it as a hobby than a means of income, since I enjoyed it more that way.

Post reply on HN