Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How A.I. Conquered Poker

nytimes.com

Post reply on HN
Print view View original

151–160 of 188 posts

Re: How A.I. Conquered Poker

#151

Post by misja111 »

randomNumber7 wrote: post #148

There is some misconception about the optimal strategy in the article. You can only win in poker if you recognize how your opponent deviates from the optimal strategy and then play a strategy to exploit him. You have to play an exploitable strategy yourself to exploit someone else. Think about playing rock paper scissors agains an opponent who always chooses rock. If you still play the optimal strategy (choosing rand…

> You can only win in poker if you recognize how your opponent deviates from the optimal strategy and then play a strategy to exploit him.

This is not true. If you play optimal strategy, you will win against any opponent except one that plays optimal as well, in which case you'll break even. But, of course you'll win a lot more if you are able to adapt your strategy to exploit any weaknesses that you have detected.

And detecting weaknesses in your opponent(s) in live play is something that humans will remain better at than AI for quite a while. Because it requires not just careful analysis of your opponent's actions but some contextual information as well. E.g., how old is your opponent, is he experienced or not, is he drunk, have you experienced similar players like hime before, etc.

Re: How A.I. Conquered Poker

#152

Post by jessriedel »

noambrown wrote: post #80
hooande wrote: post #72

Earlier quoted context omitted.

Have to say, Pluribus beating top humans for 10,000 hands isn't the same thing as being super human. It's just too small of a sample to make that claim. Further, thousands of the hands that Pluribus played against the human pros are available online in an easy to parse format [0]. I've analyzed them. Pluribus has multiple obvious deficiencies in its play that I can describe in detail. It seems like it's very difficul…

Normally 10,000 hands would be too small a sample size but we used variance-reduction techniques to reduce the luck factor. Think things like all-in EV but much more powerful. It's described in the paper.

Where is the variance-reduction technique discussed? I looked at this paper

https://www.science.org/doi/abs/10.1126/science.aay2400

https://par.nsf.gov/servlets/purl/10077416

and it just says

> Finally, we tested Libratus against top humans. In January 2017, Libratus played against a team of four top HUNL specialist professionals in a 120,000 hand Brains vs. AI challenge match over 20 days. The participants were Jason Les, Dong Kim, Daniel McCauley, and Jimmy Chou. A prize pool of $200,000 was allocated to the four humans in aggregate. Each human was guaranteed $20,000 of that pool. The remaining $120,000 was divided among them based on how much better the human did against Libratus than the worstperforming of the four humans. Libratus decisively defeated the humans by a margin of 147 mbb/hand, with 99.98% statistical significance and a p-value of 0.0002 (if the hands are treated as independent and identically distributed), see Fig. 3 (57). It also beat each of the humans individually.

Re: How A.I. Conquered Poker

#153

Post by jdm386 »

Reflecticon wrote: post #59
jstx1 wrote: post #44

Earlier quoted context omitted.

Books (there's very few good ones) Training videos 1-on-1 coaching PioSolver (and other similar software) For any of these to stick, you need to spend some amount of time studying by yourself; just consuming learning material and playing isn't enough.

Could you recommend a couple of books for a start?

Modern Poker Theory as mentioned by the other reply is good. I'd also suggest Play Optimal Poker by Andrew Brokos. The former outlines a lot of strategy in the game and talks a little about game theory and its implications. The latter gives less direct poker advice but uses "toy games" that share some similarities with poker but are simplified to demonstrate some of the implications that game theory has on poker. It does contain some direct poker advice but it's more of a book to teach you how to think properly about how game theory applies to poker and how you can use that to study with something like a solver and actually understand what you're seeing. I would suggest you pick up both, I think that the Brokos book assumes you know certain poker terms and doesn't contain a glossary whereas modern poker theory is almost like a textbook and has a section where it defines all of the terms it uses. Modern poker theory will have more actionable advice but the Brokos book is excellent to teach you how to think about game theory and requires more self-reflection and has more questions to the reader.

Applications of no limit by Janda or mathematics of poker are recommended by a different reply here. I would caution that these are extremely academic and extremely dense texts that would be a very tough read for a newer player. Mathematics is less practical and more of a math book than a poker book and applications is Janda showing how to work out solver-like solutions before solvers existed and also contains a lot of math. I think the other two above are more practical and aren't going to lead you to put the book down a 10th of the way through.

Re: How A.I. Conquered Poker

#154

Post by ramoz »

Stasis5001 wrote: post #137
ramoz wrote: post #132

Earlier quoted context omitted.

Similar story here. I’ve always played, mostly online, before Black Friday (pretty sure I was a minor when it happened) but played all the offshore sites through college. Professional software career took off and bounced out, finally ~3y ago I decided to get back into it and I would only play challenging games (only thing challenging about a 1/3 or tourneys is the grind). So, started with 2/5, within a week was playi…

When you say tech industry pays more, I'm assuming the effective long-run hourly earnings of poker players is way less glamorous than it looks from an outsider seeing highlights? Doing a basic google search led me to https://pokerdb.thehendonmob.com/ranking/6737/ , which seems to suggest that 81 players earned over 1mm USD in 2021. But presumably those players don't earn that YoY, and even if they did, plenty more so…

The other commenters said it for tourneys. I can’t stand tournaments.

For cash games, I was only playing on the casino. For the 1/3 (1$/3$ blinds) the earnings rate seems absolutely abysmal and you have to account for rake. There are grinders who play these games, along with 2/5. My assumption is any grinder playing here is down under or not making enough to compete with a professional career in the tech industry.

Next is 2/5. Here I could’ve sustained a decent salary. A pro at these stakes showed me his past year earnings which was 180k.

It’s hard to tell with the 10/25, 25/50 pros. There were definitely dudes sitting on 500k+, maybe way more, chips that they kept in the casino and I assume similar amounts of cash at home. There were guys leaving with big nights but still losing 90k in a night. I’d hear about some being down under 200K at any point. There were probably some money launders. There was the occasional private investment fund guy that came in, was short with slick back hair, very loud mouth, would brag and show off one of his 17M personal banking account. Then there were the very quiet, calculated guys. It seemed that they were operating on some formula I never cared to figure out. They’d avoid big hands for the most part. Some form of grind I didn’t have the discipline for. I also didn’t have the discipline to handle large swings at those stakes, nor the bankroll, or desire to play lower stakes to build a bankroll.

Re: How A.I. Conquered Poker

#155

Post by alasdair_ »

jessriedel wrote: post #152
noambrown wrote: post #80

Earlier quoted context omitted.

Normally 10,000 hands would be too small a sample size but we used variance-reduction techniques to reduce the luck factor. Think things like all-in EV but much more powerful. It's described in the paper.

Where is the variance-reduction technique discussed? I looked at this paper https://www.science.org/doi/abs/10.1126/science.aay2400 https://par.nsf.gov/servlets/purl/10077416 and it just says > Finally, we tested Libratus against top humans. In January 2017, Libratus played against a team of four top HUNL specialist professionals in a 120,000 hand Brains vs. AI challenge match over 20 days. The participants were Jaso…

>The remaining $120,000 was divided among them based on how much better the human did against Libratus than the worstperforming of the four humans.

Surely the correct strategy here is for the human players to collude to give as much money as possible to a single player and then split the money afterwords, no?

Also, the fact that they players can only gain money without losing anything likely changes their play somewhat. By default I'd assume (and have generally observed) that most players on a freeroll (or better than a freeroll really) tend to undervalue their position and gamble more than is usually wise.

I'd definitely be interested in seeing a "real" game where the humans are betting their own money.

Re: How A.I. Conquered Poker

#156

Post by ogogmad »

jdmoreira wrote: post #55
dwohnitmok wrote: post #51

Earlier quoted context omitted.

I think this is purely a resource issue, e.g. if Google Brain decided to make an MtG bot I would be fairly confident it would be superhuman. Even real time strategy games like Starcraft are looking like they're on the cusp of superhuman bots (Alphastar was competitive as Protoss against elite players, but did not consistently beat them).

The search tree is huge in mtg. It has to be the largest of any game. You can take actions all the time. There are triggers all the time, you can stack your actions on top of your opponent actions. Huge space really. And then of course it's also imperfect information both in the sense of your opponent hand but also his deck. The cardpool is also very large for some formats. I actually don't think it's solvable just b…

MCTS is usually paired up with Deep Learning. This doesn't appear to have any problems with games with even larger branching factors. Look up AlphaZero and AlphaStar.

Re: How A.I. Conquered Poker

#157

Post by alasdair_ »

howdydoo wrote: post #115
bloodyplonker22 wrote: post #90

Earlier quoted context omitted.

yes, that is the obvious step that bot makers have taken. When bots were barely at the online poker scene, nobody cared to even check. Of course, there are still other ways to check for bots such as a user playing for an unreasonable amount of time or an extraordinary amount of tables, or simply not answering to chat.

The natural next step is adding ELIZA-like chat responses to your bots

GPT-3 would be interesting, especially once multiple bots start chatting to each other.

Re: How A.I. Conquered Poker

#158

Post by alasdair_ »

soferio wrote: post #39

What teaching or training tools are out there for a very average player at no limit Texas hold’em who just wants to get at bit better to a respectable level at a modest time commitment , and does not need to be a pro-level player?

If you like video content, the Daniel Negranu Masterclass is extremely well done. Phil Ivey also has content on that site but I found it much less comprehensive.

After that, perhaps watch some Doug Polk on Youtube to see how he uses the concept of putting people on ranges of hands and what his thinking is in a specific spot.

Re: How A.I. Conquered Poker

#159

Post by 3pt14159 »

misja111 wrote: post #151
randomNumber7 wrote: post #148

There is some misconception about the optimal strategy in the article. You can only win in poker if you recognize how your opponent deviates from the optimal strategy and then play a strategy to exploit him. You have to play an exploitable strategy yourself to exploit someone else. Think about playing rock paper scissors agains an opponent who always chooses rock. If you still play the optimal strategy (choosing rand…

> You can only win in poker if you recognize how your opponent deviates from the optimal strategy and then play a strategy to exploit him. This is not true. If you play optimal strategy, you will win against any opponent except one that plays optimal as well, in which case you'll break even. But, of course you'll win a lot more if you are able to adapt your strategy to exploit any weaknesses that you have detected. A…

> If you play optimal strategy, you will win against any opponent except one that plays optimal as well

How does the example you're responding to not win 50% of the time?

Rock v Rock = Tie

Rock v Paper = Loss

Rock v Scissors = Win

The optimal game theoretic play of randomly choosing rock paper scissors is inferior play against this particular opponent. All that game theoretic perfect play gets you is the benefit of getting at least a tie. Possibly more, but not always more for non-perfect play.

Re: How A.I. Conquered Poker

#160

Post by bluecalm »

misja111 wrote: post #151
randomNumber7 wrote: post #148

There is some misconception about the optimal strategy in the article. You can only win in poker if you recognize how your opponent deviates from the optimal strategy and then play a strategy to exploit him. You have to play an exploitable strategy yourself to exploit someone else. Think about playing rock paper scissors agains an opponent who always chooses rock. If you still play the optimal strategy (choosing rand…

> You can only win in poker if you recognize how your opponent deviates from the optimal strategy and then play a strategy to exploit him. This is not true. If you play optimal strategy, you will win against any opponent except one that plays optimal as well, in which case you'll break even. But, of course you'll win a lot more if you are able to adapt your strategy to exploit any weaknesses that you have detected. A…

This is almost correct but not quite. Optimal strategy wins against many other strategies and breaks even against others. The ones it breaks even against are not necessarily optimal themselves, they just don't make mistakes vs the optimal one but might be exploitable themselves.

To be more precise: you only need to replicate not mixed (pure) plays of the optimal strategy to not lose against it. Your frequencies for mixed actions can be completely off though.

Post reply on HN