Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How A.I. Conquered Poker

nytimes.com

Post reply on HN
Print view View original

91–100 of 188 posts

Re: How A.I. Conquered Poker

#91

Post by bloodyplonker22 »

splonk wrote: post #52

(Former pro and high stakes player, occasional solver developer) This is actually one of the best poker articles I've ever seen in generalist media. Not too clickbaity, reasonable high level overview of game theory, a (very accurate IMO) quote from old pro Erik Seidel about the state of the game just 15 years ago, a discussion on variance vs. EV and results, and most importantly, an emphasis on math, randomization te…

> Probably the one biggest misconception people have is that pros have sick reading abilities since TV likes to emphasize staredowns, when the actual single biggest skill long term pros have is the ability to lose hand after hand for hours and still play their best game.

This is absolutely true, but let me add to this: live pros like Phil Ivey, who have played the game in person virtually their entire lives, do have sick reading ability. However, it mostly works against amateurs. Actual pros know how to hide their tells better. If you take an online pro who has been clicking at the computer all day on multiple tables, that pro will not have anywhere close to the reading ability of Phil Ivey.

Re: How A.I. Conquered Poker

#92

Post by dwohnitmok »

jdmoreira wrote: post #55
dwohnitmok wrote: post #51

Earlier quoted context omitted.

I think this is purely a resource issue, e.g. if Google Brain decided to make an MtG bot I would be fairly confident it would be superhuman. Even real time strategy games like Starcraft are looking like they're on the cusp of superhuman bots (Alphastar was competitive as Protoss against elite players, but did not consistently beat them).

The search tree is huge in mtg. It has to be the largest of any game. You can take actions all the time. There are triggers all the time, you can stack your actions on top of your opponent actions. Huge space really. And then of course it's also imperfect information both in the sense of your opponent hand but also his deck. The cardpool is also very large for some formats. I actually don't think it's solvable just b…

Search space size is no longer a great heuristic of how difficult a game is for the latest in AI approaches. For example, an RTS game has an absolutely enormous search space as well (effectively every unit of several hundred can move in every direction for every single tick of the game clock, many units have spells and many spells are meant to stack with other spells) and Alphastar is a convincing demonstration that this is not out of the reach of current AIs. And you similarly have imperfect information where you don't know what your opponent is doing unless they are sufficiently close to your current units.

Even the meta-game/deck building aspect doesn't seem all that insurmountable as it doesn't seem fundamentally different from say a build order other than that it cannot change dynamically on the fly.

Re: How A.I. Conquered Poker

#93

Post by vincentmarle »

splonk wrote: post #52

(Former pro and high stakes player, occasional solver developer) This is actually one of the best poker articles I've ever seen in generalist media. Not too clickbaity, reasonable high level overview of game theory, a (very accurate IMO) quote from old pro Erik Seidel about the state of the game just 15 years ago, a discussion on variance vs. EV and results, and most importantly, an emphasis on math, randomization te…

> The natural portion of your hands to bluff with is the absolute worst ones - you don't want to bluff with your middling hands because you have some small chance of just winning a showdown when it checks around.

I don't understand this logic, can you elaborate a bit more? What do you suggest to do with middling hands then?

Re: How A.I. Conquered Poker

#94

Post by speedyapoc »

soferio wrote: post #39

What teaching or training tools are out there for a very average player at no limit Texas hold’em who just wants to get at bit better to a respectable level at a modest time commitment , and does not need to be a pro-level player?

Training sites are probably going to be your best bet to improve quickly. I see Run it Once was mentioned. I've personally used Upswing Poker in the past and they had some great resources. Bart Hanson's Crush Live Poker videos are great too but I can't vouch for the training course.

Re: How A.I. Conquered Poker

#95

Post by [deleted] »

vincentmarle wrote: post #93
splonk wrote: post #52

(Former pro and high stakes player, occasional solver developer) This is actually one of the best poker articles I've ever seen in generalist media. Not too clickbaity, reasonable high level overview of game theory, a (very accurate IMO) quote from old pro Erik Seidel about the state of the game just 15 years ago, a discussion on variance vs. EV and results, and most importantly, an emphasis on math, randomization te…

> The natural portion of your hands to bluff with is the absolute worst ones - you don't want to bluff with your middling hands because you have some small chance of just winning a showdown when it checks around. I don't understand this logic, can you elaborate a bit more? What do you suggest to do with middling hands then?

[deleted]

Re: How A.I. Conquered Poker

#96

Post by jliptzin »

vincentmarle wrote: post #93
splonk wrote: post #52

(Former pro and high stakes player, occasional solver developer) This is actually one of the best poker articles I've ever seen in generalist media. Not too clickbaity, reasonable high level overview of game theory, a (very accurate IMO) quote from old pro Erik Seidel about the state of the game just 15 years ago, a discussion on variance vs. EV and results, and most importantly, an emphasis on math, randomization te…

> The natural portion of your hands to bluff with is the absolute worst ones - you don't want to bluff with your middling hands because you have some small chance of just winning a showdown when it checks around. I don't understand this logic, can you elaborate a bit more? What do you suggest to do with middling hands then?

I think he’s saying you don’t want to “waste” a bluff on a hand you might end up winning anyway just by checking it down.

Re: How A.I. Conquered Poker

#97

Post by tziki »

vincentmarle wrote: post #93
splonk wrote: post #52

(Former pro and high stakes player, occasional solver developer) This is actually one of the best poker articles I've ever seen in generalist media. Not too clickbaity, reasonable high level overview of game theory, a (very accurate IMO) quote from old pro Erik Seidel about the state of the game just 15 years ago, a discussion on variance vs. EV and results, and most importantly, an emphasis on math, randomization te…

> The natural portion of your hands to bluff with is the absolute worst ones - you don't want to bluff with your middling hands because you have some small chance of just winning a showdown when it checks around. I don't understand this logic, can you elaborate a bit more? What do you suggest to do with middling hands then?

> What do you suggest to do with middling hands then?

You try to get to a showdown. With middling hands there's a chance the opponent has a worse hand. It's when your hand is so bad your opponent has you almost certainly beat that you get to the bluffing territory, as that's the only way for you to win.

Re: How A.I. Conquered Poker

#98

Post by tziki »

jstx1 wrote: post #27

But it hasn't conquered it! I kept searching the article for some new recent breakthrough that I've missed but it's not there. Yes, solvers like Pio have been around for years and limit holdem has been essentially solved for a while but nobody plays limit holdem anyway. The two most popular games (no-limit Texas holdem and pot-limit Omaha) are still unsolved.

No limit holdem has been essentially solved. Pluribus & co not withstanding, you just haven't heard of it because the people who have solved it are busy printing money in online poker (yes, I know they try to detect bots, and no, they can't detect them all). With stakes this high, academic progress lags the 'actual' state-of-the-art by years.

Re: How A.I. Conquered Poker

#99

Post by splonk »

vincentmarle wrote: post #93
splonk wrote: post #52

(Former pro and high stakes player, occasional solver developer) This is actually one of the best poker articles I've ever seen in generalist media. Not too clickbaity, reasonable high level overview of game theory, a (very accurate IMO) quote from old pro Erik Seidel about the state of the game just 15 years ago, a discussion on variance vs. EV and results, and most importantly, an emphasis on math, randomization te…

> The natural portion of your hands to bluff with is the absolute worst ones - you don't want to bluff with your middling hands because you have some small chance of just winning a showdown when it checks around. I don't understand this logic, can you elaborate a bit more? What do you suggest to do with middling hands then?

One very simplified model is that on the river, all your hands fall into certain buckets, from strongest to weakest. (I'm ignoring a ton of nuance here and there are actually many more buckets in a real game)

- Worth betting, because you have the best hand

- Worth calling, because your hand is good enough to beat a bluff (and maybe some value bets as well).

- Intending to fold, because your hand is bad, but maybe it's good enough to win if the opponent's hand is worse and they don't bluff.

- Worth bluffing, because your hand is so bad it can't win otherwise.

The middling hands would literally be the middle two buckets in this example. Call with the better ones, fold with the worse ones. (To complicate this more, in a real world situation the worst part of the "intending to fold" bucket might become a "planning to bluff-raise" bucket, due to similar logic as to why you bluff with your worst hands.)

Re: How A.I. Conquered Poker

#100

Post by runnerup »

dwohnitmok wrote: post #51
isaacg wrote: post #20

Earlier quoted context omitted.

As far as I know, in Magic: the Gathering, the best bots are far worse than most players. Part of the difficulty is that the rules are so complicated that there are only a couple of complete rules implementations. Beyond that, it's an imperfect information game with far more actions per game than poker, so optimal-solver techniques haven't seen success.

I think this is purely a resource issue, e.g. if Google Brain decided to make an MtG bot I would be fairly confident it would be superhuman. Even real time strategy games like Starcraft are looking like they're on the cusp of superhuman bots (Alphastar was competitive as Protoss against elite players, but did not consistently beat them).

Alphastar also didn't play with the same limitations that a human has. Even after removing its ability to see the entire map and finally forcing it to scroll around, alphastar never misclicks (so its APM==EPM) and can still blast nearly unlimited APM for short bursts as long as its "average APM" over an x-second period matched human's APM.

I believe Alphastar would generate more interesting strategies if we limited alphastar to a bit below human APM and forced it to emulate USB K+M to click (instead of using an API, which it currently does) and adding a progressively increasing random fuzzing layer against its inputs so that as it clicks faster the precision/accuracy goes down.

By "interesting strategies" I mean strategies that humans could learn to adopt. Currently its main strategy is "perfectly juggle stalkers" which is a neat party trick, but that particular strategy is about as interesting to me as 2011-era SC AI[0]. Obviously how it arrived at that strategy is quite interesting, but the style of play is not relevant to humans, and may in fact even get beaten by hardcoded AI's.

I'm also very curious what Alphastar could come up with if it were truly unsupervised learning. AIUI, the first many rounds of training were supervised based on high level human replays -- so it would have gotten stuck in a local minima near what has already been invented by humans.

This may be relevant if Microsoft reboots Blizzard's IP. I would love to have an alphastar in SC3 to play against off-line, or have as a teammate, archon mode, etc. I think all RTS' are kind of "archon mode with AI teammate" already. The AI currently handles unit pathing, selection of units to attack, etc. With an alphastar powering the internal AI instead, more tactics/micro can be offloaded to AI and allow humans to focus more on strategy. That seems like it would be super cool.

Examples: "Here AI, I made two drop ships of marines. Take these to the main base and find an optimal place to drop them. If you encounter strong resistance or lots of static defense, just leave and come back home"

"Here AI, use these two drop ships of marines to distract while I use the main army to push the left flank. Take them into the main, natural, or 4th base -- goal is to keep them alive for as long as possible. Focus on critical infrastructure/workers where possible but mostly just keep them alive and moving around to distract the opponent."

0: Automaton 2000 AI perfectly controls 50-supply zerglings (2.5k mineral) vs. 60-supply (3k mineral, 2.5k gas) siege tanks: https://www.youtube.com/watch?v=IKVFZ28ybQs

Post reply on HN