Live data from Hacker News

How A.I. Conquered Poker

nytimes.com

171–180 of 188 posts

Re: How A.I. Conquered Poker

#171

I think there's room for a "battlebots meets online poker" product. Imagine buying your bot a real money budget and setting it loose in a tournament to combat other bots.

There's a "Pokerbots" class/competition at MIT (during January aka Independent Activities Period), along with a robotics competition and a real time strategy video game bot competition. I did both the robotics and poker ones, which were a ton of fun!

Re: How A.I. Conquered Poker

#172
post #49

Earlier quoted context omitted.

The article was fascinating but honestly read like an ad for PioSolver

The journalist who contacted me told me he did so because the software keeps coming up when he talked to pro players. While it's certainly not one to advance the science the most (talk to Noam Brown if you want that), not the fastest (talk to Oskari Tammelin about that), it's still very popular and the first to get big following. It changed the game and got into the online poker culture. I am quite proud of that and…

[deleted]

Re: How A.I. Conquered Poker

#173

Earlier quoted context omitted.

> If you play optimal strategy, you will win against any opponent except one that plays optimal as well How does the example you're responding to not win 50% of the time? Rock v Rock = Tie Rock v Paper = Loss Rock v Scissors = Win The optimal game theoretic play of randomly choosing rock paper scissors is inferior play against this particular opponent. All that game theoretic perfect play gets you is the benefit of g…

Poker isn't like this. Think rock, pair, scissors, crap where rules are that crap always loses. Now uniform strategy between rps will yield profit vs an opponent who plays crap sometimes. It's very easy to play crap in poker

Right, but so what?

If the optimal strategy is a 50-50 split against someone else playing the optimal strategy and at least a 50% win against someone who is not, it doesn't follow that if they other player plays crap every game that the optimal strategy is best against them.

Identification of the crap player means that you increase your bets vs what you would normally bet for perfect play. Perfect play is bullet proof, but it isn't necessarily maximal yield against a non-perfect player even if thats the rough trendline.

Re: How A.I. Conquered Poker

#174

Earlier quoted context omitted.

I think this is purely a resource issue, e.g. if Google Brain decided to make an MtG bot I would be fairly confident it would be superhuman. Even real time strategy games like Starcraft are looking like they're on the cusp of superhuman bots (Alphastar was competitive as Protoss against elite players, but did not consistently beat them).

I doubt very highly it would be able to sit at a game of commander. Four players with 100 card singleton decks would be an absolutely enormous space to operate in.

Many players sounds like it's still a hard nut to crack for AI approaches (although as poker demonstrates its getting easier), but the deck size doesn't sound like it'd be the main issue.

Re: How A.I. Conquered Poker

#175

Earlier quoted context omitted.

> Alphastar also didn't play with the same limitations that a human has. Even after removing its ability to see the entire map and finally forcing it to scroll around, alphastar never misclicks (so its APM==EPM) and can still blast nearly unlimited APM for short bursts as long as its "average APM" over an x-second period matched human's APM. > I believe Alphastar would generate more interesting strategies if we limit…

Why is it far worse with Terran and Zerg?

I don't know enough to answer "what mechanisms of how the AI works would cause it to be worse than Terran and Zerg."

If the question is rather "what characteristics of Alphastar's Terran and Zerg play style make me say that its Terran and Zerg play is worse than its Protoss play," the simplest answer is that Alphastar just feels a lot more like a bot. Unlike when playing Protoss, it seems to get into certain "ruts" of unit composition and tactics that are a bad match for the opponent it's facing and can't seem to reactively change based on the game is going, whereas with Protoss it seems more than happy to change its play style over the course of the game based on what the opponent is doing.

Re: How A.I. Conquered Poker

#176

Earlier quoted context omitted.

> Alphastar also didn't play with the same limitations that a human has. Even after removing its ability to see the entire map and finally forcing it to scroll around, alphastar never misclicks (so its APM==EPM) and can still blast nearly unlimited APM for short bursts as long as its "average APM" over an x-second period matched human's APM. > I believe Alphastar would generate more interesting strategies if we limit…

Edit 2: Reading through the "supplementary data" of the 2019 paper, it definitely appears that the AlphaStar which reached grandmaster was not limited in the same ways as the 2017 paper would suggest. x/y positions of units are not determined visually, but fed directly from the API. So AlphaStar absolutely can just run Attack(Position: carrier_of_interest->pos.x) and not mis-click. It's "map" / "vision" is really jus…

I unfortunately don't have the time to look at the papers in detail (I could totally see how a lot of what I observed could happen even without intentional misclicks so I do take that back), but I want to point out that January 2019 Alphastar (in exhibition matches against TLO and MaNa) is significantly worse than Fall 2019 Alphastar. Alphastar changed very markedly between those time periods.

If you look at an Alphastar Protoss game from the latter half of 2019, it's not relying on cheap tricks to win (such as the impossible stalker micro). Nothing it's doing leaps out as superhuman. Instead it just grinds down its opponent through a superior sense of timing and macro strategy. The two games I linked against Serral it wins by punishing when Serral overextends his reach or by altering its unit composition to better fit what Serral throws at it, rather than some ungodly micro. Nothing it's doing there couldn't be done by a human. In fact I would say in most of the battles, Serral's micro was better than Alphastar's.

Now it's also worth pointing out that Serral is playing on an unfamiliar computer, rather than his own, so there's a bit of a handicap going on and even Alphastar Protoss will still lose to humans, so it's not superhuman, but it's definitely an elite player and its play style is very difficult to distinguish from that of an elite player.

Re: How A.I. Conquered Poker

#177

I think there's room for a "battlebots meets online poker" product. Imagine buying your bot a real money budget and setting it loose in a tournament to combat other bots.

But hasn't the problem been solved? Two bots would play perfectly and walk away with roughly 50% of the pot. This article is about how humans have been memorizing the variations - akin to chess openings - in order to play as perfectly as possible.

No, because you have the randomness of the deck, and you have bluffing. Bots could be tuned to be aggressive or defensive, or switch strategies depending on conditions in the game.

Re: How A.I. Conquered Poker

#178

Earlier quoted context omitted.

The distinction is that semi-bluffs are for when there are still cards to be dealt.

Would we call that thin value betting on the river?

I don't really think of thin value betting and semi-bluffing as the same concept on different betting rounds. Thin value betting gets its positive expectation (versus checking) entirely from getting called with a worse hand. It doesn't really have positive expectation from inducing a fold the way that semi-bluffing does.

At least, that holds if you assume the opponent will always call with hands above a threshold and fold with hands below it. Which is correct play (though kindly ignore raises, for simplicity - but they don't blow up the argument). Only if they behave very strangely, e.g. folding with the very best hands and worst hands but calling with the decent ones, can I see a better connection with semi-bluffing on early streets.

Re: How A.I. Conquered Poker

#179
post #133
post #52

(Former pro and high stakes player, occasional solver developer) This is actually one of the best poker articles I've ever seen in generalist media. Not too clickbaity, reasonable high level overview of game theory, a (very accurate IMO) quote from old pro Erik Seidel about the state of the game just 15 years ago, a discussion on variance vs. EV and results, and most importantly, an emphasis on math, randomization te…

Good to see poker theory approaching the way I naturally thought when I played on PokerStars (before Chuck Schumer and co shut it all down for us). People used to make fun of me for playing limits and sucking out on people. And no limit I would have some crazy strategies to fake out a table and make them anger-bet into my connected straight flush draws, when I was “representing” a two pair etc. On large tables, often…

> People said I was playing wrong but I intuitively felt like this would give a lot higher payoffs than just straight play with no bluffs.

No winning players nowadays would advocate for a strategy that doesn't involve bluffing.

Not to be harsh, but just to point it out for any novices who might be seduced by the simplicity of what you propose: the "strategy" you outline is simply nonsense and would not be winning in the long run even vs a fairly weak field. Maybe vs the level of play in the pre-solver world it did okay (though I doubt it would win except vs the very weakest fields), but today it would simply be burning money, even at microstakes.

Some of the heuristics you describe work in some situations, but you make no mention of accounting for other players positions, other players ranges, or the cards on the board; it is simply based on your hand and your perception of the player populations tendencies (which have changed dramatically since the pre-solver days). Ignoring the majority of the publicly available information in a given hand, in this game of incomplete information, is a grave strategic mistake.

Re: How A.I. Conquered Poker

#180

Earlier quoted context omitted.

Would we call that thin value betting on the river?

I don't really think of thin value betting and semi-bluffing as the same concept on different betting rounds. Thin value betting gets its positive expectation (versus checking) entirely from getting called with a worse hand. It doesn't really have positive expectation from inducing a fold the way that semi-bluffing does. At least, that holds if you assume the opponent will always call with hands above a threshold and…

I should also say my explanation assumes we're in last betting position here. If we're first to act, the logic is somewhat more complicated than I can successfully conjure up here without going back into the books - it's been many years since I played seriously.
Post reply on HN