Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Mastering Stratego

deepmind.com

Post reply on HN
Print view View original

91–100 of 128 posts

Re: Mastering Stratego

#91

Post by Zircom »

michaf wrote: post #90
astrange wrote: post #48

Earlier quoted context omitted.

It generalizes as well as GPT-3 does: --- If you took the word "Wikipedia", and threw all the letters up in the air, what are some possible things they'd spell once they fell on the ground again? If you took the word "Wikipedia" and threw all the letters up in the air, some possible things they could spell once they fell on the ground again might include "kawipieda," "keiapwidi," "kipedwaia," "pwakieida," or "eipawik…

Quite interesting that it will make subtle errors in its otherwise reasonable-looking answer, e.g. "kipedwaia" has two "a"s; "kawipieda", "kipedwaia" and "pwakieida" have only two "i"s. I have seen reports that it will happily hallucinate a plausible but wrong answer to all sorts of different prompts, intermixed with many mostly correct answers. It's interesting to think about how to place trust in such a system.

Peter Watts has a series called Rifters that explores this a little. "Smart gels" which are neural nets made up of a mishmash of cultured neruons and silicon that run most of society. They're trained just like neural nets today, and therefore their decision making process is basically a black box. They do a great job, but no one is really sure how they get there, but they work great and they're so much cheaper, so they who cares.

Anyhow spoiler alert, the neural nets running the virus response have been inadvertently trained to prefer simple systems over complex ones without anyone realizing, and decide that a planet with no life on it after being wiped out from the virus is infinitely more simple than the present one and starts helping it out instead of stopping it.

So short answer to your question is I would not place much if any trust and systems like that, in as far as anything that has high stakes, real world consequences.

Re: Mastering Stratego

#92

Post by roenxi »

throwthudec1 wrote: post #82
hgsgm wrote: post #59

Earlier quoted context omitted.

How many people study Stratego as much as an International Master studies chess?

I think there are probably very much diminishing returns. A small scene is probably pretty damn good at the top. Having hundreds of thousands of competitive players helps, but even with a small sample you are probably likely to get at least some very, very strong players. It's hard to think of a relevant real world example, but a fun corollary I'm familiar with is Fedex (Federico Perez Ponsa). He is a full chess Gran…

That type of logic is unsafe - we don't know that Ponsa is playing AoE with the same intensity as chess.

In fact, the idea that someone can train with sufficient intensity to be a high ranking chess master then break in to the top 50 of AoE at the same time suggests a lower skill saturation in the AoE world.

Re: Mastering Stratego

#93

Post by roenxi »

dasil003 wrote: post #42
VHRanger wrote: post #26

Earlier quoted context omitted.

AI remains better than humans at anything that has well defined rewards and small time gap between action and feedback mechanism (either naturally, like poker, or by value function engineering, like Go or Chess) The problem here is that it's missing the "glue" to more real world applications. This is where more humdrum software engineering comes in. Diplomacy in this is much more interesting than Stratego or beating…

> The problem here is that it's missing the "glue" to more real world applications. This is where more humdrum software engineering comes in. This is a bold statement. The world does not function based on "well-defined rewards". The concept of "common sense", which some consider table stakes for a human operating competently in our world, is mostly made up of things which are neither well-defined, nor allow a tremend…

> The world does not function based on "well-defined rewards".

Humans can be neatly reduced to a "procreate, and make sure your progeny procreate" value function. A lot of apparent counterexamples to that turn out to make sense when 2nd order effects on progeny are considered.

Re: Mastering Stratego

#94

Post by athrowaway12 »

dr_faustus wrote: post #20

Call me a cynic but the fact that after almost 10 years of AI hype we are still working our way down the list of popular board games is a bit of a downer for me. I mean, having AIs to play Stratego, Risk, Go, Diplomacy and what have you against sure is nice. But there are literally billions of dollars spent on these projects and I really come to the point where I just don't believe anymore that the current AI approac…

These board games are models of real human problems. And these reasoning and tree searching tasks are very general, and humans perform these very often in work and in personal life.

I agree that these specific models are not going to be useful outside of board games. But in the future when there is the opportunity for AIs to interact with the world for real, the this kind of research will allow AIs to dramatically outperform humans on these tasks.

Re: Mastering Stratego

#95

Post by aidenn0 »

clolege wrote: post #60
VHRanger wrote: post #43

Earlier quoted context omitted.

All Poker AIs developped as yet approaches Nash Equilibrium -- it's just a "perfect" strategy that wins by default because it makes no mistakes. Since you make mistakes against the AI strategy, and the sum of the game of poker is 0, you lose by default. No poker bots yet I know of have developed "exploitative" strategies, where they deviate from the Nash Equilibrium strategy to exploit opponent mistakes. Back when I…

Nash equilibriums exist when bluffing is involved? It seems like it would introduce a level of predictability that would make it easier to know when the opponent is bluffing.

They absolutely do. Since it's world cup time I'll use a contrived soccer example.

A penalty kick where the kicker can kick left or kick right. The goalie has to jump one direction, if they jump the wrong direction a goal is scored. Both people know that this kicker is great at kicking to the left side of the goal but rather "meh" at kicking to the right, so if the kicker kicks to the left and the goalie jumps left, there's still a 20% chance of scoring, but if the kicker kicks to the right and the goalie jumps right, there's only a 5% chance of scoring.

There is a Nash equilibrium for the kicker, and it can't be "always kick left" because then the goalie would "always jump left" which would give the kicker an advantage if it kicked right.

Similarly the Nash equilibrium for poker can't be to always fold a weak hand, because that's leaving money on the table because then the opponents will always fold against a raise, which would mean the player could get easy money by raising with a weak hand.

Re: Mastering Stratego

#96

Post by aidenn0 »

clolege wrote: post #30
sdwr wrote: post #22

Earlier quoted context omitted.

Watched some of the first game. I'd bet stratego favors defence, advantage to the AI that has no/minimal concept of the value of time.

Yeah this is one of the reasons why I find it more dull than chess. There is an incentive to just not move your pieces, so that the other player thinks they're bombs. As a result, players only activate 2-3 pieces at a time. In chess, on the other hand, you are constantly moving your pawns to the other side to promotion, or otherwise trying to activate/coordinate all of your pieces for an attack. It makes me think tha…

> It makes me think that if deepmind for Stratego was trained to not lose instead of win, then the top strategy might be shuffling pieces and letting the enemy come to attack. No human would ever have the patience to play that way though.

Tournament Stratego uses a clock, which reduces some of the issue there. It's not hard to beat a player that does what you suggest; just send some middling pieces after each piece that moves. You'll take the weak pieces and reveal the strong pieces.

It is much more defensive than chess in general though, as moving a piece and capturing with a piece both give the other player information.

Re: Mastering Stratego

#97

Post by Eji1700 »

nemo44x wrote: post #70
sdwr wrote: post #22

Earlier quoted context omitted.

Watched some of the first game. I'd bet stratego favors defence, advantage to the AI that has no/minimal concept of the value of time.

A common variation of the game is to let the aggressor win battles where both pieces have the same value. The default is a draw. This promotes aggression.

I was hoping they'd have examples of this variant in the testing data, because I think it MASSIVELY alters how you have to play

Re: Mastering Stratego

#98

Post by jstummbillig »

fnbr wrote: post #85
jstummbillig wrote: post #2

Can anyone shed light on in what way this is more challenging than the starcraft or dota agents, which also had to work with imperfect information?

I don’t think it is more challenging than StarCraft or Dota. Does the blog post claim that anywhere? Stratego is way more challenging than Poker, though. StarCraft/Dota/Stratego have the property that you can’t represent their imperfect information as a vector in memory, whereas you can easily do that in Texas hold’em poker (there’s only 52C2 = 1024 possible hands). So for those games, you have to use an approximate…

> I don’t think it is more challenging than StarCraft or Dota. Does the blog post claim that anywhere?

My very naive question question being: Why tackle this after SC and Dota have already been done? What is the scientific interest? Stratego seems strictly simpler than both of these games. In what way is this an advancement over how SC/Dota AI were solved?

Re: Mastering Stratego

#99

Post by fsckboy »

nemo44x wrote: post #70
sdwr wrote: post #22

Earlier quoted context omitted.

Watched some of the first game. I'd bet stratego favors defence, advantage to the AI that has no/minimal concept of the value of time.

A common variation of the game is to let the aggressor win battles where both pieces have the same value. The default is a draw. This promotes aggression.

why would you make a move that would place your piece next to an opponent if it just gave them the potential advantage in a battle? Wouldn't it increase wariness?

Re: Mastering Stratego

#100

Post by aidenn0 »

VHRanger wrote: post #29
clolege wrote: post #13

It's interesting to watch the videos they link of deepmind playing against the top-level Stratego masters [0]. I usually find Stratego to be a bit of a dull game (less elegant and more drawn out than Go and chess), but I'm a sucker for watching top-level AIs play. Its skills for bluffing are both fascinating and a bit scary. [0] https://www.youtube.com/watch?v=HaUdWoSMjSY https://www.youtube.com/watch?v=L-9ZXmyNKgs h…

FWIW one of the big things poker AI taught humans is massive overbets (eg. going all in for $200 over a $15 pot). This is scary to do well in practice, because the mathematically optimal bluff frequency approaches 50% as you increase the overbet size.

Is that heads-up only? Early massive overbets are nearly a coin toss in heads-up, so you break even in on a call and win on a fold, which is an equilibrium.

However at a large table, you are going to get called only by the person who thinks they have the best hand, which is a lot better than the average hand of a typical opponent.

Post reply on HN