How AlphaZero Mastered Its Games • Hacker News Board

Re: How AlphaZero Mastered Its Games

#51

Post by skyyler » Sat, Dec 29, 2018, 5:38 PM UTC

skybrian wrote: post #49

assblaster wrote: post #47

Earlier quoted context omitted.

>An expert human player is an expert precisely because her mind automatically identifies the essential parts of the tree and focusses its attention there. Instead of using gender neutral pronoun like "they", the author used a feminine pronoun.

This is nitpicking. I like using singular "they" and I'm guessing it will win, but this isn't settled yet and some editors make different choices.

I don't like singular "they", we should try to establish a second person plural pronoun.

Re: How AlphaZero Mastered Its Games

#52

Post by Para2016 » Sat, Dec 29, 2018, 5:39 PM UTC

assblaster wrote: post #47

cdelsolar wrote: post #42

Earlier quoted context omitted.

what gender pronoun controversy? did we read the same article?

>An expert human player is an expert precisely because her mind automatically identifies the essential parts of the tree and focusses its attention there. Instead of using gender neutral pronoun like "they", the author used a feminine pronoun.

I'm not sure that out of those thousands of words that's the idea that deserves notice and discussion.

Re: How AlphaZero Mastered Its Games

#53

Post by sobellian » Sat, Dec 29, 2018, 5:51 PM UTC

cdelsolar wrote: post #44

Awesome article. Does anyone know how to begin applying the AlphaZero techniques to games where information is NOT perfect? I'm trying to apply it to Scrabble. There hasn't been much AI research in this game and right now the best AI just uses brute force Monte Carlo with a flawed evaluation function (which doesn't take into account the state of the board at all, just points and tiles remaining on the opponent's rack…

If you search "UCT imperfect information" on Google, you'll turn up plenty of articles and slide decks, including one from David Silver that discusses reinforcement learning. The catch is that they're mostly dated before AlphaZero's emergence, so there's some original work involved to extend AlphaZero to this domain. This is likely something that DeepMind is working on themselves. It's possible that tweaking the search query might turn up more recent results. Good luck!

EDIT - The David Silver lecture I mentioned actually mentions a Scrabble AI, Maven, which successfully applies MCTS. Here's a link: http://www0.cs.ucl.ac.uk/staff/d.silver/web/Teaching_files/g...

Re: How AlphaZero Mastered Its Games

#54

Post by mda » Sat, Dec 29, 2018, 6:07 PM UTC

ag1 wrote: post #50

mda wrote: post #45

Earlier quoted context omitted.

Like which move? I wonder why stockfish developers do not claim any of this?

glinscott said "We need a public exhibition match to settle the score, ideally with some GM commentary." above. As for the GM side, Nakamura basically said the same.

Still, zero proof of foul play. But sure i would very much enjoy watching Alphazero destroy Stockfish in a public match.

Re: How AlphaZero Mastered Its Games

#55

Post by WillPostForFood » Sat, Dec 29, 2018, 6:19 PM UTC

assblaster wrote: post #47

cdelsolar wrote: post #42

Earlier quoted context omitted.

what gender pronoun controversy? did we read the same article?

>An expert human player is an expert precisely because her mind automatically identifies the essential parts of the tree and focusses its attention there. Instead of using gender neutral pronoun like "they", the author used a feminine pronoun.

It is especially jarring when there is only one woman in the current top 100 chess players, Yifan Hou ranked 86.

Re: How AlphaZero Mastered Its Games

#56

Post by dhh2106 » Sat, Dec 29, 2018, 6:41 PM UTC

YeGoblynQueenne wrote: post #15

>> In fact, less than two months later, DeepMind published a preprint of a third paper, showing that the algorithm behind AlphaGo Zero could be generalized to any two-person, zero-sum game of perfect information (that is, a game in which there are no hidden elements, such as face-down cards in poker). I can't find this claim in the linked paper. What I can find is a statement that AlphaZero has demonstrated that 'a g…

Any explanation as to why this should not be used for games without perfect information? As an example, why couldn't the face-down card in poker be modeled as part of the MCTS?

Re: How AlphaZero Mastered Its Games

#57

Post by IanCal » Sat, Dec 29, 2018, 7:03 PM UTC

kzrdude wrote: post #39

IanCal wrote: post #25

Earlier quoted context omitted.

That doesn't follow. For you to confuse it, you need to change the inputs. For images, this is fine, we can smoothly change lots of little things. For chess games or go you don't have that freedom. You can download the weights for LCZero right now though and try out your theory. https://github.com/LeelaChessZero/lc0/wiki/Getting-Started

You can change the inputs: it depends on when (ply) and which move you play. Some moves are uncommon enough to make it possible for you to uncover something?

You absolutely can change the inputs, but the point I wanted to make is that unlike images where you can make a human-irrelevant changes you can't really do that with chess or go.

If you want to construct a particular position on the board, you'd likely need to use multiple steps, require the AI to play very particular moves and then the outcome would be a certain move from the AI. Even then, a simple incorrect classification doesn't help all that much, you need your opponent to make repeated mistakes.

I think in reality if you uncovered a type of move it wasn't expecting you are likely to uncover a new strategy in general rather than a trick. Image classification however lets you play uninterrupted with tiny pixel value changes, and you only need a single incorrect output to "win".

Re: How AlphaZero Mastered Its Games

#58

Post by IanCal » Sat, Dec 29, 2018, 7:07 PM UTC

CJefferson wrote: post #37

IanCal wrote: post #25

Earlier quoted context omitted.

That doesn't follow. For you to confuse it, you need to change the inputs. For images, this is fine, we can smoothly change lots of little things. For chess games or go you don't have that freedom. You can download the weights for LCZero right now though and try out your theory. https://github.com/LeelaChessZero/lc0/wiki/Getting-Started

You are right, I should try. I'll see if I can find time in the new year. I'd prefer to try with a go player, because as you say, in chess it's hard to exactly control the input to the network, it's easier in Go.

Here's a go setup https://github.com/gcp/leela-zero

There's current best weights available. Not alphazero, but I would expect that issues would be general and so if there are issues with leela zero they may transfer and if you don't see issues with leela zero they're unlikely to exist in alpha zero (at least, if they do they may be very particular to subtle training differences).

Would be very interested to see what you find if you get the chance.

Re: How AlphaZero Mastered Its Games

#59

Post by BurningFrog » Sat, Dec 29, 2018, 7:55 PM UTC

YeGoblynQueenne wrote: post #15

>> In fact, less than two months later, DeepMind published a preprint of a third paper, showing that the algorithm behind AlphaGo Zero could be generalized to any two-person, zero-sum game of perfect information (that is, a game in which there are no hidden elements, such as face-down cards in poker). I can't find this claim in the linked paper. What I can find is a statement that AlphaZero has demonstrated that 'a g…

> stochastic games of perfect information

Honest question:

Are games like backgammon really considered "perfect information" in the sense relevant here?

No player has any secrets from the other, but neither knows what the dice will do, which certainly is important information.

Re: How AlphaZero Mastered Its Games

#60

Post by [deleted] » Sat, Dec 29, 2018, 8:25 PM UTC

assblaster wrote: post #47

cdelsolar wrote: post #42

Earlier quoted context omitted.

what gender pronoun controversy? did we read the same article?

>An expert human player is an expert precisely because her mind automatically identifies the essential parts of the tree and focusses its attention there. Instead of using gender neutral pronoun like "they", the author used a feminine pronoun.

[deleted]