How AlphaZero Mastered Its Games • Hacker News Board

Re: How AlphaZero Mastered Its Games

#31

Post by melling » Sat, Dec 29, 2018, 2:32 PM UTC

alan_wade wrote: post #14

God what a well written article! I don't have much to say on the subject, but this was pure joy to read, it's crazy good. Clear, engaging, to the point, making a difficult subject accessible without dumbing it down, no fluff or unnecessary side stories, just awesomeness.

Welcome to The New Yorker.

Malcolm Gladwell, as well as many other great writers, can be found here:

https://www.newyorker.com/contributors/malcolm-gladwell

This weeks story on Donald Trump might be of interest:

https://www.newyorker.com/magazine/2019/01/07/how-mark-burne...

Re: How AlphaZero Mastered Its Games

#32

Post by assblaster » Sat, Dec 29, 2018, 2:32 PM UTC

alan_wade wrote: post #14

God what a well written article! I don't have much to say on the subject, but this was pure joy to read, it's crazy good. Clear, engaging, to the point, making a difficult subject accessible without dumbing it down, no fluff or unnecessary side stories, just awesomeness.

The only gripe I had with the writing was the completely unnecessary injection of gender pronoun controversy.

I highly recommend the AlphaGo movie as well, it does a great job documenting the psychology of professional Go in the world of AI.

Re: How AlphaZero Mastered Its Games

#33

Post by thom » Sat, Dec 29, 2018, 2:52 PM UTC

noir_lord wrote: post #27

pie_hacker wrote: post #26

The match between Stockfish and AlphaZero was played with certain unjustified parameters (time control, ponder off, different hardware, no opening book or endgame tablebase for Stockfish etc.). By "unjustified," I mean that the authors of the paper did not justify their choice of parameters in the paper as being designed to implement a fair match. At a glance, the parameters of the match seem unfair to me -- and tilt…

That was the first paper. Second paper they played fair even giving stockfish a 10 to 1 advantage on same hardware as tcec uses.

You can still run the games past the exact commit of Stockfish they used and it finds blunders in its own play, so it still feels like there's a lack of transparency. But I don't think anyone strongly believes AlphaZero isn't the best at this point.

Re: How AlphaZero Mastered Its Games

#34

Post by thom » Sat, Dec 29, 2018, 2:57 PM UTC

yesenadam wrote: post #28

pie_hacker wrote: post #26

The match between Stockfish and AlphaZero was played with certain unjustified parameters (time control, ponder off, different hardware, no opening book or endgame tablebase for Stockfish etc.). By "unjustified," I mean that the authors of the paper did not justify their choice of parameters in the paper as being designed to implement a fair match. At a glance, the parameters of the match seem unfair to me -- and tilt…

There has been a rematch recently vs Stockfish, with a couple of hundred games. AlphaZero won 155-6! [0] There are fascinating videos with grandmasters commentating on some of the games. They're played in an exciting, sacrificial, swashbuckling style, nothing like any other top computer engine, and it seems that may affect the play of top (human) players for the better. e.g. see Matthew Sadler on chess24 https://www.…

A lot of the moves most praised by GMs are seen as the only moves in the position by Stockfish 9/10. I think there's a huge amount of cognitive dissonance going on, so that people can label AlphaZero's play more 'human'.

Anyway, I wouldn't be surprised if AlphaZero lines have existed at the top of the game for some time. Would be a no brainer for someone to have made Google an offer after the first paper.

Re: How AlphaZero Mastered Its Games

#35

Post by thom » Sat, Dec 29, 2018, 3:02 PM UTC

CJefferson wrote: post #22

bertil wrote: post #19

Earlier quoted context omitted.

Adversarial? If the model exclusively trains against itself, you can’t really insert anything there. Do you mean, play confusing moves at the beginning of the game?

I mean, if we had the network, it would be easy to beat, the same way you can confuse image recognition systems with very minor changes.

It's suspect it's a bit harder for the network to be overfit like this, but it's probably possible it has some gaps in its evaluation. However, those gaps would have to persist beyond its search horizon and not concretely affect material or mobility and it just seems vanishingly unlikely you'll find any systematic way to exploit anything.

Re: How AlphaZero Mastered Its Games

#36

Post by tosser0001 » Sat, Dec 29, 2018, 3:10 PM UTC

> An expert human player is an expert precisely because her mind automatically identifies ...

The "Patronizing 'Her'"

Almost invariably, when the author decides to use the patronizing 'her' instead of the gender-neutral 'they' it's written by a man.

Re: How AlphaZero Mastered Its Games

#37

Post by CJefferson » Sat, Dec 29, 2018, 3:15 PM UTC

IanCal wrote: post #25

CJefferson wrote: post #22

Earlier quoted context omitted.

I mean, if we had the network, it would be easy to beat, the same way you can confuse image recognition systems with very minor changes.

That doesn't follow. For you to confuse it, you need to change the inputs. For images, this is fine, we can smoothly change lots of little things. For chess games or go you don't have that freedom. You can download the weights for LCZero right now though and try out your theory. https://github.com/LeelaChessZero/lc0/wiki/Getting-Started

You are right, I should try. I'll see if I can find time in the new year.

I'd prefer to try with a go player, because as you say, in chess it's hard to exactly control the input to the network, it's easier in Go.

Re: How AlphaZero Mastered Its Games

#38

Post by pie_hacker » Sat, Dec 29, 2018, 3:17 PM UTC

tosser0001 wrote: post #36

> An expert human player is an expert precisely because her mind automatically identifies ... The "Patronizing 'Her'" Almost invariably, when the author decides to use the patronizing 'her' instead of the gender-neutral 'they' it's written by a man.

Not in my experience...

I have noticed that people who exclusively use "his" rather than "her" or "their" tend to be men though.

Re: How AlphaZero Mastered Its Games

#39

Post by kzrdude » Sat, Dec 29, 2018, 3:18 PM UTC

IanCal wrote: post #25

CJefferson wrote: post #22

Earlier quoted context omitted.

I mean, if we had the network, it would be easy to beat, the same way you can confuse image recognition systems with very minor changes.

That doesn't follow. For you to confuse it, you need to change the inputs. For images, this is fine, we can smoothly change lots of little things. For chess games or go you don't have that freedom. You can download the weights for LCZero right now though and try out your theory. https://github.com/LeelaChessZero/lc0/wiki/Getting-Started

You can change the inputs: it depends on when (ply) and which move you play. Some moves are uncommon enough to make it possible for you to uncover something?

Re: How AlphaZero Mastered Its Games

#40

Post by bko » Sat, Dec 29, 2018, 3:30 PM UTC

CJefferson wrote: post #22

bertil wrote: post #19

Earlier quoted context omitted.

Adversarial? If the model exclusively trains against itself, you can’t really insert anything there. Do you mean, play confusing moves at the beginning of the game?

I mean, if we had the network, it would be easy to beat, the same way you can confuse image recognition systems with very minor changes.

The way general adversarial networks work on tricking image recognition systems is that they vary pixels of the input image slightly to manipulate the output of the neural network.

For alphazero, the input is the board, which you can't manipulate arbitrarily. You can run an evaluation of a board based on a move and see if its significantly different than the evaluation that alphazero comes up with, and maybe try to exploit that. But if you have a better evaluation of some state than that of alphazero, you're likely a stronger player anyway so this extra step is unnecessary. Most of the value of the bot comes from the evaluation function of a board, along with some hyper-parameters. But the evaluation is probably the most important part and the most difficult to replicate.