Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Is AlphaZero really a breakthrough in AI?

medium.com

Post reply on HN
Print view View original

61–70 of 84 posts

Re: Is AlphaZero really a breakthrough in AI?

#61

Post by super_mario »

I would like to see a rematch with Stockfish configured correctly. I give Stockfish at least 1 GB of hash per thread. In AlphaZero match they had 64 threads and only 1 GB of hash for all of them.

No one knows how Stockfish behaves with that many search threads, since no one tested it. I don't know if there is any data on how Stockfish scales with number of CPUs but I seem to remember that being one of the weaknesses of the engine, and that commercial engines like Komodo scaled better with larger number of CPUs.

Anecdotally on 2.8 GHz 8 core 64bit CPU with 8 threads and 16 GB hash size it calculates about 7-8 million ply per second at the beginning of the game, and much more later when there are less peaces on the board. AlphaZero's setup, with 64 threads on 32 physical CPU cores, calculated about 70 million ply per second, with tiny 1 GB hash (i.e. the engine could remember less of what it calculated previously).

But my Stockfish on computationally weaker setup clearly flags some of the moves in the match as mistakes on the 64 threaded Stockfish. I would really like to understand why. Is it because if you have more resources to see deeper you see how hopeless the situation is, or is it something else?

I am sure we will see more of these matches. It would be nice if Google volunteered some computing resources and entered TCEC regularly.

Re: Is AlphaZero really a breakthrough in AI?

#62

Post by jraut »

An AI which excels in imperfect information games (card games, Starcraft) would be a real breakthrough. Raw calculation power is bound to win games with a finite set of possibilities. The huge leap would be the ability to handle probabilites: taking guesses, making assumptions and coming into some kind of successful conclusions based on those.

Re: Is AlphaZero really a breakthrough in AI?

#63

Post by roganp »

super_mario wrote: post #61

I would like to see a rematch with Stockfish configured correctly. I give Stockfish at least 1 GB of hash per thread. In AlphaZero match they had 64 threads and only 1 GB of hash for all of them. No one knows how Stockfish behaves with that many search threads, since no one tested it. I don't know if there is any data on how Stockfish scales with number of CPUs but I seem to remember that being one of the weaknesses…

It seems to me that that someone can prove that the configuration mattered by pitting a correctly configured Stockfish against the configuration used by AlphaZero, and see what the outcome is for 100 games. I haven't read the paper, but some of the constraint choices seem odd to me (1 minute max per move?).

Re: Is AlphaZero really a breakthrough in AI?

#64

Post by dingo_bat »

jraut wrote: post #62

An AI which excels in imperfect information games (card games, Starcraft) would be a real breakthrough. Raw calculation power is bound to win games with a finite set of possibilities. The huge leap would be the ability to handle probabilites: taking guesses, making assumptions and coming into some kind of successful conclusions based on those.

Go cannot be won with raw power though. It has a lot of the constraints of incomplete information problems.

Re: Is AlphaZero really a breakthrough in AI?

#65

Post by joelthelion »

sanxiyn wrote: post #10
joelthelion wrote: post #6

Earlier quoted context omitted.

Is it really impossible to adapt Deepmind's approach to, say, poker? I don't see anything fundamentally impossible, but I'm no RL expert.

I would consider a successful application of AlphaZero algorithm to imperfect information games to be a separate achievement, not a done deal. David Silver himself, who led AlphaGo project, wrote in 2016 in "Deep Reinforcement Learning from Self-Play in Imperfect-Information Games" https://arxiv.org/abs/1603.01121 "While many machine learning methods have achieved near-optimal solutions to classical, perfect-informat…

Thanks for that paper, very interesting.

Re: Is AlphaZero really a breakthrough in AI?

#66

Post by narimiran »

soVeryTired wrote: post #18
rstuart4133 wrote: post #4

The article glosses over why the 4 hours was possible. Firstly, a major challenge in training an AI of this sort is getting enough labelled data. They played 300,000 games from memory. Under normal circumstances, that requires access to 300,000 games played by experts so the AI can learn to copy what the export does. That is how Alpha Go did it. AlphaZero neatly side steps this by generating it's own training data by…

> AlphaZero neatly side steps this by generating it's own training data by playing itself. If how to do this was "obvious", it would have been done a long time ago. Learning by self-play is nearly as old as AI itself. TD Gammon, one of the very first algorithms to reach superhuman levels in a nontrivial game, learned by self play. The basic ingredients for AlphaGo Zero, Monte-Carlo tree search and use of a convnet to…

> TD Gammon, one of the very first algorithms to reach superhuman levels in a nontrivial game, learned by self play.

Thank you for mentioning this! Neural-nets self-trained bots have been a thing for quite a while in backgammon. Chronologically, from my memory: TD Gammon, Jellyfish, Snowie, GnuBG [1], XG (eXtreme Gammon) [2]

[1] http://gnubg.org/

[2] http://extremegammon.com/

Re: Is AlphaZero really a breakthrough in AI?

#67

Post by super_mario »

roganp wrote: post #63
super_mario wrote: post #61

I would like to see a rematch with Stockfish configured correctly. I give Stockfish at least 1 GB of hash per thread. In AlphaZero match they had 64 threads and only 1 GB of hash for all of them. No one knows how Stockfish behaves with that many search threads, since no one tested it. I don't know if there is any data on how Stockfish scales with number of CPUs but I seem to remember that being one of the weaknesses…

It seems to me that that someone can prove that the configuration mattered by pitting a correctly configured Stockfish against the configuration used by AlphaZero, and see what the outcome is for 100 games. I haven't read the paper, but some of the constraint choices seem odd to me (1 minute max per move?).

Unfortunately, AlphaZero is private and no one has access to either the code (as commercial product) or resources it needs to run (TPUs). So, only Google can do this experiment.

Re: Is AlphaZero really a breakthrough in AI?

#68

Post by thisacctforreal »

super_mario wrote: post #67
roganp wrote: post #63

Earlier quoted context omitted.

It seems to me that that someone can prove that the configuration mattered by pitting a correctly configured Stockfish against the configuration used by AlphaZero, and see what the outcome is for 100 games. I haven't read the paper, but some of the constraint choices seem odd to me (1 minute max per move?).

Unfortunately, AlphaZero is private and no one has access to either the code (as commercial product) or resources it needs to run (TPUs). So, only Google can do this experiment.

OP is suggesting properly-configured-stockfish versus how-alphazero-configured-stockfish for 100 games.

Re: Is AlphaZero really a breakthrough in AI?

#69

Post by sireat »

super_mario wrote: post #61

I would like to see a rematch with Stockfish configured correctly. I give Stockfish at least 1 GB of hash per thread. In AlphaZero match they had 64 threads and only 1 GB of hash for all of them. No one knows how Stockfish behaves with that many search threads, since no one tested it. I don't know if there is any data on how Stockfish scales with number of CPUs but I seem to remember that being one of the weaknesses…

I too was dissapointed at the 1GB of hash given to Stockfish.

Turns out that there is a good reason for 1GB of hash : it is an easy way to get a high number nodes searches per second (ie high kn/sec) on many threads.

http://support.stockfishchess.org/discussions/questions/655-...

Stockfish is a major PIA to configure to use correctly and I do not blame AlphaZero for taking the easy way out..

I turned on 30 threads for Stockfish 8 on my 2x Xeon 2670 with 64GBs of RAM and tried to adjust hash memory.

When going past 1GB hash my kn/sec dropped from 25kn/sec to less than 10kn/sec

It might have to do with how 2x CPU configurations work, locality, who knows.

A0 should have let Stockfish use its opening book though.

Re: Is AlphaZero really a breakthrough in AI?

#70

Post by inopinatus »

Marazan wrote: post #38
inopinatus wrote: post #14

Earlier quoted context omitted.

Lines of Acton sounds like an excellent game for Londoners. (Acton is a suburb of London and has more railway stations bearing its name than any other place in the UK)

Ah yes, a derivative of Mornington Crescent more suitable for computers to play.

It'll be a mark of truly general AI when computers can learn to play Mornington Crescent, and play to win.

They are, of course, still likely to struggle against an advanced player experienced with the Abingdon Gambit.

Post reply on HN