Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: ChessCoach – A neural chess engine that comments on each player's moves

chrisbutner.github.io

Post reply on HN
Print view View original

71–80 of 92 posts

Re: Show HN: ChessCoach – A neural chess engine that comments on each player's moves

#71

Post by shmageggy »

modeless wrote: post #27
cbutner wrote: post #2

This took about a year and a half – a little over a year coding in between experiments and training. It's a chess engine with a primary neural network just like AlphaZero or Leela Chess Zero's, but it adds on a secondary "commentary decoder" network based on Transformer architecture to comment on positions and moves. All of the code and data for training and search is from scratch, although it does use Stockfish code…

Where did you source the commentary dataset?

Seconded. I looked around the writeup site for a bit and couldn't figure that out. That's arguably the most important piece of info about this project.

Re: Show HN: ChessCoach – A neural chess engine that comments on each player's moves

#72

Post by conistonwater »

bskrobisz wrote: post #65

> 3... e6: "A solid move, and I have never seen it before. It has been played by many strong players, including GMs, Zvjaginsev, and others. It seems quite good overall, but occasionally there are fun AI-isms like this :)

> 1. e4: "This game is a game that I lost because of a tactical oversight on my part. I had a tactical idea in mind, but in this game, black resigned. I would have played on for a while, but I was able to find a way to convert my material advantage into a win."

Re: Show HN: ChessCoach – A neural chess engine that comments on each player's moves

#74

Post by mushishi »

Hey, this is great, have been waiting for something like this.

One bug found: https://lichess.org/NvbQTf2O/black

> PlayChessCoach 23. Nc7: "The point of White's previous move. The knight on d7 is trapped and the rook on a8 is not protected."

There is no knight on d7 or no knight that could be moved there. (Or so it seems to me.)

In that same game this was funny banter:

> PlayChessCoach careful not to get mated in the long run."

Thank you for writing architectural overview, enjoying it!

Re: Show HN: ChessCoach – A neural chess engine that comments on each player's moves

#75

Post by sixstringtheory »

Very cool project. I'm currently reading The Alignment Problem by Brian Christian on AI (very well written and easy to read), and also (very slowly) My Great Predecessors, Vol 1 by Garry Kasparov. This is a great combination of the two topics!

I'm particularly interested in the second neural net that generates explanations. Is it only a neural net to generate the natural language using artifacts from the original engine net, or is it actually inspecting the state of the engine net to derive the insights?

It's an interesting application of explainability of AI algorithms that Christian talks about in his chapter on transparency. In particular, he discusses "saliency" of algorithms (knowing what parts of the input were most important in producing a prediction) and "multitask nets" that output multiple predictions (so, maybe here, one output is the best move, and another output is the explanation).

The writeups are fantastic reading. I see almost no sources in common with the bibliography in The Alignment Problem (which has a 50 page bib) which makes them nice complements. Only common citations I could find were Sutton 1988 (Temporal Differences) and Silver et al 2016 (AlphaGo), 2017 (AlphaGo Zero) and 2018 (AlphaZero).

There's a note (44) from chapter 5 entitled "Shaping" where Christian talks about "meta-reasoning: the right way to think about thinking. When you play a game–for instance, chess–you win because of the moves you chose, but it was the thoughts you had that enabled you to choose those moves...Figuring out how an aspiring chess player–or any kind of agent–should learn about its thought process seemed like a more important but also dramatically harder task than simply learning how to pick good moves." This note might as well be direct inspiration for this project. It goes on to quote Stuart Russell about "a computation that changes your mind about what is a good move to make...reward that computation by how much you changed your mind...so you could change your mind in the sense of discovering that what was the second-best move is actually even better than what was the best move." That's in the context of a cautionary tale where only optimizing for those "changes of mind" doesn't necessarily find a correct outcome, and that you have to "arrange these internal pseudorewards so that along a path, they add up to the same as true, eventually." This sounds pretty much like the task of a coach.

Re: Show HN: ChessCoach – A neural chess engine that comments on each player's moves

#76

Post by dmurray »

gowld wrote: post #61
cbutner wrote: post #2

This took about a year and a half – a little over a year coding in between experiments and training. It's a chess engine with a primary neural network just like AlphaZero or Leela Chess Zero's, but it adds on a secondary "commentary decoder" network based on Transformer architecture to comment on positions and moves. All of the code and data for training and search is from scratch, although it does use Stockfish code…

Can you have it play more games by giving it less time per turn (~2500 rating is plenty good for an opponent/coach) and playing games concurrently while it waits for human to play? how much does a game cost in CPU time money? How do I get the commentary for a game I played? Oh, it's in Analysis page. It plays chess very well, but the commentary is incoherent and doesn't match the game well -- The attacks described ar…

> the commentary is incoherent and doesn't match the game well > The attacks described are nonsense and the coordinates are wrong.

Agreed, this looks superficially like commentary on the game, but honestly it doesn't seem more pertinent to the game score than a Markov chain trained on all the commentary would be (presumably this isn't true, and the author started with something like that Markov chain and the current version is way better in terms of some fitness function).

I wonder if there just is not enough training data available. GPT-3 overcomes this by harvesting a ridiculous amount of training data. AlphaZero, and the chess engine here, which is excellent, overcome it by generating their own training data through self play. But that's not applicable to the task of generating commentary.

Re: Show HN: ChessCoach – A neural chess engine that comments on each player's moves

#77

Post by ritchiea »

cbutner wrote: post #4
evanhu_ wrote: post #3

Earlier quoted context omitted.

I'm super impressed with what you've managed to create, do you have any further plans with this project? I'm curious now that it's finished and documented to such an extent will you try to bring it publicity and actual usage or was this just a passion project. Thanks

Thank you! I do get that itch to jump in and improve things whenever I see it lose a game, but I don't have further plans (development or commercial) in the near-term. The goal originally was to see whether I liked ML, to decide on my next industry/career move, but there was a lot of "one more month". I'm actually hopeful that some search techniques such as SBLE-PUCT[1] or better derivations can make their way into o…

You estimate it’s rated 3400-ish and it loses games????

Re: Show HN: ChessCoach – A neural chess engine that comments on each player's moves

#78

Post by dmurray »

modeless wrote: post #27
cbutner wrote: post #2

This took about a year and a half – a little over a year coding in between experiments and training. It's a chess engine with a primary neural network just like AlphaZero or Leela Chess Zero's, but it adds on a secondary "commentary decoder" network based on Transformer architecture to comment on positions and moves. All of the code and data for training and search is from scratch, although it does use Stockfish code…

Where did you source the commentary dataset?

Not the author, but ChessBase sell a product (Megabase) which includes 85,000 annotated games in a more-or-less machine readable format. [0]

To me it's probably OK to train a model on this, at least for hobby purposes, though some GitHub Copilot critics might disagree. And a large part of ChessBase's business model is based on ripping off other people's IP and presenting it as their own [1]. But still, I can see why the author might want to be coy about answering this question.

[0] https://en.chessbase.com/post/new-mega-database-2021

[1] https://lichess.org/blog/YCvy7xMAACIA8007/fat-fritz-2-is-a-r...

Re: Show HN: ChessCoach – A neural chess engine that comments on each player's moves

#79

Post by C19is20 »

prezjordan wrote: post #8

Remarkably cool. My only bit of feedback is that it's unfortunate so much of the commentary uses masculine pronouns (I found it distracting). Is that just a consequence of the training data?

Would 'queen' be considered masculine or feminine? Bishops? Knights (joan of arc excepted)? Are pawns slaves? Ahhh, the sport of kings, reduced to 2021.

Re: Show HN: ChessCoach – A neural chess engine that comments on each player's moves

#80

Post by MauranKilom »

One half-baked idea that came to my mind while watching the bot play on Lichess: You could maybe attempt something similar to how Lichess extracts puzzles. The puzzles generally revolve around sharp positions, so Lichess presumably has some metric for identifying those. They also have some auto-tagging for what kind of situation it is (Fork? Pin? Zugzwang?), although from what I gather these are community-corrected.

That's overall less general than "feed position network output into a transformer", but presumably less data-constrained.

Post reply on HN