Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: ChessCoach – A neural chess engine that comments on each player's moves

chrisbutner.github.io

Post reply on HN
Print view View original

11–20 of 92 posts

Re: Show HN: ChessCoach – A neural chess engine that comments on each player's moves

#11

Post by prezjordan »

cbutner wrote: post #10
prezjordan wrote: post #8

Remarkably cool. My only bit of feedback is that it's unfortunate so much of the commentary uses masculine pronouns (I found it distracting). Is that just a consequence of the training data?

Yeah, that's a massive problem with the natural language domain all across machine learning. Unfortunately it's very difficult to track down training data for chess commentary in the first place, let alone trim down biases. For reference, I was able to gather about 1 million samples, but it really needs a billion. Hopefully through data augmentation and better general intelligence models we can make better progress o…

Appreciate the honesty here. Pretty wild how natural this model feels with 1 million samples.

Re: Show HN: ChessCoach – A neural chess engine that comments on each player's moves

#12

Post by cbutner »

prezjordan wrote: post #11
cbutner wrote: post #10

Earlier quoted context omitted.

Yeah, that's a massive problem with the natural language domain all across machine learning. Unfortunately it's very difficult to track down training data for chess commentary in the first place, let alone trim down biases. For reference, I was able to gather about 1 million samples, but it really needs a billion. Hopefully through data augmentation and better general intelligence models we can make better progress o…

Appreciate the honesty here. Pretty wild how natural this model feels with 1 million samples.

Sometimes it seems really accurate (like the cherry-picked GIF in the overview docs) and sometimes really off.

I think for the most part, it knows more than it lets on, but finding the right sampling methods (or better yet, generalized search) to generate the best comments is a tough problem because it's difficult to evaluate quality.

There's some info on the sampling methods here: https://chrisbutner.github.io/ChessCoach/high-level-explanat...

Re: Show HN: ChessCoach – A neural chess engine that comments on each player's moves

#14

Post by cbutner »

egberts1 wrote: post #13

I look forward to reading things like “a classic Boris move” or Fisher or any of the great chessmasters.

It does tend to name-drop: often famous names, but also just "Jeff".

And if you spice up the commentary sampling parameters, it gets even more inventive, making up names, and saying that "the rook is pinning Fischer against the king".

Re: Show HN: ChessCoach – A neural chess engine that comments on each player's moves

#15

Post by mxwsn »

cbutner wrote: post #4
evanhu_ wrote: post #3

Earlier quoted context omitted.

I'm super impressed with what you've managed to create, do you have any further plans with this project? I'm curious now that it's finished and documented to such an extent will you try to bring it publicity and actual usage or was this just a passion project. Thanks

Thank you! I do get that itch to jump in and improve things whenever I see it lose a game, but I don't have further plans (development or commercial) in the near-term. The goal originally was to see whether I liked ML, to decide on my next industry/career move, but there was a lot of "one more month". I'm actually hopeful that some search techniques such as SBLE-PUCT[1] or better derivations can make their way into o…

So, do you like ML?

Re: Show HN: ChessCoach – A neural chess engine that comments on each player's moves

#16

Post by iooi »

cbutner wrote: post #2

This took about a year and a half – a little over a year coding in between experiments and training. It's a chess engine with a primary neural network just like AlphaZero or Leela Chess Zero's, but it adds on a secondary "commentary decoder" network based on Transformer architecture to comment on positions and moves. All of the code and data for training and search is from scratch, although it does use Stockfish code…

> It only plays one game at a time, so you may need to wait a little bit

Why this limitation? Is it fairly computationally expensive to run?

Re: Show HN: ChessCoach – A neural chess engine that comments on each player's moves

#17

Post by cbutner »

iooi wrote: post #16
cbutner wrote: post #2

This took about a year and a half – a little over a year coding in between experiments and training. It's a chess engine with a primary neural network just like AlphaZero or Leela Chess Zero's, but it adds on a secondary "commentary decoder" network based on Transformer architecture to comment on positions and moves. All of the code and data for training and search is from scratch, although it does use Stockfish code…

> It only plays one game at a time, so you may need to wait a little bit Why this limitation? Is it fairly computationally expensive to run?

Yes, each bot uses a v3-8 Cloud TPU VM, and tries to be constantly playing a game. The search tree is also very memory-hungry. And right now it's also using the Python API for TensorFlow, which is likely wasting a lot of potential.

Lots of room for improvement!

Re: Show HN: ChessCoach – A neural chess engine that comments on each player's moves

#18

Post by cbutner »

mxwsn wrote: post #15
cbutner wrote: post #4

Earlier quoted context omitted.

Thank you! I do get that itch to jump in and improve things whenever I see it lose a game, but I don't have further plans (development or commercial) in the near-term. The goal originally was to see whether I liked ML, to decide on my next industry/career move, but there was a lot of "one more month". I'm actually hopeful that some search techniques such as SBLE-PUCT[1] or better derivations can make their way into o…

So, do you like ML?

Haha - I dislike how much of a black box it is, despite the statistical basis (for example, the back and forth on batch normalization rationale). But lots of interesting problems and tech to dig into.

Re: Show HN: ChessCoach – A neural chess engine that comments on each player's moves

#19

Post by eeegnu »

Just tried it out, really interesting to read post game: https://lichess.org/dKbbqymG/

I don't think this comment was accurate:

> 23. c4: "White tries to get his pawns moving. I am still thinking that I have to move my bishop, but that is too slow. I think white should have moved his king back to c3 to prevent my pawn from becoming a passer."

The idea behind moving the pawn was because black playing c4 would have instantly lost both pawns since the bishop has nowhere safe to go where it still defends c2 (Bh7 leading to g6 losing the piece.) I don't think the king could have made it to the c file in time to stop that.

> 34... Rxg4: "Now I can't stop him from advancing his pawn."

Is this meant to be speaking from whites perspective (since black just got the unopposed h file pawn.)

Interestingly only 10. O-O links with a move, I think it would be really helpful if they all linked to moves. Also i'm really excited for this kind of analysis! It would be really cool if request a computer analysis eventually generated such analyses trying to figure out each sides ideas.

Re: Show HN: ChessCoach – A neural chess engine that comments on each player's moves

#20

Post by sobellian »

One of my pet peeves is when folks compare human elo ratings from FIDE to (presumably) engine elo ratings from CCRL, a league for chess engines. They're not the same! This is like comparing points between your local youth soccer league and La Liga. All we need to say is that this engine is only 100 elo behind Stockfish. That is akin to occasionally beating God in a game of chess.
Post reply on HN