Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Grandmaster-Level Chess Without Search

arxiv.org

Post reply on HN
Print view View original

111–120 of 133 posts

Re: Grandmaster-Level Chess Without Search

#111

Post by EGreg »

Next up, claims of ...

  Text generation without human writers
  Image generation without human artists
  Video generation without production crews
The rub is that all the training data has been done by lots and lots of search, human writing, human artists, and human production crews. At every step, they curated the results and generated a beautiful latent space that the model was then trained on.

The impressive part about AlphaZero is that it did the Monte-Carlo Tree Search itself, without being trained on human decision-making.

While this ... well, this is just taking credit for all the search and work that was done by Stockfish, or humans, at a massive scale over centuries and then posted nearly for free online. It's then using all that data and generating similar stuff. Whoop de doo. Oh, and it's even used cheap labor for the last mile, too:

https://time.com/6247678/openai-chatgpt-kenya-workers/

It's not the same thing as actual search to, for example, automatically derive scientific laws (e.g. Kepler's laws of motion) from raw data fed to it (e.g. of star movements). AI doing that can actually model the real space, not the latent space. It can go out and learn without humans or stockfish massively bootstrapping its knowledge.

I mean, don't get me wrong ... learning a lot about the latent space is what students strive to do in schools and universities, and the AIs are like a very smart student. In fact, they can be huge polymaths and polyglots and therefore uncover a lot of interesting connections and logical deductions from the latent space. They can do so at a huge scale... and I have often said that swarms of AIs will be unstoppable. So at the end of the day, although this isn't very impressive when it comes to the credit of who did the search and curation, AI is going to be extremely impressive with what it can do with the results on the next N levels.

Re: Grandmaster-Level Chess Without Search

#112

Post by mtlmtlmtlmtl »

epups wrote: post #109
mtlmtlmtlmtl wrote: post #102

Earlier quoted context omitted.

Yeah, NNUE is a separate invention that unfortunately, Deepmind often get undeserved credit for inspiring. It didn't even originate in chess engines but a shogi version of Stockfish. Architecture is completely different from the nets in Leela or Alpha Zero.

Wait, so progress on Stockfish would happen regardless of Alpha Chess? I always thought they were inspired by it in the newer versions, and got much improved rating from incorporating it.

Well, NNUE is surprisingly similar to what Stockfish was doing before NNUE. Before it was doing what's called piece-square tables. The basic idea(the stockfish evaluator had a lot more going on in addition, using multiple tables and interpolating between them based on game phase) is to assign some heuristic value to every square, for every piece. So it's just a 6x8x8 array that maps piece positions to values.

To get the evaluation of the whole position, you add up all of these mappings for the pieces on the board with opposite signs for the opposing players.

If you blur your eyes a little, this already looks a lot like a neural net. It's just a big summation of terms, and if you leave in a 0*(whatever value) for every piece that's not present, you've effectively embedded your lookup table into a giant mathematical expression that can be optimised by gradient descent.

The reason computer shogi programmers stumbled on this is that they were experimenting with adding more dimensions to the piece-square table, specifically via indexing by king position as well. So now you have 4 or 5 dimensions, making for a pretty massive array. Hand-tuning all the values becomes less and less feasible, and so I think discovering this idea of rearchitecting it as a neural-net was more or less inevitable.

So NNUE is actually just a pretty natural evolution of what they were doing before.

Re: Grandmaster-Level Chess Without Search

#113

Post by dmurray »

zniturah wrote: post #25
paulddraper wrote: post #18

There is rampant misunderstanding of some parts of this article; allow me to help :) The "no-search" chess engine uses search (Stockfish) in in two ways: 1. To score positions in the training data. This is only training data, no search is performed when actually playing. 2. To play moves when the position has many options with a 99% win rate. This is to prevent pathological behavior in already won positions, and is n…

"Aready won position" or "99% win rate" is statistics given by Stockfish (or professional chess player). It is weird to assume that the same statement is true for the trained LLM since we are assessing the LLM itself. If it is using during the game then it is searching, thus the title doesn't reflect the actual work.

It's quite clear from the article that the 99% is the model's predicted win rate for a position, not its evaluation by Stockfish (which doesn't return evaluations in those terms).

It's true that this is a relatively large deficiency in practice: how strong would a player be if he played the middlegame at grandmaster strength but couldn't reliably mate with king and rook?

The authors overcame the practical problem by just punting to Stockfish in these few cases. However, I think it's clearly solvable with LLM methods too. Their model performs poorly because of an artifact in the training process where mate-in-one is valued as highly as mate-in- fifteen. Train another instance of the model purely on checkmate patterns - it can probably be done with many fewer parameters - and punt to that instead.

Re: Grandmaster-Level Chess Without Search

#114

Post by samus »

joe_the_user wrote: post #44
janalsncm wrote: post #40

Earlier quoted context omitted.

A sufficiently large nn can learn an arbitrary function, yes. But stockfish is also theoretically perfect given infinite computational resources. What is interesting is performing well under reasonable computational constraints i.e. doing it faster/with fewer flops than stockfish.

Is the model more efficient than Stockfish? I think Stockfish runs on regular CPU computer and I'd guess this " 270M parameter transformer model" requires a GPU but I can't find any reference to efficiency in the paper. Also found in the paper: "While our largest model achieves very good performance, it does not completely close the gap to Stockfish 16". It's actually inferior but they still think it's an interesting…

270M are very tractable even on CPUs.

Re: Grandmaster-Level Chess Without Search

#115

Post by taneq »

YeGoblynQueenne wrote: post #51
paulddraper wrote: post #18

There is rampant misunderstanding of some parts of this article; allow me to help :) The "no-search" chess engine uses search (Stockfish) in in two ways: 1. To score positions in the training data. This is only training data, no search is performed when actually playing. 2. To play moves when the position has many options with a 99% win rate. This is to prevent pathological behavior in already won positions, and is n…

>> 1. To score positions in the training data. This is only training data, no search is performed when actually playing. That's like saying you can have eggs without chickens, because when you make an omelette you don't add chickens. It's completely meaningless and a big fat lie to boot. The truth is that the system created by DeepMind consists of two components: a search-based system used to annotate a dataset of mo…

It's more like saying you can make omelette without killing chickens, even though chickens were clearly involved at some point. So I see your point, that this doesn't allow grandmaster level chess play with no search at any point, but I also think it's fair to say that this approach allows you to use search to build an agent which can play grandmaster-level chess without, itself, using search.

Re: Grandmaster-Level Chess Without Search

#116

Post by taneq »

Lerc wrote: post #67
YeGoblynQueenne wrote: post #51

Earlier quoted context omitted.

>> 1. To score positions in the training data. This is only training data, no search is performed when actually playing. That's like saying you can have eggs without chickens, because when you make an omelette you don't add chickens. It's completely meaningless and a big fat lie to boot. The truth is that the system created by DeepMind consists of two components: a search-based system used to annotate a dataset of mo…

It doesn't matter where the egg came from, just that it is an egg. It could have luckily coalesced from gas (a Boltzmann egg), or perhaps even more radically, been laid by a duck. you say >They didn't train their system by self-play and they certainly didn't hire an army of low-paid workers to annotate moves for them. So you are certainly aware that there are avenues to creating the data set. Given that, it is quite…

Bold of you to assume that low-paid (and yet somehow grandmaster level chess playing) workers have never been exposed to search in any fashion.

Re: Grandmaster-Level Chess Without Search

#117

Post by red75prime »

EGreg wrote: post #111

Next up, claims of ... Text generation without human writers Image generation without human artists Video generation without production crews The rub is that all the training data has been done by lots and lots of search, human writing, human artists, and human production crews. At every step, they curated the results and generated a beautiful latent space that the model was then trained on . The impressive part abou…

I would very much prefer a model trained on human data that had absorbed a part of human values, than a completely de novo intelligence that bootstrapped itself using experimentation in the physical world.

Re: Grandmaster-Level Chess Without Search

#118

Post by valdiorn »

joe_the_user wrote: post #37
paulddraper wrote: post #18

There is rampant misunderstanding of some parts of this article; allow me to help :) The "no-search" chess engine uses search (Stockfish) in in two ways: 1. To score positions in the training data. This is only training data, no search is performed when actually playing. 2. To play moves when the position has many options with a 99% win rate. This is to prevent pathological behavior in already won positions, and is n…

Sort of but it seems a bit of a cheat. Neural networks are universal approximators. Choose a function and get enough data from it and you can approximate it very closely and maybe exactly. If initially creating function F required algorithm Y ("search" or whatever), you can do your approximation to F and then say "Look F without Y" and for all we know, the approximation might be doing things internally that are actua…

Only on the same way that DNA is a cheat for constructing organisms. After all they are universal recipes for organisms. Study enough DNA sequences and splice together a sequence and you can theoretically design any mythical creature you can think of.

But if someone grew an actual dragon in a lab by splicing together DNA fragments, that would still be a major feat. Similarly, training a neural net to play grandmaster level chess is simple in theory but extremely difficult in practice.

Re: Grandmaster-Level Chess Without Search

#119

Post by chaeronanaut »

shmageggy wrote: post #91
ad8e wrote: post #89

A quote from discord: "apparently alpha-zero has been replicated in open source as leela-zero, and then leela-zero got a bunch of improvements so it's far ahead of alpha-zero. but leela-zero was barely mentioned at all in the paper; it was only dismissed in the introduction and not compared in the benchmarks. in the stockfish discord they are saying that leela zero can already do everything in this paper including us…

Yeah, they need to compare against the latest BT2 policy head. It's probably about the same performance.

BT2 is old news, we have BT4 now

Re: Grandmaster-Level Chess Without Search

#120

Post by paulddraper »

YeGoblynQueenne wrote: post #51
paulddraper wrote: post #18

There is rampant misunderstanding of some parts of this article; allow me to help :) The "no-search" chess engine uses search (Stockfish) in in two ways: 1. To score positions in the training data. This is only training data, no search is performed when actually playing. 2. To play moves when the position has many options with a 99% win rate. This is to prevent pathological behavior in already won positions, and is n…

>> 1. To score positions in the training data. This is only training data, no search is performed when actually playing. That's like saying you can have eggs without chickens, because when you make an omelette you don't add chickens. It's completely meaningless and a big fat lie to boot. The truth is that the system created by DeepMind consists of two components: a search-based system used to annotate a dataset of mo…

> That's like saying you can have eggs without chickens, because when you make an omelette you don't add chickens. It's completely meaningless and a big fat lie to boot.

It's like saying ChatGPT isn't a human brain.

It was trained with human brains. But it isn't a human brain.

Post reply on HN