A recent chess controversy

Whom did he accuse? Kramnik is known to accuse other players. I've never heard this about Nakamura.

>>The researchers note that there’s a problem with this argument, too, as it violates the likelihood principle. This principle tells us the interpretation should only rely on the actual data observed, not the context in which it was collected.

and then in the publication itself:

>>The likelihood principle [Edwards et al., 1963] is a fundamental concept in Bayesian statistics that states that the evidence from an experiment is contained in the likelihood function. It implies that the rules governing when data collection stops are irrelevant to data interpretation. It is entirely appropriate to collect data until a point has been proven or disproven, or until the data collector runs out of time, money, or patience

Surely there is a difference when you look at someone who played 46 games online in his life and scored 45.5 and when you look at someone who played 46000 games and scored 45.5/46 once.

The difference is that Kramnik wasn't "collecting the data" but looked at the whole Nakamura's playing history and found a streak.

Another example would be looking at coinflips and discarding everything before and after you encounter 10 heads in a row to claim you have solid evidence that the coin is biased.

They are misapplying the principle here. If what they wrote was correct then someone claiming: "Look, Nakamure won 100 out of 100 if you just look at games 3, 17, 21, 117...." would be proving Nakamura cheated if they applied methodology from the paper even assuming one in 10000 guilty players. Just because you can choose sampling strategy and stopping rules (what the likelyhood principle states) doesn't mean you can discard data you collected or cherry pick parts that support your hypothesis.

How the data is collected is absolutely relevant and Nakamura is right to point it out.

The vast, overwhelming majority of chess games are not played in front of cameras or even in-person. The accusation in the article was about online play, and specifically blitz which is played online even more commonly than slower formats of chess because moving quickly is easier for many people with a mouse than a physical board.

The way people cheat online is by running a chess engine that analyzes the state of the board in their web browser/app and suggests moves and/or gives a +/- rating reflecting the balance of the game. Sometimes people run it on another device like their phone to evade detection, but the low-effort ways are a browser extension or background app that monitors the screen. The major online chess platforms are constantly/daily banning significant amounts of people trying to cheat in this way.

Chess.com and Lichess catch these cheaters using a variety of methods, some of which are kept secret to make it harder for cheaters to circumvent them. One obvious way is to automatically compare people's moves to the top few engine moves and look for correlations, which is quite effective for, say, catching people who are low-rated but pull out the engine to help them win games occasionally. It's not that good for top-level chess because a Magnus or Hikaru or basically anyone in the top few hundred players can bang out a series of extremely accurate moves in a critical spot - that's why they're top chess players, they're extremely good. Engine analysis can still catch high-level cheaters, but it often takes manual effort to isolate moves that even a world-champion-class human would not have come up with, and offers grounds for suspicion and further investigation rather than certainty.

For titled events and tournaments, Chess.com has what's effectively a custom browser (Proctor) that surveils players during their games, capturing their screen and recording the mics and cameras that Chess.com requires high-level players to make available to show their environment while they play. This is obviously extremely onerous for players, but there's often money on the line and players do not want to play against cheaters either so they largely put up with the inconvenience and privacy loss.

Despite all of the above, high-level online cheating still happens and some of it is likely not caught.

Edit: More information on Proctor here: https://www.chess.com/proctor

> Nakamura responded to Kramnik’s allegations by arguing that focusing on a particular streak while ignoring other games was cherry-picking. The researchers note that there’s a problem with this argument, too, as it violates the likelihood principle. This principle tells us the interpretation should only rely on the actual data observed, not the context in which it was collected.

I don't quite understand this objection? If I won the lottery at odds of 10 million to 1, you'd say that was a very lucky purchase. But if it turned out I bought 10 million tickets, then that context would surely be important for interpreting what happened, even if the odds of that specific ticket winning would be unchanged?

> It's not that good for top-level chess because a Magnus or Hikaru or basically anyone in the top few hundred players can bang out a series of extremely accurate moves in a critical spot - that's why they're top chess players, they're extremely good.

Interesting; I thought I'd read that even the very best players only average ~90% accuracy, whereas the best engines average 99.something%?