Now we have a computer effectively accusing people of fraud (if I understand correctly). How does Irwin address the obvious risks and dangers?
Irwin – the protector of Lichess from all chess players villainous
71–80 of 122 posts
Re: Irwin – the protector of Lichess from all chess players villainous
#72I see a lot of comments about the general topic, but none on the actual tool, which appears to have no (public-facing) updates in about three years. Presumably it could be a tool that incrementally improves over time with the explosion of data lichess has, but surely there would be a single commit since 2019. Do we know that this is what's actually running on lichess's backend today?
https://github.com/lakinwecker/irwin has more recent commits and I think is what actually runs on Lichess, or at least close to it. Not sure much fundamentally changed though. This isn't "the whole of Lichess' anticheat". It's a tool doing one specific thing which it mostly has been doing that way since it was created. There are a bunch of other parts to cheat detection, some of which are much more recent developments. For example kaladin, which has also been linked in the other thread about Lichess. But there's a fair bit more. Understandably though, Lichess doesn't really talk much about all of them, even though pretty much all of it is open source if you know where to look and how to use it.
Re: Irwin – the protector of Lichess from all chess players villainous
#73I see a lot of comments about the general topic, but none on the actual tool, which appears to have no (public-facing) updates in about three years. Presumably it could be a tool that incrementally improves over time with the explosion of data lichess has, but surely there would be a single commit since 2019. Do we know that this is what's actually running on lichess's backend today?
https://github.com/lakinwecker/irwin has more recent commits and I think is what actually runs on Lichess, or at least close to it. Not sure much fundamentally changed though. This isn't "the whole of Lichess' anticheat". It's a tool doing one specific thing which it mostly has been doing that way since it was created. There are a bunch of other parts to cheat detection, some of which are much more recent development…
Thanks for the context - I haven't spent a ton of time looking around but it does seem like there are several moving parts (and evidently some important stuff isn't in the same github org)
Re: Irwin – the protector of Lichess from all chess players villainous
#74Now we have a computer effectively accusing people of fraud (if I understand correctly). How does Irwin address the obvious risks and dangers?
What obvious risks and dangers?
Re: Irwin – the protector of Lichess from all chess players villainous
#75Earlier quoted context omitted.
It's not that simple. I'm an amateur player (2200 at lichess) but there are plenty of situations where I simply _know_ the best variation. Plenty of chess players analyze their own games using an engine and then use their memory when they are confronted with the same position. The same with opening theory: when I'm using my opening preparation, I'm playing at GM level as these are just the moves played by GMs in that…
Your explanation doesn't really distinguish high-level human play (not obvious cheating) from engine play (obvious cheating). There are probably plenty of games in which your first 20 moves (or 5-10 for me) are in GM databases and evaluated favorably by the engine, ass a 2200 you're probably booked up pretty well. But you know computer moves aren't so common in openings; they're much more common in complex middlegame…
My point is that high level human play online can be caused by computer analysis offline. One cannot observe a difference, as the moves are exactly the same.
Re: Irwin – the protector of Lichess from all chess players villainous
#76Earlier quoted context omitted.
Your explanation doesn't really distinguish high-level human play (not obvious cheating) from engine play (obvious cheating). There are probably plenty of games in which your first 20 moves (or 5-10 for me) are in GM databases and evaluated favorably by the engine, ass a 2200 you're probably booked up pretty well. But you know computer moves aren't so common in openings; they're much more common in complex middlegame…
My point is that high level human play online can be caused by computer analysis offline. One cannot observe a difference, as the moves are exactly the same.
But for the vast majority of players there will be a difference in computer lines and human evaluation; even if you're playing a strong game there is still a world of difference between 2500 lichess and ~3300 FIDE stockfish. This is more true for the median ~1500 lichess player. Even if you put two GMs up against each other and give one a computer, some portion of games would include an obvious computer line that a ~2800 FIDE human wouldn't evaluate the same way as an engine.
Re: Irwin – the protector of Lichess from all chess players villainous
#77Recent and related: Lichess: The free and open source chess server - https://news.ycombinator.com/item?id=32045763 - July 2022 (65 comments)
It takes $420k per year to run Lichess -
https://news.ycombinator.com/item?id=29955204 - February 2022 (266 comments)
Re: Irwin – the protector of Lichess from all chess players villainous
#78Earlier quoted context omitted.
High level humans and chess engines play differently. You can see commentary on chess YouTube videos when they run across cheaters. Also the chess engines people use are accessible, you can compare what a suspected cheater does with what the cheater does and if they’re exactly the same consistently, you have a pretty strong signal. One of the bigger tells are strange moves that set up a many move series resulting in…
It's not that simple. I'm an amateur player (2200 at lichess) but there are plenty of situations where I simply _know_ the best variation. Plenty of chess players analyze their own games using an engine and then use their memory when they are confronted with the same position. The same with opening theory: when I'm using my opening preparation, I'm playing at GM level as these are just the moves played by GMs in that…
This is a very strong claim that is almost certainly false.
Would you be willing to reveal your account so that it can be independently verified? For example I'm 2116 on lichess and looking over the last 10 opponents who are in the neighborhood of 2200, it is never the case that their moves are optimal compared to a chess engine. For the first 10 or so moves yeah sure, just play a book opening, but beyond that people at 2200 make plenty of mistakes every single game including blunders.
The idea that you can consistently make optimal moves over the course of a 30-40 move game beyond the book opening requires some kind of evidence because in examining the last 10 games of 10 accounts arbitrarily picked, there isn't a single one that isn't absolutely full of inaccuracies and mistakes.
Re: Irwin – the protector of Lichess from all chess players villainous
#79I’ve seen a video of Magnus Carlsen playing on Lichess before. Does Irwin ever accidentally flag him or people like him? Do these sorts of folks have to be verified in some fashion?
What it looks like to cheat is more than just playing relatively accurate moves. Average move time, centipawn loss over multiple games, blunder/mistake frequency across multiple games, strengh of moves while in time trouble, etc. Cheaters tend to stand out when you look at a short history of games.
That would clearly work for someone who is cheating by letting an engine do all the work, but how about someone who mostly plays the game themselves just using the engine rarely?
Give Nakamura a minute with Stockfish any two times of his choosing in each game, and he would have probably won the Candidates.
Heck, just give a good player a blunder alert that tells them after they have made a blunder that they have done so and it could make a big difference.
There were games in the Candidates where a player would make a blunder that would completely turn the game around if the opponent found the one move that exploited it, but the opponent didn't see it. The first player could have then saved themselves but had not yet realized they blundered so didn't. Then the other player realized what was going on and exploited the blunder.
Re: Irwin – the protector of Lichess from all chess players villainous
#80Earlier quoted context omitted.
High level humans and chess engines play differently. You can see commentary on chess YouTube videos when they run across cheaters. Also the chess engines people use are accessible, you can compare what a suspected cheater does with what the cheater does and if they’re exactly the same consistently, you have a pretty strong signal. One of the bigger tells are strange moves that set up a many move series resulting in…
> High level humans and chess engines play differently. You can see commentary on chess YouTube videos when they run across cheaters. That's similar to people describing how to catch other frauds, such as fake Amazon comments or bots. It's medieval 'science': They usually have no evidence of their accuracy, either false negatives (frauds who they overlook) and false positives (people falsely accused of fraud). So it'…
If it makes you feel better (or worse), signal "fingerprinting" is used in laboratory science to verify things like purity and identity. Detecting a lack of divergence from a known chess bot seems like a good fingerprint to me.