I love this.
I watched every game years ago between Lee Sedol, even though it was late at night. I love AI and I love Go.
Off topic, but I wrote the first commercial Go program for the Apple II in the late 1970s.
Go grandmaster Shin defeats AI KataGo with a two-stone handicap
161–170 of 184 posts
Re: Go grandmaster Shin defeats AI KataGo with a two-stone handicap
#162Earlier quoted context omitted.
...on a 4x 3090 rig. The game ran 299 moves, giving katago 100 minutes if it exhausted time on each move (which must be the optimal strategy under that time control). Shin used about 205 minutes, over twice as much time and of course had leeway to spend more time on difficult moves. Based on the youtube video, it looks like katago was only using 16 seconds per move, is that right? https://www.youtube.com/watch?v=-86z…
Considering the original AlphaGo ran on a full Google TPU rack, and here KataGo seems to run on a $10k computer, and won with 2 stones handicap, this is showing just how much advance there was in computer go
AlphaGo is a bit difficult to compare because their parallelism efficiency was abysmal. You might as well consider it a regular 8x or so GPU machine.
Re: Go grandmaster Shin defeats AI KataGo with a two-stone handicap
#163It’s worth understanding that Shin Jinse has been significantly stronger than his nearest human opponents for a while now, more so than Magnus was even at his very peak. In go ELO like scoring he’s something like 120 points over the next strongest player. No other player has ever broken a 3800 rating let alone 3850. Ke Jie (the previous long time champion) peaked at 3755. Shin Jinseo’s strength graph is the most absu…
This was only running on 4x3090s man. The gap is far, far wider than you think. 8 5090s or 6000s and at least as many extra dedicated to doing nothing but running disgusting amounts of monte carlo in parallel from anything resembling a good choice (fuck it check some bad ones too) would lay him to waste. I am going to go out on a limb and say part of it is a matter of respect, and part "lets not discourage and scare…
If I remember the numbers correctly, your setup seems to be wrong if you only get 1.4k e/s with 4x 3090. What network did you use?
Re: Go grandmaster Shin defeats AI KataGo with a two-stone handicap
#164It’s worth understanding that Shin Jinse has been significantly stronger than his nearest human opponents for a while now, more so than Magnus was even at his very peak. In go ELO like scoring he’s something like 120 points over the next strongest player. No other player has ever broken a 3800 rating let alone 3850. Ke Jie (the previous long time champion) peaked at 3755. Shin Jinseo’s strength graph is the most absu…
A small note - it's Elo scoring, not ELO. It's named after Arpad Elo, it's not an acronym of any kind.
The consequences of League of Legends. The brain trust over there use ELO and its caused everyone to think Elo is an acronym with the wrong pronunciation.
Re: Go grandmaster Shin defeats AI KataGo with a two-stone handicap
#165It’s worth understanding that Shin Jinse has been significantly stronger than his nearest human opponents for a while now, more so than Magnus was even at his very peak. In go ELO like scoring he’s something like 120 points over the next strongest player. No other player has ever broken a 3800 rating let alone 3850. Ke Jie (the previous long time champion) peaked at 3755. Shin Jinseo’s strength graph is the most absu…
For another comparison, top world class chess players will have solid odds to beat Leela Chess Zero when given a knight odds handicap (Leela Chess Zero starts with 1 fewer knight). For human vs human, I think this would be somewhere in the ballpark of the ~10,000th best chess player having fair odds against Magnus. I wonder if this means the best Go play is closer to theoretically perfect play or if it just happened…
I'm not sure if I'm allowed to name the source on this here, but someone very familiar with the top Go program scene explained that there is a fair chance that a perfect game has been played already. This is based on the probability of a mistake per move in relation to the number of games there were played by top programs. Hard to believe at first, but perchance it is true. The problem is, of course, that we don't know which games are perfect.
Re: Go grandmaster Shin defeats AI KataGo with a two-stone handicap
#166Earlier quoted context omitted.
For a non-Go player, do you think this trend will persist, or is it more of a dead-cat/human bounce?
Could someone sufficiently motivated invest in training Katago to be able to beat Shin Jinseo with 3 stones of handicap? Unfortunately - probably yes. This in no way detracts from how absurd and remarkable it is that Shin Jinseo can beat KataGo (it gets a LOT of training and architecture refinements https://katagotraining.org/#eloGraphButtons ) with 2 stones of handicap.
Depends on where you put the perfect play ceiling. I think each side has good points in their respective favor. There have been strong pushes by both humans and computers in improving their best play in the past few years.
A 2 stone handicap, however, does not scale linearly with strength. It becomes relatively more impactful at higher levels. For pro players, 2 stones are gigantic, and for beginner players, they make zero difference. Same for a point advantage (komi adjustment). So there might come a point where it is physically impossible for a computer to beat a top human under some handicap.
Re: Go grandmaster Shin defeats AI KataGo with a two-stone handicap
#167Earlier quoted context omitted.
I read this comment before looking at the article and thought that the grandmaster beat the AI even giving the AI 2 stones. Too bad. But this way around is of course more realistic. And of course you would need to take into consideration the scale of go ratings and chess ratings when making that comparison. With top chess ratings being around 2800, being 1000 less than the top go ratings, one would have to apply a fa…
If you scaled Shin Jinseo to 2800, you would have players with extremely negative ratings. This page shows ratings of European players on a roughly aligned scale: https://europeangodatabase.eu/EGD/createalleuro3.php?country... . It still has negative numbers on it, and this only contains players who have attended a tournament (though it's more common for beginners to play tournaments in the west, since it's hard to f…
This is a bit of a digression, but: It's an interesting question what (if anything) that larger dynamic range means.
One thing you'll hear people say sometimes -- I've said it myself -- is that this shows that in some sense go is a "deeper" game than chess; there's more to know and understand, more variety of possible human skill.
That might well be true. It certainly feels a more elegant game, and involves longer tactical sequences, and so forth. But this may be misleading.
Consider the game of treblechess. To play a game of treblechess, you play three games of ordinary chess and look at the overall result.
Suppose that when we play ordinary chess, I win with probability W, lose with probability L, and draw with probability D = 1-(W+L). And suppose separate games are independent of one another (which might not be true in reality, but never mind). What happens when we play treblechess?
I win 3/0 with probability W^3. I win 2.5/0.5 with probability 3W^2D, because that happens when I draw any one of our three games and win both of the other two. I win 2/1 with probability 3(W^2L+WD^2), because that happens when I win two and lose one or win one and draw two, and for each of those there are three choices for which game is which. So I win at treblechess with probability W^3 + 3(W^2(1-W)+WD^2).
I can draw by getting one each of WDL (probability 6WDL) or by drawing all three (probability D^3).
Suppose that when we play chess I win 40% of the time, draw 50% of the time, and lose 10% of the time. Then our Elo difference is about 107 points. In triplechess, I will win 65.2% of the time, draw 24.5% of the time, and lose 10.3% of the time. Our Elo difference is about 214 points.
If in ordinary chess I win 65% of the time, draw 25% of the time and lose 10% of the time -- about the same odds as for treblechess in the last example -- then our chess Elo difference is about 215 points. At treblechess I will win 84% of the time, draw 11% of the time, and lose 5% of the time, and our Elo difference will be about 375 points.
If in ordinary chess I win 15%, draw 75%, lose 10%, then our Elo difference is about 17 points; in treblechess I will win 31%, draw 49%, lose 20% and our Elo difference will be about 41 points.
Treblechess Elo differences are on the order of double ordinary chess Elo differences! Clearly treblechess is a game with twice the depth of ordinary chess!
But it isn't. It's just longer and gives more opportunities for the better player to come out ahead overall.
Go is also a longer game than chess, though of course not in the same way as treblechess is. Perhaps the larger Elo range of go is more because of that than it is because of actual deeper strategy and tactics?
Re: Go grandmaster Shin defeats AI KataGo with a two-stone handicap
#168KataGo isn't very good at exploiting weaker opponents. In chess people were convinced a grandmaster can never be beaten with a knight odds. It's just too easy to simplify the position and win. It was very easy (for a grandmaster) vs already super human Stockfish. It was still kinda easy (for a strong GM) vs 200+ ELO stronger NNUE Stockfish. And then someone made a net optimized for exploiting humans. Its games are am…
I follow some chess channels on YouTube (among them the usual suspects, Gotham, Chessbrah, Eric Rosen) but do you know of any video content that does game reviews of what you describe?
You need to add Chess-Network to your usual suspects.
Re: Go grandmaster Shin defeats AI KataGo with a two-stone handicap
#169Earlier quoted context omitted.
> It's never made sense to me that so many go players study AI go play in the hope of emulating it in a human game. > We're not machines. We can't do thousands of Monte Carlo tree searches per second. You sound so sure of that, but I have seen people catch a ball, and I am not so sure that any artificial person should be any more aware of the tremendous maths they are "solving" From the perspective of a game with far…
> but I have seen people catch a ball from what i remember, dogs sometimes run in curves so that the perceived trajectory of the object they try to catch is more linear so not everything might be in-brain math but also good trickery
What exactly do you think the difference is?
If you have an LLM add 2+2, it is doing a tremendous number of additions and multiplications it doesn't have to, same as us, and I am not sure the LLM can answer any more questions about that process than we can about ourselves.
Re: Go grandmaster Shin defeats AI KataGo with a two-stone handicap
#170Earlier quoted context omitted.
If you scaled Shin Jinseo to 2800, you would have players with extremely negative ratings. This page shows ratings of European players on a roughly aligned scale: https://europeangodatabase.eu/EGD/createalleuro3.php?country... . It still has negative numbers on it, and this only contains players who have attended a tournament (though it's more common for beginners to play tournaments in the west, since it's hard to f…
This is a bit of a digression, but: It's an interesting question what (if anything) that larger dynamic range means . One thing you'll hear people say sometimes -- I've said it myself -- is that this shows that in some sense go is a "deeper" game than chess; there's more to know and understand, more variety of possible human skill. That might well be true. It certainly feels a more elegant game, and involves longer t…
As an amateur (but competent) player of both, when I play go it almost feels like I'm playing multiple smaller games at the same time on the same board. Even if I might be struggling in one area, I can make it up in another. So it's very hard for a lesser opponent to beat me, even if they do gain an advantage in one area. Likewise in reverse when I'm playing a superior opponent. I might get in a nice kill, but next thing I know we're playing a different battle on the other side of the board and I get crushed.
So I think there's truth to what you're saying.