[dead]
Go grandmaster Shin defeats AI KataGo with a two-stone handicap
151–160 of 186 posts
Re: Go grandmaster Shin defeats AI KataGo with a two-stone handicap
#152Earlier quoted context omitted.
Idiomatically a handicap is a disadvantage not an advantage. So if Shin had two extra stones, KataGo had a two stone handicap.
Handicaps can be positive, e.g. in fixed distance racing one way to handicap is to "give" a starting distance to the racers who are slower than the scratch racer; in this system a higher handicap is more of an advantage. Since the AI was playing scratch (the normal, unmodified play style), it seems odd to say it had a handicap. Rather, the human had a positive handicap of two stones.
Youre just mixing up who has the handicap. The handicap gives starting distance to the slower racer, but its still the faster racer that has the handicap.
Re: Go grandmaster Shin defeats AI KataGo with a two-stone handicap
#153Earlier quoted context omitted.
I agree that the drift in chess Elo ratings is slow enough that we can say Bobby Fischer at his peak would still be a strong grandmaster today (if he were given some time to study developments in opening theory). But I don't think we can say whether he would land at 2700 or 2800 in today's Elo scale.
This is within the realm of reasonable opinion, but I'd tentatively say it's an uncommon one--I thought it's generally agreed that the very best players of that era didn't lag contemporary players in terms of skill, only theory. I may be wrong, though.
Chess has developed significantly over the last 50 years. The best players now can be reasonably expected to be much more skilled than the best players of 50 years ago, even if you discount opening theory knowledge.
Re: Go grandmaster Shin defeats AI KataGo with a two-stone handicap
#154Earlier quoted context omitted.
You can't compare Elo ratings over long stretches of time, period. Ratings drift over time, based on the total population of people competing. I think the most accurate way to view Elo ratings is as a measure of skill vs. the average rated player. If you want to compare Magnus Carlsen's peak rating of 2882 in the year 2014 to Garry Kasparov's peak rating of 2851 in the year 1999, you have to know how strong the total…
Even that would be slightly malinformative because of opening theory. If you warped a very strong player from the past to the present, he'd do very poorly at first simply because of advances in opening theory. But give him a bit to catchup and he'd likely have his rating zoom on up. So modern players would do better against the static computer because of the same advantage, but that doesn't mean they're necessarily s…
In Go, we had an extensive documentary on this, based on the strange case of a man possessed by the spirit of an ancient master Go player. I'm not going to spoil the results here; it's an enjoyable series and includes many small introductory tutorials to the game.
Re: Go grandmaster Shin defeats AI KataGo with a two-stone handicap
#155It’s worth understanding that Shin Jinse has been significantly stronger than his nearest human opponents for a while now, more so than Magnus was even at his very peak. In go ELO like scoring he’s something like 120 points over the next strongest player. No other player has ever broken a 3800 rating let alone 3850. Ke Jie (the previous long time champion) peaked at 3755. Shin Jinseo’s strength graph is the most absu…
A small note - it's Elo scoring, not ELO. It's named after Arpad Elo, it's not an acronym of any kind.
Re: Go grandmaster Shin defeats AI KataGo with a two-stone handicap
#156Earlier quoted context omitted.
The distribution of Elo ratings will not tell you how strong the average player is in an absolute sense. Elo ratings measure differences in skill between different players. A player rated 400 points above another player will win 90% of the time. Only rating differences are meaningful. Absolute ratings aren't.
Yes, I can understand why absolute ratings are impossible to extract. However, since the population maintains some continuity over time (players gradually enter and then leave over time), would it not be possible to reconstruct relative ratings between players that didn't play during the same era?
Only if you were to assume that a player's prowess remains constant throughout their career, which we generally know to be false. (I'm completely inventing dates here) If Fisher played Kasparov in 1990 and Kasparov played Carlsen in 2020, you can only compare Carlsen to Fisher if you assume Kasparov's skill was about the same for this entire duration, which no one believes to be the case.
Re: Go grandmaster Shin defeats AI KataGo with a two-stone handicap
#157Earlier quoted context omitted.
Important to note that KataGo was double-handicapped. 20 seconds per move maximum; it couldn’t read deep. Against an amateur, it doesn’t matter, but against a historically strong pro it matters a lot.
...on a 4x 3090 rig. The game ran 299 moves, giving katago 100 minutes if it exhausted time on each move (which must be the optimal strategy under that time control). Shin used about 205 minutes, over twice as much time and of course had leeway to spend more time on difficult moves. Based on the youtube video, it looks like katago was only using 16 seconds per move, is that right? https://www.youtube.com/watch?v=-86z…
I've not looked at the specific game in question, but in general, doubling the thinking time leads to a very notable strength improvement regardless of the base level of play. The increase becomes slightly smaller at higher levels but does not really diminish, and it currently looks like the ceiling is still a bit away.
If it is "only" 4x3090 at 16s, you will definitely get a drastic playing strength boost from doubling the thinking time. This is still clearly within the interval of a linear relationship between thinking time and playing strength, i.e. elo ~ log time. The relationship, to my knowledge, becomes less clear only starting at about 10-20x the number of playouts.
Source: Wrote a paper on this. https://ieeexplore.ieee.org/document/10645535
Re: Go grandmaster Shin defeats AI KataGo with a two-stone handicap
#158Earlier quoted context omitted.
Trouble is, from the existing title it's not clear who had the handicap. So the poster is right to try and fix it.
With a two-stone handicap, Go grandmaster Shin defeats AI KataGo. Or Go grandmaster Shin, playing with a two-stone handicap, defeats AI KataGo.
The problem is that "with a handicap" is in itself ambiguous. People sometimes use it positively, sometimes negatively. That is, you say that Shin had a two-stone handicap, that can in itself be understood as either "Shin was given an advantage of two stones over the AI" (the reality) or "Shin was playing with a disadvantage, giving the AI two extra stones" (the more common reading of "to have a handicap").
Re: Go grandmaster Shin defeats AI KataGo with a two-stone handicap
#159Earlier quoted context omitted.
...on a 4x 3090 rig. The game ran 299 moves, giving katago 100 minutes if it exhausted time on each move (which must be the optimal strategy under that time control). Shin used about 205 minutes, over twice as much time and of course had leeway to spend more time on difficult moves. Based on the youtube video, it looks like katago was only using 16 seconds per move, is that right? https://www.youtube.com/watch?v=-86z…
As another question, does it not operate similarly to the top chess engines? The way the neural network systems work is by using the probabilistic matching paired with a Monte Carlo simulation. So you can get to extreme depth very rapidly. Obviously the breadth is going to be limited, but if the neural network side is well tuned (so high probability hits are indeed generally the most challenging moves), then that's n…
The tree search in KataGo does not use Monte Carlo simulations at all. It is purely policy/value network based, and works insanely well indeed.
Re: Go grandmaster Shin defeats AI KataGo with a two-stone handicap
#160It’s worth understanding that Shin Jinse has been significantly stronger than his nearest human opponents for a while now, more so than Magnus was even at his very peak. In go ELO like scoring he’s something like 120 points over the next strongest player. No other player has ever broken a 3800 rating let alone 3850. Ke Jie (the previous long time champion) peaked at 3755. Shin Jinseo’s strength graph is the most absu…
I don’t know that it’s that shocking, remember Go it’s not solved game, so the the limits of what’s really possible is not known in all cases.
For example, we don’t even know whether perfect White play can possibly overcome two correctly placed Black stones against perfect play.