(Side note: what the hell is that website trying to do to my computer? It's trying to open so many connections on weird ports!)
Looks like they're using a bunch of websockets for something. Probably that live player count up in the navigation.
Strongest chess player, ever
51–60 of 113 posts
Re: Strongest chess player, ever
#52(Side note: what the hell is that website trying to do to my computer? It's trying to open so many connections on weird ports!)
lichess opens only one websocket connection, on a port between 9021 to 9029. That port range prevents a weird behavior on firefox when the connection is interrupted.
Re: Strongest chess player, ever
#53This is extremely impressive - it's cool that talented programmers are pushing the limits of computer science to advance the state of the art of chess engines.
However, I also wish that there were comparable efforts to create AIs that train humans. Basically, figure out a way to systematically, efficiently, and scalably train amateurs into masters. That IMO would be absolutely amazing (and something I'd gladly pay for).
Re: Strongest chess player, ever
#54What I would love to see is work on chess engines having better strength levels for amateurs. For me at least there is a line where everything below a certain level I can beat 95 percent of the time and everything at or above that level I lose to 95 percent of the time.
Pole Position mode. The AI quality ramps up when losing and down when winning.
Re: Strongest chess player, ever
#55> During the final event, after playing 64 games against Komodo, Stockfish won with the score of 35½-28½. No doubt is further allowed: Stockfish is the best chess player ever! From a statistical point of view, this isn't actually significant, despite the fact that draws help reduce the variance. 45 of those games are draws, leaving a 13-6 score in favor of Stockfish. Considering a null hypothesis of a binomial distri…
But it fundamentally _isn't_ binomial across 19 games, because of draws. You can't just ignore draws from the analysis, to do so is terrible application of statistics.
The first analysis isn't terrible. It gets the important points right (that the draws are not evidence of difference in skill) and moves on to the remaining evidence (difference in wins to losses).
The draws are at best evidence towards equality (not against it). Allow them to vary and the likelihood of seeing a difference of 9 wins in 64 games with 45 draws moves up to 0.13 or 13% (when we assume the two players are identical, an appropriate null hypothesis) (even less significant). So in about one tournament in 8 you would expect this much of a lead, even if it was one algorithm playing itself. So from one tournament we say it is likely the one algorithm is in fact better, but it doesn't rise to the standard of being statistically significant.
# R code to empirically estimate two-sided probablity of # seeing a lead of 9 games when 64 games are played # and the assumed probability of a draw is 45/64 # with the null assumption win/loss odds are equal simulate wldiff set.seed(350920) stats print(sum(stats>=13-6)/length(stats)) ## [1] 0.1341
} sample(c('w','d','l'),size=nplay,replace=TRUE,
prob=c((nplay-ndraw)/2,ndraw,(nplay-ndraw)/2)/nplay)
(it is weird that somebody, not me, created a throw-away account to make the original comment. likely they are involved in chess development, or know how quickly stat discussions go sideways)
Re: Strongest chess player, ever
#56Stockfish 5 was released yesterday: http://stockfishchess.org/
As mentioned elsewhere in this thread, Stockfish is just an engine - you must install a GUI separately. XBoard is well known, but there are better alternatives:
Re: Strongest chess player, ever
#57> During the final event, after playing 64 games against Komodo, Stockfish won with the score of 35½-28½. No doubt is further allowed: Stockfish is the best chess player ever! From a statistical point of view, this isn't actually significant, despite the fact that draws help reduce the variance. 45 of those games are draws, leaving a 13-6 score in favor of Stockfish. Considering a null hypothesis of a binomial distri…
But it fundamentally _isn't_ binomial across 19 games, because of draws. You can't just ignore draws from the analysis, to do so is terrible application of statistics.
Once you condition on the number of draws, you do get that binomial distribution.
Suppose you have a coin, which gives a random outcome X. But you can only observe the outcome of X when another independent binary random variable Y is true. How can you tell if X is biased? Since X and Y are independent, the observations where Y is false are irrelevant since they don't tell you anything about X. So you just keep the observations where Y is true, and from there you can apply a binomial statistical test to the observations of X.
[ In case you're wondering whether applying statistical tests to variable sample sizes is valid, the answer is yes: a p-value is a uniform random variable from the set of observables (augmented by a continuous random variable, since our set of observables is discrete) to [0,1]. Our p-value is a mixture of p-values on smaller sample sizes, so it is still uniform. ]
This is exactly what happens here: consider a random outcome {win,lose,draw}. If you don't have a draw, let Y be true and X be the outcome of the game. If you have a draw, let Y be false and X be a random coin with the same distribution as for non-drawn games. Then X and Y are independent random variables and the above applies.
Informally: draws are not useful information in determining whether there are more wins than losses.
Re: Strongest chess player, ever
#58Earlier quoted context omitted.
Stockfish has been available for quite a while, and over the last few years has been recognised as being the strongest open-source chess engine. It's fantastic to see it take on the best commercial engines toe-to-toe and come out on top. These ratings are computer ratings, and because they are playing very much isolates (computers vs other computers), without enough external human encounters, the ratings are consiste…
It sounds like you think humans can still beat top computers? According to my research the last time a human beat a top computer was 2005. Almost a decade ago!
Computer software don't play that many human tournaments. It's a rare sight, even when they were clearly weaker. There's a lack of data, so the result is inconclusive.
Computer software doesn't play chess. It doesn't understand positions. It has a bunch of algorithms and processes that turn a board position, tries each candidate move, turns the resulting position into some sort of number, through an evaluation function. Goes down a tree of that until either it reaches a definite conclusion, or reaches a depth where trying to go deeper takes far too much more time.
All of this is an artificial simulation of how chess is played. And it is, long-term, at the mercy of the accuracy of the evaluation function. Because right at the end of it's search-depth, it has to evaluate that position, effectively "a guess".
The most accurate way of evaluating a position is to try each move and go down a search tree of best moves until you reach a definite conclusion. But the evaluation is used because at that point it is computationally too expensive to go down the move tree any deeper. It's a fudge of brute force analysis - there isn't enough computational resources available to go any deeper, and so the computer must guess. This is the horizon effect, computers can't see past it.
That evaluation function is a human written piece of code, that takes various on-board factors: piece placement, pawn structures, strong/weak-squares, king-safety, piece activity, central control. In effect, the human is trying to program intuition into the machine. Humans don't understand intuition, let alone program a computer to do it.
Chess strength of software is determined substantially by the number and speed of CPUs, Memory capacity, and the evaluation function. The evaluation function is the weakest part of that chain, so there's considerable effort to hold off the use of the evaluation function for as long as possible.
I think only the Rybka developers spent more time trying to teach the engine how to play chess, and that proved more successful for a few years than the "more power quicker" led industry.
The evaluation function is the most difficult part of a chess engine. It is the typical AI problem. And humans can only take that so far. But how do you find a human who can comprehend how a grandmaster thinks about a chess position and determining the right move, and still be capable of emulating that process in software. At least Rybka's developers were International Masters.
Also, humans have the strength to adapt and refine - patch their own chess playing abilities. Look at Carlsen's style, it's an ever more refined Karpov style, which itself was a more refined Capablanca style. Carlsen excels in the kinds of positions computers don't manage well - deep strategical long-range plans, well executed. A lot of Carlsen's chess strength is intuition and feel, with an impeccable analysis to confirm his hypothesis. And he's one of the post-Chessbase crowd, grown up learning chess with computers. That is an opponent a computer should fear, if it could ever comprehend the notion.
So yes, in terms of chess playing strength, humans still play chess better than computers.
Re: Strongest chess player, ever
#59Earlier quoted context omitted.
But it fundamentally _isn't_ binomial across 19 games, because of draws. You can't just ignore draws from the analysis, to do so is terrible application of statistics.
The first analysis isn't terrible. It gets the important points right (that the draws are not evidence of difference in skill) and moves on to the remaining evidence (difference in wins to losses). The draws are at best evidence towards equality (not against it). Allow them to vary and the likelihood of seeing a difference of 9 wins in 64 games with 45 draws moves up to 0.13 or 13% (when we assume the two players are…
(it is weird that somebody, not me, created a throw-away account to make the original comment. likely they are involved in chess development, or know how quickly stat discussions go sideways)
I'm not involved in chess, I just don't like long-term accounts.
You get a slightly different p-value because the ordering you chose is slightly different from mine. Compared to mine, it favors matchups where the draw probability is low.
Re: Strongest chess player, ever
#60This is extremely impressive - it's cool that talented programmers are pushing the limits of computer science to advance the state of the art of chess engines. However, I also wish that there were comparable efforts to create AIs that train humans. Basically, figure out a way to systematically, efficiently, and scalably train amateurs into masters. That IMO would be absolutely amazing (and something I'd gladly pay fo…
Are you sure you wouldn't rather learn, say, mathematical physics or computer science?
The neurological topiary for chess is strictly harder, because chess has fewer analogues in day-to-day experience.