But it fundamentally _isn't_ binomial across 19 games, because of draws. You can't just ignore draws from the analysis, to do so is terrible application of statistics.
Once you condition on the number of draws, you do get that binomial distribution. Suppose you have a coin, which gives a random outcome X. But you can only observe the outcome of X when another independent binary random variable Y is true. How can you tell if X is biased? Since X and Y are independent, the observations where Y is false are irrelevant since they don't tell you anything about X. So you just keep the ob…
I'm not sure that discounting draws is the right thing to do either. For example, Petrosian was not the strongest attacking player but was very, very tough to beat.
This also calls into the question the notion of "strongest chess player." Who is strongest, the flashy attacking player that wins half the time and loses the other half, or the stonewall that poses little threat but that you can never beat?
This is extremely impressive - it's cool that talented programmers are pushing the limits of computer science to advance the state of the art of chess engines. However, I also wish that there were comparable efforts to create AIs that train humans. Basically, figure out a way to systematically, efficiently, and scalably train amateurs into masters. That IMO would be absolutely amazing (and something I'd gladly pay fo…
In my opinion, this would be a lot more difficult. I am slightly naive as to exactly how stockfish works, but: Computers can do a lot of accurate brute-forcing; humans must see the position in a more holistic, intuitive way. Excellent human players and excellent computer players are presumably doing completely different calculation tasks. I would suggest that computers are still bad at approaching the task in a human…
>Stockfish might be able to tell you what it was doing, but not in a way that it would be reasonable for a human to follow.
Looking at the code posted, if the sub-scores were stored in an array and only added at the end, it would be possible to compare the positions after two moves by sub-score and find the biggest differences between subscores. Then you could say that position A is better than position B because it avoid doubled pawns, or has better bishops, etc.
> but top chess players generally consider computers to be excellent calculators but dumb in terms of general strategy. Is computer assisted chess a thing? Perhaps with standardized hardware, but any software. In chess I'm good at strategy but terrible at calculating and I miss obvious stuff all the time in my fight for strategy. I always thought I'd do great with computer assistance to look for the obvious stuff, an…
Computer assisted chess is becoming more popular in human-only tournaments. In that context it's usually called cheating, of course. Ken Regan has developed some interesting statistical methods to detect it, see for example his blog post on Gödel's Lost Letter and P=NP: http://rjlipton.wordpress.com/2013/07/27/thirteen-sigma/
What a coincidence, just yesterday The United States Chess Federation published an article about Ken Regan and his methods for detecting cheating.
I'm rather surprised at how relatively simple and small the codebase is: https://github.com/mcostalba/Stockfish/tree/master/src
Especially interesting to see everything hardcoded into C++. Quite different from most modern codebases that use a mix of languages, configuration files, etc.
Wow, very cool! Can someone explain what's going on at 20:24 : http://youtu.be/7YWYS209ydE?t=20m20s ? He forks the king and the queen, then his opponent moved his queen!?! This is not a legal move right?
Had to re-watch a couple times: on that move, white had already moved pawn from g2 to g3, the knight from d6 to e8 is a pre-move that happens very quickly (you can see the red square); while white was setting up that pre-move, black moved the queen. Does that clarify or did I miss what you are asking?
It sounds like you think humans can still beat top computers? According to my research the last time a human beat a top computer was 2005. Almost a decade ago!
Computer software don't play that many human tournaments. It's a rare sight, even when they were clearly weaker. There's a lack of data, so the result is inconclusive. Computer software doesn't play chess. It doesn't understand positions. It has a bunch of algorithms and processes that turn a board position, tries each candidate move, turns the resulting position into some sort of number, through an evaluation functi…
> So yes, in terms of chess playing strength, humans still play chess better than computers.
... but humans can't beat top computers any more? You're not making any sense here. The fact is no human has beat a top computer in almost 10 years. Not even once! Claiming that humans are still better is just nonsense.
This is extremely impressive - it's cool that talented programmers are pushing the limits of computer science to advance the state of the art of chess engines. However, I also wish that there were comparable efforts to create AIs that train humans. Basically, figure out a way to systematically, efficiently, and scalably train amateurs into masters. That IMO would be absolutely amazing (and something I'd gladly pay fo…
> a way to systematically, efficiently, and scalably train amateurs into masters. Why do you need an AI? I don't play chess, but I suppose the above is more or less what an elite chess school provides, and you could likely reproduce it with books + practice + private lessons. That is, what Ericson calls deliberate practice and coaching.
By far, the fastest way to improve at chess is with a coach. Books work in the beginning, but soon you are crawling around in the dark. You can't identify your weaknesses, so you can't correct them. After studying the wrong thing for a year, you fix one of your weaknesses by accident, and you improve. A coach bypasses all of that wasted time. The challenge is not to automate a chess curriculum. That already exists on many websites selling chess software. Those are useful to learn certain theory and burn it into your brain by drilling over and over. The challenge is to create a chess teacher that can identify your specific weaknesses and correct them. A middle ground approach might work well, where you take one mental model of chess and develop a program to train that specific mental model. For example there is the Nimzowitsch model where chess is seen as siege warfare, with specific meta-strategies. If that model fits with how you think, then great. But it doesn't fit everyone. One day this super efficient learning will be reality. It sounds kind of boring. With an exponential game like chess, everyone will be at about the same level, except a few who throw their life away chasing n+1 while everyone else settled for n and having a life.
Can anyone comment on what makes stockfish different from other chess engines? If I'm curious about state of the art in computer chess, is it worthwhile to study its source? What interesting ideas should I expect to see there beyond what I vaguely know to be the standard approach from introductory AI courses, i.e. some sort of alpha-beta pruning search?
I have not seen the Stockfish source, but my impression is that computer chess was going round in circles in the last 10 years, and it's still good old alpha-beta search with more refined heuristics. The biggest value in studying computer chess for a programmer is IMO in seeing all the different performance optimization tricks.
Performance optimization, while it is not neglected, is not what separates chess engines. You don't win an exponential race with linear advantages.
I always thought of the heuristics for evaluating a chess position as the really hard part of building a chess engine; i.e. how do you capture all of the positional subtleties in a number to feed into minimax? But looking at the source, it's not really that complicated [1]. Can someone who knows more than me comment on that? Is it that the innovations are elsewhere? That good chess really can be boiled down to [1] ht…
What separates one chess engine from another is its ability to efficiently navigate and evaluate the game tree. Stockfish effectively discards probably 95% or more of the nodes in any position. Easier said than done, better not discard the best move in that 95%. Evaluation is relatively simple, a fancy lookup table. They say love covers a multitude of sins. Well, searching one move deeper than your opponent covers a multitude of static positional evaluation mistakes. Searching deeper can figure out the complexity of the position better than anything that can be statically evaluated. That's where virtually all improvements exist today, figuring out which moves can be safely discarded.
I bet it would be unnerving for someone like Kasparov or Magnus Carlsen to play this program, where it would have 1 minute on the clock and they could have the whole day. It would make many of the moves in under a second and they'd be better than the grandmasters' moves!
It wouldn't surprise me if the top players regularly played against the best algorithms as part of their training.
Olympic sprinters do not practice racing against race cars. Maybe for a publicity stunt. But not for improvement.