Saw the AlphaGo movie at a festival recently. Been following the AlphaGo Zero developments, which leap-frog what was going on in the movie (although still very much worth seeing). One thing I was curious about is if Go would be considered solved, either hard or weakly solved, since AlphaGo Zero at this point doesn't seem to be able to be beat by any living human. Wikipedia does not list it as solved in either sense,…
Go still has not been Ultra-weakly solved (eg. we do not know who is supposed to win).
If AlphaGo has weakly solved go, then it should either have a 100% win rate when playing against itself as white, or a 100% win rate when playing against itself as black.
Interestingly, the idea behind temporal difference learning is more or less the intuition behind how people price derivatives in finance. The expected value of a contract at time T, estimated at some time t expected estimate, made n months from now, of the value [12-n] months later, will also be $100 . This allows you to shrink the state space considerably. You can usually work out the payoff of a derivatives in diff…
I think the bellman equation (which is used extensively in reinforcement learning) is also taught in stochastic calculus for finance (except in the continuous form?). https://en.wikipedia.org/wiki/Hamilton%E2%80%93Jacobi%E2%80%... My memory is hazy so there might not be a real connection here.
Yup, and a lot more! The Hamilton-Jacobi-Bellman equations come up in anything that can be formulated as an optimal control problem.
I wonder how AlphaGo Zero would fare against the others if they were all using the same search algorithm, and I wonder how the search depth vs breadth changes in Zero compared to earlier variants.
The main reason AlphaGo Zero learns so much faster than its predecessors is because it uses temporal-difference learning.[1] This effectively removes a huge amount of the value network's state space for the learning algorithm to search through, since it bakes in the assumption that a move's value ought to equal that of the best available move in the following board position, which is exactly what you'd expect for a g…
Interesting, a TD algorithm, developed by a Canadian AI researcher now working with Deepmind in the early 1990s, was previously used to beat expert players at Backgammon and advanced human understanding of the game: > TD-Lambda is a learning algorithm invented by Richard S. Sutton based on earlier work on temporal difference learning by Arthur Samuel. This algorithm was famously applied by Gerald Tesauro to create TD…
TD-Gammon was taught to us as a part of a classroom course on Reinforcement Learning (RL) in 2007. ML was known to a small set of people back then, there weren't many jobs in the area (this is in India), and even to many in this set, RL was either not known or not well known. It's interesting to see RL surge in popularity. In fact just a couple of weeks back, I was talking to the professor who taught us that course, and it was fun comparing ML/RL related awareness then to now :-)
The main reason AlphaGo Zero learns so much faster than its predecessors is because it uses temporal-difference learning.[1] This effectively removes a huge amount of the value network's state space for the learning algorithm to search through, since it bakes in the assumption that a move's value ought to equal that of the best available move in the following board position, which is exactly what you'd expect for a g…
Interestingly, the idea behind temporal difference learning is more or less the intuition behind how people price derivatives in finance. The expected value of a contract at time T, estimated at some time t expected estimate, made n months from now, of the value [12-n] months later, will also be $100 . This allows you to shrink the state space considerably. You can usually work out the payoff of a derivatives in diff…
Or in the parlance of probability theory: "The expectation of the posterior probability, after viewing the evidence, must equal the prior probability."
Would be really cool to see a generic framework for this, where you can plug in the rules of your discrete-deterministic-game-with-perfect-information and get a superhuman bot. Does something like this already exist?
Saw the AlphaGo movie at a festival recently. Been following the AlphaGo Zero developments, which leap-frog what was going on in the movie (although still very much worth seeing). One thing I was curious about is if Go would be considered solved, either hard or weakly solved, since AlphaGo Zero at this point doesn't seem to be able to be beat by any living human. Wikipedia does not list it as solved in either sense,…
"On a long enough timeline, everything is a discrete game." (With apologies to _Fight Club_) Personally, I look forward to the day when the software I own works for me to the extent of optimizing the decisions I make during the day, even many mundane ones. Properly executed, such a system could make a big difference in my quality of life. I believe that a big piece that is missing is a solid life-model, a life-repres…
I don't know why you're downvoted. It's an interesting thing to think about. Personally, I'd be eager to try it out for a year or so. I set the goal, my computer tells me what to do.
At least initial reports are that alphaGo Zero is more human-like than Master. Zero packs even more of the inhuman ability to pick the most critical part of the board for each move, but less weird looking stuff. In fact, one of the obvious differences between AlphaGo Zero and top human players, is much more play on safe opening spots, which has been out of fashion among human pros for a hundred years or so.
>At least initial reports are that alphaGo Zero is more human-like than Master. Empirically, this is not correct. The original AlphaGo achieved a 57% accuracy at predicting expert moves. I can't find an exact number, but based on the graph in the nature article, AlphaGo Zero has a less than 50% accuracy at predicting human moves. Eyeballing the graph, it looks like the supervized learning variant of AlphaGo Zero scor…
I'd have thought the metric to watch was "% of moves made which seemed kooky to human experts", not "% of human experts moves with which the program agrees."