This is pretty incredible, especially the power dissipation results. Only 4 TPUs? Humans are toast.
What is a TPU?
AlphaGo Zero: Learning from scratch
91–100 of 324 posts
Re: AlphaGo Zero: Learning from scratch
#92I would be willing to spend an entire lifetime to perfectly understand how this algorithm works. Currently I can barely write Djikstra's algorithm.
You can watch the RL course given by one of the inventors of AlphaGo, David Silver.
https://www.youtube.com/playlist?list=PLzuuYNsE1EZAXYR4FJ75j...
Re: AlphaGo Zero: Learning from scratch
#93How I wish Marvin Minsky would have stayed alive for one more year and seen this. He would have been so happy!
I think he was cryopreserved, so he surely will be surprised once they wake him up in the future, assuming cryonics really works.
Re: AlphaGo Zero: Learning from scratch
#94Re: AlphaGo Zero: Learning from scratch
#95I'm reminded of Eliezer Yudkowski's article "There is no fire Alarm for Artificial General Intelligence." Is this smoke? https://intelligence.org/2017/10/13/fire-alarm/ Yes, this is not an AGI. But the hockey-stick takeoff from defeats some players, to defeats an undefeated world-champion, to defeats the version of itself that beat the world champion 100% of the time is nuts . If this happens in other domains, like f…
Thanks for that link, well worth the read.
This is an interesting question to ask in these "how far away is AGI" discussions:
I was once at a conference where there was a panel full of famous AI luminaries, and most of the luminaries were nodding and agreeing with each other that of course AGI was very far off, except for two famous AI luminaries who stayed quiet and let others take the microphone.
I got up in Q&A and said, “Okay, you’ve all told us that progress won’t be all that fast. But let’s be more concrete and specific. I’d like to know what’s the least impressive accomplishment that you are very confident cannot be done in the next two years.”
There was a silence.
Eventually, two people on the panel ventured replies, spoken in a rather more tentative tone than they’d been using to pronounce that AGI was decades out. They named “A robot puts away the dishes from a dishwasher without breaking them”, and Winograd schemas. Specifically, “I feel quite confident that the Winograd schemas—where we recently had a result that was in the 50, 60% range—in the next two years, we will not get 80, 90% on that regardless of the techniques people use.”
Re: AlphaGo Zero: Learning from scratch
#96> The system starts off with a neural network that knows nothing about the game of Go. It then plays games against itself
I might have missed this, but: Where are the actual rules of Go encoded? Mustn't there be some enumeration of what constitutes "capturing," how the win condition of the game is calculated, correct?
Re: AlphaGo Zero: Learning from scratch
#97Why don't use the same approach for chess? It's very interesting to see if it is able to handle much more advanced and tuned engines that exist for chess, game with considerable much more complicated rules?
Because Chess is a simpler game than Go.
Minimax with Alpha Beta pruning works in Chess because the search tree is way smaller. The reason why all this "Monte-Carlo Tree Search + Neural Nets" are being used in Go because Minimax + Alpha Beta pruning DOESN'T work in Go.
Re: AlphaGo Zero: Learning from scratch
#98Earlier quoted context omitted.
The key part from the paper: > To assess the merits of self-play reinforcement learning, compared to learning from human data, we trained a second neural network (using the same architecture) to predict expert moves in the KGS Server data set; this achieved state-of-the-art prediction accuracy compared to pre vious work 12,30–33 (see Extended Data Tables 1 and 2 for current and previous results, respectively). Supe…
That is really interesting. Given a neural network that solely exist to play Go, one that is influenced by the human mind is limited compared to the exact same set of neurons that doesn't have that influence. EDIT: changed a set of neurons to neural network per andbbergers comments
Please don't refer to it as 'a set of neurons' - it only serves to fuel the (IMO) absolutely ridiculous AI winter fearmongering, and is also just a bad description. Neural nets are linear algebra blackboxes, the connections to biology are tenuous at best.
Sorry to be that guy, but the AI hype is getting out hand. COSYNE this year was packed with papers comparing deep learning to the brain... it drives me nutty. Convnets can be reasonably put into analogy with the visual system.... because they were inspired from it. But that's about it.
To address your actual comment: I would argue that this is not really interesting or surprising (at least to the ML practitioner), it is very well known that neural nets are incredibly sensitive to initialization. Think of it like this: as training progresses, parameters of neural nets move along manifolds in parameter space, but they can get nudged off of the "right" manifold and will never be able to recover.
Sorry for the rant, the AI hype is just getting really out of hand recently.
Machine learning is specifically not magic. Blackboxes are not useful. Convnets work so well because they build the symmetries of natural scenes directly into the model - natural scenes are translation invariant (as well as a couple of other symmetries), anything that models them sure as hell better have those symmetries too, or you're just crippling your model with extra superfluous parameters.
Re: AlphaGo Zero: Learning from scratch
#99> Previous versions of AlphaGo initially trained on thousands of human amateur and professional games to learn how to play Go. AlphaGo Zero skips this step and learns to play simply by playing games against itself, starting from completely random play. So technically this version has lost every game it's ever won. Jokes aside, it's pretty interesting to note that they were able to combine the "policy" and "value" net…
TD-gammon is a well known version of this technique (with 2 ply lookahead, vs a 1600 deep mcts) https://en.m.wikipedia.org/wiki/TD-Gammon Temporal difference learning was previously consider weak at 'tactical' games, ie ones with gamestates that require long chains of precise moves to improve position (like many checkmate scenarios in chess) . For anyone more familiar with this technique, is it clear how the mcts/che…
It's my opinion that TD Gammon was solved in the 1990s because backgammon is a 1 dimensional board. It didn't need the convolutional techniques of the Go neural nets to gain insight into the game and could thus be solved by a traditional neural net.
Re: AlphaGo Zero: Learning from scratch
#100The fact that they only used self play with no outside input here is really interesting. I wonder if this system produced more new styles of play. While I am not that familiar with Go, I know in some of the other articles they talk about things like Chinese starts that are specific to certain cultures. I wonder if the fact that it had no outside reinforcement made it produce movements that we have already seen that a…
> I wonder if this system produced more new styles of play.
One thing Alpha go has told us clearly is that it thinks human players over value the margin of victory vs the probability of victory.