How much did AlphaGo Zero cost? (2018)
1–10 of 179 posts
Re: How much did AlphaGo Zero cost? (2018)
#2Alpha Go Zero*, which was trained from scratch, without human games.
I've also heard rumors that AlphaStar (https://deepmind.com/blog/article/alphastar-mastering-real-t...) was essentially put on hold because it was too expensive to improve/train. The bot wasn't able to beat StarCraft champions and _only_ got to a grandmaster level.
Re: How much did AlphaGo Zero cost? (2018)
#3Does Sarbanes Oxley apply to zero rating ML costs? Alpha go might have unfair kyu ranking, if Google don't have to "pay" to acquire rank. (95% joking)
Re: How much did AlphaGo Zero cost? (2018)
#4Alpha Go Zero inspired the development of an open source version, Leela Go Zero which Leela Chess Zero is forked from by the same guy who made Stock Fish.
Lots of people contribute what I imagine are amounts of CPU Power/money to the Leela Chess Zero project[1].
Would love to see Alpha Chess vs Leela Chess.
[1] https://training.lczero.org/
[edit] I've caused terrible confusion by melding Leela Go and Leela Chess when Leela Chess was originally forked from Leela Go and that's basically when similarities end.
Edited for a bit more clarity.
Re: How much did AlphaGo Zero cost? (2018)
#5Are they using the on-demand price instead of the preemptible price? It seems like the sort of job that can run on preemptible machines, just because it's a batch job. Also, should the cost really be calculated using public market prices at all, as opposed to the running costs of the TPUs? It is not guaranteed at all that the opportunity cost to Google of using all those TPUs is equal to the price that you or I would pay Google to use them. I understand it cost a lot, but I'm not convinced by the headline figure of $36M.
Re: How much did AlphaGo Zero cost? (2018)
#6Alpha Go Zero*, which was trained from scratch, without human games. I've also heard rumors that AlphaStar ( https://deepmind.com/blog/article/alphastar-mastering-real-t... ) was essentially put on hold because it was too expensive to improve/train. The bot wasn't able to beat StarCraft champions and _only_ got to a grandmaster level.
Interesting ! i didn't understand why they stopped the alphastar project. They pretended they reached their goal, but clearly haven't.
Re: How much did AlphaGo Zero cost? (2018)
#7Another way of thinking about how efficient the brain is: By the article’s numbers, about 5.5 million TPU hours were required to train the machine to play as well as a Go champion.
A Go champion might have trained for 8 hours a day, for 15 years (age 5 to 20). That is about 40 000 hours.
In other words, machines required 137 times longer to learn the game, and at twice the power consumption! There is still a lot of room for improvement.
Re: How much did AlphaGo Zero cost? (2018)
#8> Each move during self-play uses about 0.4 seconds of computer thinking time.
> Over 72 hours, 4.9 million matches were played.
One of this claim must be incorrect or misinterpreted, I highly doubt they used so many TPU's as the article claims. That would be not only impractical but also it would raise a lot of other issues like networking, disk speed... etc...
My statement is not against this article, if anyone can confirm they used so many TPUs in parallel feel free to post it
Re: How much did AlphaGo Zero cost? (2018)
#9> The power consumption of the experiment is equivalent to 12,760 human brains running continuously.
Given the experiment lasts for just days, this actually sounds pretty impressive I think.
Many humans studied the game for a big portion of their lives in order to get Go knowledge where it is.
Re: How much did AlphaGo Zero cost? (2018)
#10Wondering when researchers will switch from "race to the moon" mode to looking at better optimization techniques instead of just throwing money at the problem.
I know some companies are doing that, but I think looking at AlphaGo or AGZ and making it go faster should be an interesting problem in itself.