This is a lot of money.
However, if you want to reliably make an AI the best in the world at a range of complicated tasks, can you reasonably expect this to be cheap?
How much did AlphaGo Zero cost? (2018)
31–40 of 179 posts
Re: How much did AlphaGo Zero cost? (2018)
#32Another way of thinking about how efficient the brain is: By the article’s numbers, about 5.5 million TPU hours were required to train the machine to play as well as a Go champion. A Go champion might have trained for 8 hours a day, for 15 years (age 5 to 20). That is about 40 000 hours. In other words, machines required 137 times longer to learn the game, and at twice the power consumption! There is still a lot of r…
This comparison is not entirely fair because the human brain also benefits from priors baked in over the entire course of evolution.
That's a pretty big claim. One could argue that the topology of the brain is a prior, analogous to the architecture of a neural net. But considering that we really have no idea how learning happens in the brain on a large scale, you really can't say.
Re: How much did AlphaGo Zero cost? (2018)
#33Are they using the on-demand price instead of the preemptible price? It seems like the sort of job that can run on preemptible machines, just because it's a batch job. Also, should the cost really be calculated using public market prices at all, as opposed to the running costs of the TPUs? It is not guaranteed at all that the opportunity cost to Google of using all those TPUs is equal to the price that you or I would…
For me I think it's very interesting just to get a ballpark order-of-magnitude estimate. I'm sure the cost isn't much less than $3.6M and so the underlying story doesn't change that much, i.e., this is not something accessible to hobbyists.
Re: How much did AlphaGo Zero cost? (2018)
#34The amount was removed from the submission title, which sucks if you're like me and don't like to visit yet another possibly JS-heavy site and drain your battery. For others: It's $36M.
I was of half a mind to un-upvote because I liked knowing the number but didn't care enough about the article. Now I don't think it's as much worth it for this to be higher up.
Also nobody mentioned the title is inaccurate so I guess it's just pedantic "thou shalt not change zhe title" rather than "title was misleading/clickbait"...
Re: How much did AlphaGo Zero cost? (2018)
#35Are they using the on-demand price instead of the preemptible price? It seems like the sort of job that can run on preemptible machines, just because it's a batch job. Also, should the cost really be calculated using public market prices at all, as opposed to the running costs of the TPUs? It is not guaranteed at all that the opportunity cost to Google of using all those TPUs is equal to the price that you or I would…
One could imagine an 'ideal' allocation of TPU's, where compute time is allocated to the project that earns the most dollars per FLOP.
Minor improvements to Google Books OCR might not be worth much, whereas better search result scoring would be worth lots. An automated system would decide where it was most efficient to spend the TPU's. Management would set how many dollars a 10% performance improvement was worth.
I'm sure the reality is a bunch of middle managers arguing over why their team deserves them more than another.
Re: How much did AlphaGo Zero cost? (2018)
#36Alpha Go Zero inspired the development of an open source version, Leela Go Zero which Leela Chess Zero is forked from by the same guy who made Stock Fish. Lots of people contribute what I imagine are amounts of CPU Power/money to the Leela Chess Zero project[1]. Would love to see Alpha Chess vs Leela Chess. [1] https://training.lczero.org/ [edit] I've caused terrible confusion by melding Leela Go and Leela Chess when…
There are several people who wrote Stockfish, including Tord Romstad, whose Glaurung was the initial base for Stockfish.
Glaurung was pretty innovative at the time.
Re: How much did AlphaGo Zero cost? (2018)
#37Alpha Go Zero inspired the development of an open source version, Leela Go Zero which Leela Chess Zero is forked from by the same guy who made Stock Fish. Lots of people contribute what I imagine are amounts of CPU Power/money to the Leela Chess Zero project[1]. Would love to see Alpha Chess vs Leela Chess. [1] https://training.lczero.org/ [edit] I've caused terrible confusion by melding Leela Go and Leela Chess when…
As far as I know Gian-Carlo Pascutto is not among the original authors of Stockfish, though he did work on chess engines.
Perhaps you were confused because Leela Chess Zero was forked from Leela Zero (neural network Go engine by Pascutto) but it includes Stockfish's move generation logic.
Re: How much did AlphaGo Zero cost? (2018)
#38Misleading numbers, and wrong calculations. The TPU and CPU cost them almost nothing as they use and build them anyway, and renting them out for this PR stunt just cost them the missed rental time, if customers would really pay that much. Maybe around 20.000.
Energy cost? I don't see much additional costs as those machines run all the time, regardless if improving the model or doing something else.
I bet the much higher cost was the PR team, including the film team, press support, TV team, travels, inviting the expert Go players, building the stage, and such. Estimated 100.000.
Not counting the man hours, they were just doing their normal job.
Re: How much did AlphaGo Zero cost? (2018)
#39Are they using the on-demand price instead of the preemptible price? It seems like the sort of job that can run on preemptible machines, just because it's a batch job. Also, should the cost really be calculated using public market prices at all, as opposed to the running costs of the TPUs? It is not guaranteed at all that the opportunity cost to Google of using all those TPUs is equal to the price that you or I would…
The article actually addresses that. The precise number is not the point but the ballpark is:
"In terms of actual cost to DeepMind (a subsidiary of Google’s parent company) to run the experiment, there are other factors that need to be taken into account, such as researcher salaries, or that the quoted TPU rate probably includes a healthy amount of margin. But for someone outside Google, this number is a good ballpark estimate of how much it would cost to replicate this experiment."
Re: How much did AlphaGo Zero cost? (2018)
#40Our main compute doesn't go towards machine learning, but we do rely heavily on GPU power. I recently had to come up with the figures for us to invest in an expansion of our compute power, and it turned out that buying the machines ourselves would be cheaper than renting them from Google in 3-4 months. We don't run on those fancy V100 cards though, just regular old gaming cards suffice, and I suppose if we bought the…
How Google started out: Run servers from a shack, using commodity PC hardware
How modern startups start out: Spend $50,000/month to run hundreds of microservices on a managed Kubernetes cluster