Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How much did AlphaGo Zero cost? (2018)

yuzeh.com

Post reply on HN
Print view View original

41–50 of 179 posts

Re: How much did AlphaGo Zero cost? (2018)

#41

Post by eeeficus »

rurban wrote: post #38

Misleading numbers, and wrong calculations. The TPU and CPU cost them almost nothing as they use and build them anyway, and renting them out for this PR stunt just cost them the missed rental time, if customers would really pay that much. Maybe around 20.000. Energy cost? I don't see much additional costs as those machines run all the time, regardless if improving the model or doing something else. I bet the much hig…

The OP talks about how much would’ve cost a third party to replicate the experiments. That’s what I get from it anyway.

Re: How much did AlphaGo Zero cost? (2018)

#42

Post by slx26 »

raverbashing wrote: post #10

Wondering when researchers will switch from "race to the moon" mode to looking at better optimization techniques instead of just throwing money at the problem. I know some companies are doing that, but I think looking at AlphaGo or AGZ and making it go faster should be an interesting problem in itself.

Can you share the names of those companies, or some of their projects? At least that way people who thinks like you can follow those efforts and try to give them more visibility.

Re: How much did AlphaGo Zero cost? (2018)

#43

Post by gridlockd »

It is estimated to be 36 million for someone else to train AlphaGo Zero, assuming they use Google TPU instances and pay the sticker price.

Google isn't operating with that cost, unless we assume that they are prioritizing AlphaGo to the point where they lose such customers 100% of the time.

It's way more likely that AlphaGo is trained on spare time, the cost for the hardware is sunk anyway, so only the cost for upkeep is real.

Re: How much did AlphaGo Zero cost? (2018)

#44

Post by tpetry »

londons_explore wrote: post #35
conistonwater wrote: post #5

Are they using the on-demand price instead of the preemptible price? It seems like the sort of job that can run on preemptible machines, just because it's a batch job. Also, should the cost really be calculated using public market prices at all, as opposed to the running costs of the TPUs? It is not guaranteed at all that the opportunity cost to Google of using all those TPUs is equal to the price that you or I would…

One could imagine an 'ideal' allocation of TPU's, where compute time is allocated to the project that earns the most dollars per FLOP. Minor improvements to Google Books OCR might not be worth much, whereas better search result scoring would be worth lots. An automated system would decide where it was most efficient to spend the TPU's. Management would set how many dollars a 10% performance improvement was worth. I'm…

You mean better ad revenue, because search results are getting worse and worse. So search results can't really be google's primary focus anymore.

Re: How much did AlphaGo Zero cost? (2018)

#45

Post by Scarblac »

hjnilsson wrote: post #7

Another way of thinking about how efficient the brain is: By the article’s numbers, about 5.5 million TPU hours were required to train the machine to play as well as a Go champion. A Go champion might have trained for 8 hours a day, for 15 years (age 5 to 20). That is about 40 000 hours. In other words, machines required 137 times longer to learn the game, and at twice the power consumption! There is still a lot of r…

But there are also many other people spending time studying Go who didn't reach that level. We ran all that studying in parallel and then selected the best person by running a world championship. You can't only count his effort alone.

Re: How much did AlphaGo Zero cost? (2018)

#46

Post by visarga »

londons_explore wrote: post #35
conistonwater wrote: post #5

Are they using the on-demand price instead of the preemptible price? It seems like the sort of job that can run on preemptible machines, just because it's a batch job. Also, should the cost really be calculated using public market prices at all, as opposed to the running costs of the TPUs? It is not guaranteed at all that the opportunity cost to Google of using all those TPUs is equal to the price that you or I would…

One could imagine an 'ideal' allocation of TPU's, where compute time is allocated to the project that earns the most dollars per FLOP. Minor improvements to Google Books OCR might not be worth much, whereas better search result scoring would be worth lots. An automated system would decide where it was most efficient to spend the TPU's. Management would set how many dollars a 10% performance improvement was worth. I'm…

> One could imagine an 'ideal' allocation of TPU's, where compute time is allocated to the project that earns the most dollars per FLOP.

That's a short sighted, immediate benefit or bust mentality. Not to mention that projects have a ramp-up time where they are not profitable yet, but still very valuable strategically.

Re: How much did AlphaGo Zero cost? (2018)

#47

Post by jacquesm »

bonzini wrote: post #37
irjustin wrote: post #4

Alpha Go Zero inspired the development of an open source version, Leela Go Zero which Leela Chess Zero is forked from by the same guy who made Stock Fish. Lots of people contribute what I imagine are amounts of CPU Power/money to the Leela Chess Zero project[1]. Would love to see Alpha Chess vs Leela Chess. [1] https://training.lczero.org/ [edit] I've caused terrible confusion by melding Leela Go and Leela Chess when…

As far as I know Gian-Carlo Pascutto is not among the original authors of Stockfish, though he did work on chess engines. Perhaps you were confused because Leela Chess Zero was forked from Leela Zero (neural network Go engine by Pascutto) but it includes Stockfish's move generation logic.

He's also well known for his absolutely amazing work on audio fingerprinting.

Re: How much did AlphaGo Zero cost? (2018)

#48

Post by melbourne_mat »

rurban wrote: post #38

Misleading numbers, and wrong calculations. The TPU and CPU cost them almost nothing as they use and build them anyway, and renting them out for this PR stunt just cost them the missed rental time, if customers would really pay that much. Maybe around 20.000. Energy cost? I don't see much additional costs as those machines run all the time, regardless if improving the model or doing something else. I bet the much hig…

"renting them out for this PR stunt just cost them the missed rental time"

Because renting them out generates no revenue, right?

"Maybe around 20.000"

At least the article used a formula for the calculation. You just picked a number at random.

Re: How much did AlphaGo Zero cost? (2018)

#49

Post by Intermernet »

irjustin wrote: post #4

Alpha Go Zero inspired the development of an open source version, Leela Go Zero which Leela Chess Zero is forked from by the same guy who made Stock Fish. Lots of people contribute what I imagine are amounts of CPU Power/money to the Leela Chess Zero project[1]. Would love to see Alpha Chess vs Leela Chess. [1] https://training.lczero.org/ [edit] I've caused terrible confusion by melding Leela Go and Leela Chess when…

I'd love to see some CPU/GPU/TPU donations to @chewxy's "agogo"[1] which is AlphaGo (AlphaZero) re-implemented in Go as a sort of proof of concept / demo of Gorgonia[2].

[1]: https://github.com/gorgonia/agogo

[2]: https://github.com/gorgonia/gorgonia

Re: How much did AlphaGo Zero cost? (2018)

#50

Post by jaekash »

> This accomplishment is truly remarkable in that it shows that we can develop systems that teach themselves to do non-trivial tasks from a blank slate, and eventually become better than humans at doing the task.

"non-trivial" is a bit of a red herring here. Playing go is pretty trivial compared to something like walking or scratching your face. Winning go may be non-trivial compared to those in some ways but it is very trivial in comparison in other ways.

Post reply on HN