How much did AlphaGo Zero cost? (2018)

Post by vegesm » Thu, Jun 11, 2020, 2:54 PM UTC

tinco wrote: post #23

Our main compute doesn't go towards machine learning, but we do rely heavily on GPU power. I recently had to come up with the figures for us to invest in an expansion of our compute power, and it turned out that buying the machines ourselves would be cheaper than renting them from Google in 3-4 months. We don't run on those fancy V100 cards though, just regular old gaming cards suffice, and I suppose if we bought the…

Post by runarberg » Thu, Jun 11, 2020, 3:07 PM UTC

thomasahle wrote: post #77

29athrowaway wrote: post #13

Earlier quoted context omitted.

I think KataGo is stronger than Leela Zero. https://github.com/lightvector/KataGo

I was suspecting another boring clone, but Kata looks like a cool project with nice new ideas! Thanks for sharing

Post by 317070 » Thu, Jun 11, 2020, 3:18 PM UTC

imtringued wrote: post #73

andbberger wrote: post #32

Earlier quoted context omitted.

That's a pretty big claim. One could argue that the topology of the brain is a prior, analogous to the architecture of a neural net. But considering that we really have no idea how learning happens in the brain on a large scale, you really can't say.

I don't remember the name but I definitively saw attempts to build a general AI that was designed to solve 50 different games. There was one long learning phase where the AI learns mechanics that are common to all of the 50 games and then there is a much shorter learning phase that is just specific to a single game. Same attempt was made with a Minecraft bot. First it just learned how to live and interact in a vanill…

Post by geodel » Thu, Jun 11, 2020, 3:31 PM UTC

sukilot wrote: post #114

tech-historian wrote: post #111

Achieving a new breakthrough in computing is often very expensive. Deep Blue is estimated to cost IBM over $100 million over a decade [1]. And in comparison to large tech company R&D budgets, the amount cited in the article is a drop in the bucket. Consider the fact that Google spent $26 billion in R&D budget in 2019 alone [2]. Microsoft spent almost $17 billion [3]. [1] https://www.extremetech.com/computing/76552-pr…

Note that everything in software development is R&D. Building (not operating) Gmail and Android and Office and Azure are R&D.

Post by elcritch » Thu, Jun 11, 2020, 3:54 PM UTC

simiones wrote: post #63

elcritch wrote: post #24

Earlier quoted context omitted.

It'd be really interesting if a research group could calculate an entropic calculation on how efficient training any given neural network would be. As in what is the thermodynamic limit of the most optimal NN training could be in terms of watts per bit trained. My hunch would be that human brains would operate close to this limit. At least in our standard environmental conditions. Based on how near optimal biomateria…

I think the problem you'd find is that "bit trained" is probably highly non-trivial. For example, I expect that the training required to go from 7-year-old child to Go grand master requires a completely different number of bits of information than the training required to go from blanks-late NN to NN Go Grand master. I also suspect that the difference in what is being learned may well dominate the difference in train…