Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How much did AlphaGo Zero cost? (2018)

yuzeh.com

Post reply on HN
Print view View original

61–70 of 179 posts

Re: How much did AlphaGo Zero cost? (2018)

#61

Post by est31 »

bonzini wrote: post #37
irjustin wrote: post #4

Alpha Go Zero inspired the development of an open source version, Leela Go Zero which Leela Chess Zero is forked from by the same guy who made Stock Fish. Lots of people contribute what I imagine are amounts of CPU Power/money to the Leela Chess Zero project[1]. Would love to see Alpha Chess vs Leela Chess. [1] https://training.lczero.org/ [edit] I've caused terrible confusion by melding Leela Go and Leela Chess when…

As far as I know Gian-Carlo Pascutto is not among the original authors of Stockfish, though he did work on chess engines. Perhaps you were confused because Leela Chess Zero was forked from Leela Zero (neural network Go engine by Pascutto) but it includes Stockfish's move generation logic.

I think OP was referring to Gary Linscott who has made major contributions to both Stockfish and made an adaptation of Leela Zero to Chess, now living under the Leela Chess Zero Github org but apparently a different adaptation is now the officially sanctioned one, at least his commits don't show up in the new lc0 repo.

https://github.com/official-stockfish/Stockfish/graphs/contr...

https://github.com/LeelaChessZero/lczero/graphs/contributors

Re: How much did AlphaGo Zero cost? (2018)

#62

Post by PeterisP »

andbberger wrote: post #32
phreeza wrote: post #16

Earlier quoted context omitted.

This comparison is not entirely fair because the human brain also benefits from priors baked in over the entire course of evolution.

That's a pretty big claim. One could argue that the topology of the brain is a prior, analogous to the architecture of a neural net. But considering that we really have no idea how learning happens in the brain on a large scale, you really can't say.

I believe that there's a reasonable (but unprovable) assumption that any games which humans usually play - because the rulesets are learnable and interesting for humans - implicitly rely on priors of human brains and behavior.

The space of possible games is huge (infinite?), but only a tiny subset of these games could reasonably become a popular game for humans.

E.g. it's not an arbitrary random coincidence that the scoring rules for each grid intersection in go are the same (I mean, it could vary in an arbitrary pattern), it ensures that the ruleset is small enough so that humans can learn it.

It's not an arbitrary random coincidence that the playing of go involves pattern recognition on some level, since that's what we're good at and find interesting in many games.

It's not an arbitrary random coincidence that in Mario game after jumping the sprite falls back down eventually; that's reusing the priors from real world physics.

Re: How much did AlphaGo Zero cost? (2018)

#63

Post by simiones »

elcritch wrote: post #24
hjnilsson wrote: post #7

Another way of thinking about how efficient the brain is: By the article’s numbers, about 5.5 million TPU hours were required to train the machine to play as well as a Go champion. A Go champion might have trained for 8 hours a day, for 15 years (age 5 to 20). That is about 40 000 hours. In other words, machines required 137 times longer to learn the game, and at twice the power consumption! There is still a lot of r…

It'd be really interesting if a research group could calculate an entropic calculation on how efficient training any given neural network would be. As in what is the thermodynamic limit of the most optimal NN training could be in terms of watts per bit trained. My hunch would be that human brains would operate close to this limit. At least in our standard environmental conditions. Based on how near optimal biomateria…

I think the problem you'd find is that "bit trained" is probably highly non-trivial.

For example, I expect that the training required to go from 7-year-old child to Go grand master requires a completely different number of bits of information than the training required to go from blanks-late NN to NN Go Grand master. I also suspect that the difference in what is being learned may well dominate the difference in training efficiency. Both the prior knowledge and the mechanism of learning are so different that I doubt you could get a meaningful comparison based on current understanding.

You should remember that we have no idea basically how human beings actually learn things, and no idea how much prior knowledge we have encoded. Just for an example, I once saw a documentary that claimed chess grandmasters seem to recognize valid chess positions using the parts of the brain that usually recognize faces. Assuming that was true (I'm not claiming it is) perhaps a part of their chess learning consisted in taking a built-in face recognizing NN and training it to recognize chess boards. How much did the built-in knowledge of recognizing faces help? I don't think it would be possible to calculate.

Re: How much did AlphaGo Zero cost? (2018)

#64

Post by simiones »

PeterisP wrote: post #51
jaekash wrote: post #50

> This accomplishment is truly remarkable in that it shows that we can develop systems that teach themselves to do non-trivial tasks from a blank slate, and eventually become better than humans at doing the task. "non-trivial" is a bit of a red herring here. Playing go is pretty trivial compared to something like walking or scratching your face. Winning go may be non-trivial compared to those in some ways but it is v…

Playing go at that level is non-trivial compared to walking because (a) most humans can walk, but not even the best human go masters can play go at that level; (b) we had algorithms that allow bipedal robots to walk long before we had algorithms for playing go at that level.

Do we have algorithms that allow bipedal robots to walk at human level? Or run at, let's say, 10th grade standard student level?

Re: How much did AlphaGo Zero cost? (2018)

#65

Post by sanxiyn »

raverbashing wrote: post #10

Wondering when researchers will switch from "race to the moon" mode to looking at better optimization techniques instead of just throwing money at the problem. I know some companies are doing that, but I think looking at AlphaGo or AGZ and making it go faster should be an interesting problem in itself.

KataGo optimized AlphaZero and achieved 50x compute reduction.

https://arxiv.org/abs/1902.10565

Re: How much did AlphaGo Zero cost? (2018)

#66

Post by yamrzou »

andbberger wrote: post #32
phreeza wrote: post #16

Earlier quoted context omitted.

This comparison is not entirely fair because the human brain also benefits from priors baked in over the entire course of evolution.

That's a pretty big claim. One could argue that the topology of the brain is a prior, analogous to the architecture of a neural net. But considering that we really have no idea how learning happens in the brain on a large scale, you really can't say.

Does anyone have an idea about the advances in research about this topic i.e. human intelligence and how learning happens in the brain?

I know Josh Tenenbaum from MIT [1] works on this, see for example :

- How to Grow a Mind: Statistics, Structure and Abstraction [2]

- Steps towards more human-like learning in machines [3]

Wondering if there are other researchers exploring similar questions.

[1] http://web.mit.edu/cocosci/josh.html

[2] https://www.youtube.com/watch?v=97MYJ7T0xXU

[3] https://www.youtube.com/watch?v=WTK6eaSVTjo

Re: How much did AlphaGo Zero cost? (2018)

#67

Post by PeterisP »

simiones wrote: post #64
PeterisP wrote: post #51

Earlier quoted context omitted.

Playing go at that level is non-trivial compared to walking because (a) most humans can walk, but not even the best human go masters can play go at that level; (b) we had algorithms that allow bipedal robots to walk long before we had algorithms for playing go at that level.

Do we have algorithms that allow bipedal robots to walk at human level ? Or run at, let's say, 10th grade standard student level?

Bipedal walking and running on uneven terrain has been demonstrated years ago - Boston Dynamics has a lot of nice videos; e.g. here's a four year old video on walking https://www.youtube.com/watch?v=rVlhMGQgDkY , here's two year old videos on running https://www.youtube.com/watch?v=vjSohj-Iclc and https://www.youtube.com/watch?v=LikxFZZO2sk - perhaps it's debatable if it's "full human level" but it seems sufficient to me for most purposes. IMHO any improvements to that would be mostly for sensors (seeing what terrain is there even in poor conditions e.g. fog/rain/snow) and modelling environment (understanding which objects might break or slide if you step on them), not walking/running as such.

The main problem for bipedal robots that makes them still impractical is the hardware expense (wheels are simpler and cheaper) and the power supply required, so for most use cases it's more efficient to use something other than a bipedal robot and there's limited business application and future revenue in scaling up research demos of bipedal walking to practicality, so most people who are working on walking algorithms are doing so in simulated virtual environments (where we have algorithms that can learn walking and running "from scratch" through experimentation) and not building very expensive hardware.

Re: How much did AlphaGo Zero cost? (2018)

#69

Post by visarga »

andbberger wrote: post #32
phreeza wrote: post #16

Earlier quoted context omitted.

This comparison is not entirely fair because the human brain also benefits from priors baked in over the entire course of evolution.

That's a pretty big claim. One could argue that the topology of the brain is a prior, analogous to the architecture of a neural net. But considering that we really have no idea how learning happens in the brain on a large scale, you really can't say.

Not just the topology of the brain, but the environment is also important. Human life is more diverse than that of AlphaGo, we can borrow concepts gained while doing something else. Should we count those external tasks as part of the learning to play Go?

Re: How much did AlphaGo Zero cost? (2018)

#70

Post by sauercloud »

Certhas wrote: post #39
conistonwater wrote: post #5

Are they using the on-demand price instead of the preemptible price? It seems like the sort of job that can run on preemptible machines, just because it's a batch job. Also, should the cost really be calculated using public market prices at all, as opposed to the running costs of the TPUs? It is not guaranteed at all that the opportunity cost to Google of using all those TPUs is equal to the price that you or I would…

The article actually addresses that. The precise number is not the point but the ballpark is: "In terms of actual cost to DeepMind (a subsidiary of Google’s parent company) to run the experiment, there are other factors that need to be taken into account, such as researcher salaries, or that the quoted TPU rate probably includes a healthy amount of margin. But for someone outside Google, this number is a good ballpar…

True, but "giving a ballpark estimate to replicate the experiment" is quite different to "How much did it cost"
Post reply on HN