Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Alpha Go Zero: How and Why It Works

tim.hibal.org

Post reply on HN
Print view View original

111–116 of 116 posts

Re: Alpha Go Zero: How and Why It Works

#111

Post by _0ffh »

dfan wrote: post #98
_0ffh wrote: post #64

Earlier quoted context omitted.

For anyone interested: Learn more on TD and RL in general from Sutton (inventor of TD-lambda) and Barto's book: http://www.incompleteideas.net/sutton/book/the-book.html Sidenote: It used to be that simply googling "sutton barto book" would bring you to the right place with the first suggested link. Now this stuff is so popular all of a sudden, I needed to consult the link I had set on my own page in order to find it.…

Here is the latest publicly available draft of the 2nd edition (June 2017, 538 pages): http://incompleteideas.net/sutton/book/bookdraft2017june19.p...

Better to use my link, not the deep one. On mine you can click through to the latest draft (as of writing this, November 5th, 2017)!

Re: Alpha Go Zero: How and Why It Works

#112

Post by pmarreck »

thomasahle wrote: post #80
pmarreck wrote: post #77

> It is interesting to see how quickly the field of AI is progressing. Those who claim we will be able to see the robot overlords coming in time should take heed - these AI's will only be human-level for a brief instant before blasting past us into superhuman territories, never to look back. This final paragraph is just editorializing. A computer will never care about anything (including games like Go and domination…

What do you mean with inverting a matrix? In the usual definition it is well studied and understood https://en.m.wikipedia.org/wiki/Invertible_matrix#Methods_of...

I just... Here's the deal. Most good math is elegant, it has a beauty about it. And as a programmer, good code has the same "feel." But when I took Andrew Ng's class, gradient descent felt... not beautiful at all... and just based on that beauty-seeking intuition, there MUST be a more elegant way to get directly at the inverse of a matrix (assuming it exists and is possible) than all of these... complicated expensive workarounds. And the pot at the end of that rainbow is you get the EXACT value that gradient descent is wasting all this time trying to snowboard down N-dimensional space to get to!

I know I may be asking to make what may be a fundamentally-hard problem, easy... I'm just saying that I don't get the same, eh, "buzz" I get when I see a really cool solution for something (which I've often gotten from math, physics, programming, etc.). This is very hard to explain as it's probably irrational and thus I can't make a strong rational argument for it lol (admittedly!)

I really love linear algebra, btw, so this should be right up my alley.

Re: Alpha Go Zero: How and Why It Works

#113

Post by pmarreck »

gugagore wrote: post #102
thomasahle wrote: post #80

Earlier quoted context omitted.

What do you mean with inverting a matrix? In the usual definition it is well studied and understood https://en.m.wikipedia.org/wiki/Invertible_matrix#Methods_of...

They might be talking about an interpretation of gradient descent that this article provides: http://blog.mrtz.org/2013/09/07/the-zen-of-gradient-descent....

I elaborated here: https://news.ycombinator.com/item?id=15639863

Re: Alpha Go Zero: How and Why It Works

#114

Post by shard »

ben_w wrote: post #8
javajosh wrote: post #3

"On a long enough timeline, everything is a discrete game." (With apologies to _Fight Club_) Personally, I look forward to the day when the software I own works for me to the extent of optimizing the decisions I make during the day, even many mundane ones. Properly executed, such a system could make a big difference in my quality of life. I believe that a big piece that is missing is a solid life-model, a life-repres…

If it is optimising all your decisions, even the mundane ones, is it really your life any more? If it plugged into your brainstem and took over, by definition nobody else would notice the difference (you always did what it said anyway, or you’d be suboptimal), so why not just flood the 20 watt carbon-based neural network with bliss drugs and let the silicon-based neural network be the person instead?

Or just replace the brain completely with the computer, like in Greg Egan's jewel stories. http://will.tip.dhappy.org/blog/Compression%20Trees/.../book...

Re: Alpha Go Zero: How and Why It Works

#115

Post by javajosh »

dlwdlw wrote: post #61
javajosh wrote: post #3

"On a long enough timeline, everything is a discrete game." (With apologies to _Fight Club_) Personally, I look forward to the day when the software I own works for me to the extent of optimizing the decisions I make during the day, even many mundane ones. Properly executed, such a system could make a big difference in my quality of life. I believe that a big piece that is missing is a solid life-model, a life-repres…

This may interest you, AI as an artificial belief system. http://us1.campaign-archive.com/?u=78cbbb7f2882629a5157fa593... My interpretation is that AI can allow you to rapidly load and unload mental models so that you're free to devote 100% of your mind to what only you (a human) can do.

Replying late (a day is ancient history in HN time) but I liked your comment a lot. My view is that our belief system is encoded in our behavior, not our words. So an AI could provide us with the first honest measurement of morality, according to our own standards.

Re: Alpha Go Zero: How and Why It Works

#116

Post by gwern »

gwern wrote: post #105
psb217 wrote: post #101

Earlier quoted context omitted.

If you read the AGZ paper closely, they actually use checkpoints during training. Specifically, during training they only perform updates to the "stable" set of parameters when the current "learning" set of parameters produces a policy which beats the stable set at least 55% of the time. The current stable parameters are what they use for generating the self-play data which they use to update the current "learning" p…

I did and I would point out that while they use checkpoints, the training curves indicate this is not necessary, and what I meant is that they do not use the usual self-play (and evolutionary) mechanism of checkpoints from throughout the training history which is necessary to combat catastrophic forgetting (and which apparently wasn't enough to stabilize Zero on its own as it is the single most obvious thing to do bu…

(And Anthony et al 2017 don't use checkpoints at all, noting that it slows things down a lot for no benefit in their Hex agent.)
Post reply on HN