Curious, has this method been used for solving linear systems? How would it perform e.g. against conjugate gradient? And how would it perform for non-positive-definite systems?
[deleted]
Why Momentum Works
11–20 of 100 posts
Re: Why Momentum Works
#12I'm really loving the choice of articles, especially since you're just getting started. Edit: I'm referring to the journal, not the author.
Thanks! We're just lucky to have authors like Gabe (@gabrielgoh) come to us with incredible articles. :)
Re: Why Momentum Works
#13Related? Flat minima
Re: Why Momentum Works
#14I'm really loving the choice of articles, especially since you're just getting started. Edit: I'm referring to the journal, not the author.
Thanks! We're just lucky to have authors like Gabe (@gabrielgoh) come to us with incredible articles. :)
I'm curious, did the author write the whole article including figures, or did someone else give life to the figures?
I can see this type of interactive journal becoming very popular in other fields, but not if the author has to create the diagrams him/herself.
Re: Why Momentum Works
#15Related? Flat minima http://www.bioinf.jku.at/publications/older/3304.pdf
This is indeed related. See the section on polynomial regression!
Re: Why Momentum Works
#16Curious, has this method been used for solving linear systems? How would it perform e.g. against conjugate gradient? And how would it perform for non-positive-definite systems?
Author here - yes! It can be used to solve symmetric PSD Systems, as the solution of Ax = b is also the minimizer of 0.5*x'Ax - b'x. Conjugate gradient can be seen as a special form of momentum with adaptive alphas and betas
> Conjugate gradient can be seen as a special form of momentum
Just to be clear, though, CG doesn't use the negative gradients as search directions, as steepest descent would.
Re: Why Momentum Works
#17Reminds me of this paper, "Causal Entropic Forces":
http://math.mit.edu/~freer/papers/PhysRevLett_110-168702.pdf
Re: Why Momentum Works
#18It seems have no improvement for badly conditioned problems, i.e. k >> 1. The convergence rate is 1 with or without the damped momentum.
Re: Why Momentum Works
#19But simply not on Firefox
it spun my firefox hard
Re: Why Momentum Works
#20I'm really loving the choice of articles, especially since you're just getting started. Edit: I'm referring to the journal, not the author.
Likewise, I'm very impressed with the early article selection. The site itself is beautiful, and interactive figures like the one at the top of this link are incredibly helpful.
Overall -- huge fan!