61–70 of 100 posts
It would be nice if the introduction made clear that the "Momentum" that "works" is some algorithm and not at all the physical concept "momentum".
Earlier quoted context omitted.
Hi! This article is about momentum in the mathematical field of optimization. Acceleration also refers to a phenomenon in optimization. While there are deep connections to their physical analogues, they aren't aren't quite the same thing. If you want to make your analogy work, the momentum algorithm adds mass to an object. In terms of literal acceleration at any point in time it is neutral, but the added mass causes…
The use of the term "accelerator" is misleading as it's introduced during the physical metaphor. You should consider editing that sentence to clarify your meaning: The added inertia smooths out variation in velocity, dampening oscillations and causing us to barrel through narrow valleys, small humps and local minima. Keeping our speed steadier, we arrive at the global optimum faster. Note the alliteration :-) > momen…
It would be nice if the introduction made clear that the "Momentum" that "works" is some algorithm and not at all the physical concept "momentum".
Earlier quoted context omitted.
I disagree. The most useful replies can be upvoted to deal with noise, and using mathematical formulas to help explain responses is absolutely crucial. I can't tell you how helpful it has been to be looking at an obscure proof, post a question to Math Overflow, and have the answer explained in an intuitive way with reference to the symbols and notation used. These articles on distill I believe could greatly benefit f…
I see where you're coming from. But maintaining another service and the added expenses, community managers, spam control etc. might be a bit much for something that intended to be a publishing platform. And if there is a question that requires more control, like math formatting, etc. I would actually suggest posting to cross validated[1] and then linking it here. [1]: http://stats.stackexchange.com/
I'm curious about the method chosen to give short term memory to the gradient. The most common way I've seen when people have a time sequence of values X[i] and they want to make a short term memory version Y[i] is to do something of this form: Y[i+1] = B * Y[i] + (1-B) * X[i+1] where 0 Note that if the sequence X becomes a constant after some point, the sequence Y will converge to that constant (as long as B != 1).…
I assume that multiplying by a given factor shouldn't matter since you still have the learning rate as a factor (which is itself a factor of the gradient). This might just mean that the learning rate should be lower or higher with this method.
It would be nice if the introduction made clear that the "Momentum" that "works" is some algorithm and not at all the physical concept "momentum".
Earlier quoted context omitted.
Which part of the math did you find difficult to follow?
Better question: What background is the reader expected to have? Until Xi, not a single variable is defined prior to inclusion into an expression. Even Alpha and Beta are only defined in the header diagram rather than the body of the text. Also, why are the iteration notations in the superscript rather than subscript? And before someone chimes in and says that rudiments are necessary to understand this work, no they…
Earlier quoted context omitted.
Thanks! We're just lucky to have authors like Gabe (@gabrielgoh) come to us with incredible articles. :)
How to know if an article would fit there? For example, I was thinking about adjusting my http://p.migdal.pl/2017/01/06/king-man-woman-queen-why.html (already with some interactive components) or writing about RoI polling ( https://deepsense.io/region-of-interest-pooling-explained/ - by my colleague, but more interactive). Would it be on-topic? (After changing style accordingly.)
In brief, Distill needs to see three things to publish an article: outstanding communication, advancing the dialogue, and scientific integrity. Distill often works with authors to help them bring their articles up to our standards.
Additionally, as a primary publication, Distill will not republish content already published elsewhere, or publish "translations" of papers where someone rewrites the content of a previous paper. (This relates to advancing the dialogue.)
If you reach out to editors@distill.pub, we're happy to discuss pre-submission inquiries about journal scope and related topics.
Earlier quoted context omitted.
How to know if an article would fit there? For example, I was thinking about adjusting my http://p.migdal.pl/2017/01/06/king-man-woman-queen-why.html (already with some interactive components) or writing about RoI polling ( https://deepsense.io/region-of-interest-pooling-explained/ - by my colleague, but more interactive). Would it be on-topic? (After changing style accordingly.)
Please check out our journal policies page: http://distill.pub/journal/ In brief, Distill needs to see three things to publish an article: outstanding communication, advancing the dialogue, and scientific integrity. Distill often works with authors to help them bring their articles up to our standards. Additionally, as a primary publication, Distill will not republish content already published elsewhere, or publish "…
(For some reason I though that this t-SNE article was published elsewhere. Now I see that it was on Distill, but just before its big start.)
Hm. So that helps with high-frequency noise. Any progress on what to do when the dimensions are of vastly different scales? I have an old physics engine which had to solve about 20-value nonlinear differential equations. During a collision, the equations go stiff, and some dimensions may be 10 orders of magnitude steeper than others. Gradient descent then faces very steep knife edges. This is called a "stiff system"…
This could be worth looking into for you.