Empiricism and the limits of gradient descent
togelius.blogspot.com
Empiricism and the limits of gradient descent
1–10 of 46 posts
Re: Empiricism and the limits of gradient descent
#2I'm optimistic about the potential for evolutionary algorithms. I've used both EAs and gradient descent in developing robot controllers.
But the argument here about why gradient descent won't be able to learn certain things is weak. Thought experiments are not a reliable guide to what what GD can or can't do.
It's fair enough to say that F=ma and E=mc² aren't in the data. Indeed, it took thousands of years of human thought to arrive at them. So the argument "it's not clear how an algorithm could extract F=ma from the data" isn't a strong criticism, because humans also can't do it by induction.
The long process culminating in F=ma involved a lot of abstract symbolic thought. Whether human-level abstract symbolic thought can be learned through GD (probably in combination with some sort of Monte Carlo tree search) is an open question. It can only be answered by trying to build things and seeing if they work.
If you want to make an argument about the limits of GD and induction, it'd be better to compare to a problem humans can solve reliably, rather than an insight that one genius had after decades of thought while standing on the shoulders of other geniuses.
Re: Empiricism and the limits of gradient descent
#3I'm optimistic about the potential for evolutionary algorithms. I've used both EAs and gradient descent in developing robot controllers. But the argument here about why gradient descent won't be able to learn certain things is weak. Thought experiments are not a reliable guide to what what GD can or can't do. It's fair enough to say that F=ma and E=mc² aren't in the data. Indeed, it took thousands of years of human t…
I don't understand how E=mc^2 can not be in the data. If it's a universal law, isn't it in more (all) data than any pattern that isn't universal?
Re: Empiricism and the limits of gradient descent
#4I don't see why stochastic has to be worse than evolutionary algorithms. In high-dimensional spaces, there is a large space of possible "mutations". SGD just biases certain mutations based on the gradient from a minibatch. That sounds a lot like evaluating the fitness of a mutated population on a minibatch and culling members with low fitness. In fact, there are many demonstrations that the stochastic nature of SGD (coming from the use of minibatches) is crucial for effective learning.
Re: Empiricism and the limits of gradient descent
#5The points the author makes about gradient descent are accurate, in a sense. However, they oversimplify the technique (as it is currently applied today) and the context in which it is used. It seems as if the author, like many others, has a basic understanding of the subject's basic mechanisms, but not the context in which experts understand them.
The example the author cites regarding evo algorithms learning physical laws is laughable - "It's just not in the data - it has to be invented" applies equally to both the backprop and the evolutionary learning algorithms.
"In this case, the representation (mathematical expressions represented as trees) is distinctly non-differentiable, so could not even in principle be learned through gradient descent."
This is incorrect, almost like saying NLP data is not differentiable. For instance, set this representation up as the output of a network (or, if you wanted to be fancier, the central component of an autoencoder), and see how well it predicts/correlates with the experimental data. This is the error, which is back-propagated through the network's nodes.
FWIW, many theoreticians believe that the unreasonable effectiveness of neural networks and especially transfer learning is a result of their well-suitedness to encode laws of physics and Euclidean geometry. The author's final points about a nine-year-old survey may be out of date w.r.t. contemporary neural networks, which often have spookily good local minima and do not behave in the way intuition about gradient descent might suggest.
Re: Empiricism and the limits of gradient descent
#6I'm optimistic about the potential for evolutionary algorithms. I've used both EAs and gradient descent in developing robot controllers. But the argument here about why gradient descent won't be able to learn certain things is weak. Thought experiments are not a reliable guide to what what GD can or can't do. It's fair enough to say that F=ma and E=mc² aren't in the data. Indeed, it took thousands of years of human t…
I don't understand how E=mc^2 can not be in the data. If it's a universal law, isn't it in more (all) data than any pattern that isn't universal?
Data has to be obtainable and is often dimensional.
How would you have measured the speed of light in the forties? Even having lots of test cases, you wouldn't be able to deduce something like that with test cases alone.
Re: Empiricism and the limits of gradient descent
#7Earlier quoted context omitted.
I don't understand how E=mc^2 can not be in the data. If it's a universal law, isn't it in more (all) data than any pattern that isn't universal?
Data has to be obtainable and is often dimensional. How would you have measured the speed of light in the forties? Even having lots of test cases, you wouldn't be able to deduce something like that with test cases alone.
Speed of light has been experimentally determined long before the 40s. The discovery of the speed of light being constant lead to the special relativity (aka before GR which is the E=mcc). http://www.speed-light.info/measure/speed_of_light_history.h...
Re: Empiricism and the limits of gradient descent
#8I'm optimistic about the potential for evolutionary algorithms. I've used both EAs and gradient descent in developing robot controllers. But the argument here about why gradient descent won't be able to learn certain things is weak. Thought experiments are not a reliable guide to what what GD can or can't do. It's fair enough to say that F=ma and E=mc² aren't in the data. Indeed, it took thousands of years of human t…
I don't understand how E=mc^2 can not be in the data. If it's a universal law, isn't it in more (all) data than any pattern that isn't universal?
The article is making a simpler point than that. If I show you the table:
A B C
1 4 3
20 45 15
8 15 7
Re: Empiricism and the limits of gradient descent
#9In my experience if you have even a little smoothness in your problem's cost manifold, taking advantage of gradients is invaluable to sample efficiency. Many losses which don't seem differentiable can be reformulated as such - you can look around and see a wide array of algorithms being put into end-to-end learned frameworks. If the dimensionality is small, second-order methods (or approximations thereof) can do dramatically better yet. However, I'm also a fan of evolutionary algorithms. I see no reason why evolutionary rules can't be defined with awareness of gradient signals.
Re: Empiricism and the limits of gradient descent
#10I'm optimistic about the potential for evolutionary algorithms. I've used both EAs and gradient descent in developing robot controllers. But the argument here about why gradient descent won't be able to learn certain things is weak. Thought experiments are not a reliable guide to what what GD can or can't do. It's fair enough to say that F=ma and E=mc² aren't in the data. Indeed, it took thousands of years of human t…
In wikipedia and numerous references, I get the impression that EAs, evolutionary algorithms, are a huge and fuzzily defined field.
With Genetic algorithms, just one kind of evolutionary algorithm, the fitness, the mutation and the crossover function seem to require the implementor to look at the problem domain and "come up with something" whereas once you have a goal, gradient descent requires lots of tuning but is more or less defined, you can track how well you're doing and so-forth.
Perhaps there's something I'm missing. Pointers would be welcome.
Looked at: https://en.wikipedia.org/wiki/Evolutionary_algorithm https://en.wikipedia.org/wiki/Genetic_algorithm