Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Empiricism and the limits of gradient descent

togelius.blogspot.com

Post reply on HN
Print view View original

41–46 of 46 posts

Re: Empiricism and the limits of gradient descent

#41

Post by goatlover »

mannykannot wrote: post #38
goatlover wrote: post #37

Earlier quoted context omitted.

But how would a neural network know to connect data for mass with data for the measured speed of light? Why would a neural network be looking for an equation for energy conversion in the first place? If you just provide tons of raw data from instruments, what does that mean? What do yo do with it? Sure, a human can clean the data and put it into a format that gives meaningful results. But if we're just talking about…

As someone who finds this interesting but does not know enough to take a position, I think a bigger question would be how does it come up with the abstract concept of energy? I am aware that Alpha Go Zero came up with various strategic abstractions of the game that are recognized by competent players, and some novel ones, but I do not know where this program and its self-play training stands in the dichotomy of this…

Go has a clear objective that you can train for. What would be the objective in coming up with a physics law from raw data? What would it even be training for? That sounds like asking whether DL could create a new board game from a bunch of data on human behavior.

> think a bigger question would be how does it come up with the abstract concept of energy?

I don't see how it could, and that's kind of how Kant argued against empiricism. You can't derive a conceptual understanding of the world from raw data. There's nothing in raw data to structure or make sense of it without some way to interpret the data. Even calling it data is an interpretive act (as opposed to noise).

Re: Empiricism and the limits of gradient descent

#42

Post by Eridrus »

tlb wrote: post #2

I'm optimistic about the potential for evolutionary algorithms. I've used both EAs and gradient descent in developing robot controllers. But the argument here about why gradient descent won't be able to learn certain things is weak. Thought experiments are not a reliable guide to what what GD can or can't do. It's fair enough to say that F=ma and E=mc² aren't in the data. Indeed, it took thousands of years of human t…

Right, I think it's very easy to fall into a trap of thinking that something is too hard for ML methods, because the way we solve them is different than how computers can solve them.

There is ongoing work to try to see if we can learn NN-based controllers that do their own learning or thinking. The Neural Turing Machine work being the most famous, but there's also some amount of single-shot/few-shot learning literature that is exploring other ways of learning because trying to learn from examples via SGD is too slow.

EAs are certainly interesting and I think are show some interesting results for hyperparameter tuning/architecture search of neural nets where we don't have gradients, and doing it with far less computation than RL solutions.

Re: Empiricism and the limits of gradient descent

#43

Post by hmartiros »

vinn124 wrote: post #40
hmartiros wrote: post #9

In my experience if you have even a little smoothness in your problem's cost manifold, taking advantage of gradients is invaluable to sample efficiency. Many losses which don't seem differentiable can be reformulated as such - you can look around and see a wide array of algorithms being put into end-to-end learned frameworks. If the dimensionality is small, second-order methods (or approximations thereof) can do dram…

> Many losses which don't seem differentiable can be reformulated as such... agreed, especially with policy gradients. > If the dimensionality is small, second-order methods (or approximations thereof) can do dramatically better yet. i have not seen second order derivatives in practice, presumably due to memory limitations. can you point me to examples?

They aren't common in deep learning, but if you look to estimation problems like odometry, optimal control, and calibration, the typical approach is to build a least squares estimator that optimizes with a gauss-newton approximation to the Hessian, or other quasi-newton methods. Gradient descent comparatively exhibits very slow convergence in these cases, especially when there is a large condition number. In the case of an actual quadratic loss function, it can (by definition) be solved in one iteration if you have the Hessian. However, getting it efficiently within most learning frameworks is difficult, as they primarily only compute VJPs or HVPs.

Re: Empiricism and the limits of gradient descent

#44

Post by srean »

roywiggins wrote: post #19
simonster wrote: post #14

There are a couple of factual errors here. First, the difference between backprop and evolution is smaller than the author indicates. The error signal used in modern backprop training is stochastic because it is computed on a minibatch (which is why it's called stochastic gradient descent). This stochasticity seems important to achieving good results. And the most popular evolutionary algorithm in the deep learning w…

> Given sufficient data, according to the Universal Approximation Theorem, a neural network can learn to model physics. It just says there are weights to approximate any function, not that you can actually learn the weights. Neural networks trivially can't learn how to approximate noncomputable functions to any accuracy, and there might be a lot of other functions that neural networks are terrible at actually learnin…

I am waiting for this uneducated drivel of explaining NN performance by their 'universal function approximator property' to stop. There are tons other schemes that are also universal approximators, they were known before NN was a thing. Why don't we use those ? Why don't they work as well ?

Learning from examples and generalizing is a much different problem from function approximation.

Re: Empiricism and the limits of gradient descent

#45

Post by sannee »

srean wrote: post #44
roywiggins wrote: post #19

Earlier quoted context omitted.

> Given sufficient data, according to the Universal Approximation Theorem, a neural network can learn to model physics. It just says there are weights to approximate any function, not that you can actually learn the weights. Neural networks trivially can't learn how to approximate noncomputable functions to any accuracy, and there might be a lot of other functions that neural networks are terrible at actually learnin…

I am waiting for this uneducated drivel of explaining NN performance by their 'universal function approximator property' to stop. There are tons other schemes that are also universal approximators, they were known before NN was a thing. Why don't we use those ? Why don't they work as well ? Learning from examples and generalizing is a much different problem from function approximation.

Maybe suggest using polynomials instead of neural networks next time that happens? :)

Re: Empiricism and the limits of gradient descent

#46

Post by thesz »

simonster wrote: post #14

There are a couple of factual errors here. First, the difference between backprop and evolution is smaller than the author indicates. The error signal used in modern backprop training is stochastic because it is computed on a minibatch (which is why it's called stochastic gradient descent). This stochasticity seems important to achieving good results. And the most popular evolutionary algorithm in the deep learning w…

>This stochasticity seems important to achieving good results.

No, it is not and may be counter resultive, so to say.

https://arxiv.org/pdf/1605.02026.pdf - page 8, figure 2(b). SGD optimized neural networks stops learning at the accuracy at which whole-dataset methods start!

Also please note that the figure I pointed to is about high energy particles analysis. SGD trained NN cannot even distinguish particles with good precision, let alone discover physics.

Post reply on HN