Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why don't machine learning research agents overfit?

amazon.science

Post reply on HN
Print view View original

81–90 of 91 posts

Re: Why don't machine learning research agents overfit?

#81

Post by srean »

hodgehog11 wrote: post #63
5555watch wrote: post #53

Earlier quoted context omitted.

Another useful link [0], page 91. Effectively, the more complex the solution, the heavier is the upper bound on true risk. It doesn't mean a simpler model is necessarily better. But the complexity brings its own larger support for mistakes to live in. The _likelihood_ of being _more correct_ is probably related to larger sample required to learn a more complex model. [0]: https://www.cs.huji.ac.il/~shais/Understandin…

Absolutely not. This link is a reference on PAC learning, which is thoroughly misleading in the land of deep learning and inevitably leads to vacuous bounds. This is common knowledge in deep learning. I would not recommend that any student learn any part of this theory at this point, since we have far better alternatives in terms of simplicity, accuracy, and generality. PAC-Bayes is genuinely superior, for example. I…

"Can be misleading" is an accurate characterization.

It is an uniform bound and will have problems with very large hypothesis classes but its statement isnt wrong (just that the tool is a little heavy handed). For simpler models they are adequately useful. So I would suggest new users to learn if they want to use simpler models rather than deep learning methods.

Real world isn't the adversarial/ worst-case that these models of generalization assume. So the generalization performance you experience is usually a better than what PAC indicates, but estimating that experience was not PAC's charter. It's charter was to quantify a adversarial/legal guarantee, the minimum (infimum) guaranteed program.

I too like PAC-Bayes a lot (see my other comments) but it does have a cop out in that by definition you do not know how good your prior is, yet the quantitative estimate of generalization error depends on how good it was.

Nevertheless, PAC-Bayesian and statistical physics based bounds are closer to what we experience, but you can't guarantee that the worst case will not bite you.

Re: Why don't machine learning research agents overfit?

#82

Post by hodgehog11 »

jmalicki wrote: post #74
hodgehog11 wrote: post #62

Earlier quoted context omitted.

This is provably not true, and you can use the marginal likelihood / PAC-Bayes to prove it (or any other framework for measuring model quality). Increase the number of parameters in a linear model way beyond the point of interpolation, and concentrate the likelihood around the zero loss set. Then reduce the variance on a Gaussian prior. You can balance the two temperatures at exactly the right rate so that any measur…

You are saying something interesting, but talking like Grok and skipping a lot of the details, without any references to common check-in points like terminology or specific studies. > and concentrate the likelihood around the zero loss set. Then reduce the variance on a Gaussian prior. Those phrases could mean a lot of different things. What are you proposing? > so that any measure of model quality will monotonically…

Apologies, I'm skipping details, because that's how I speak with my colleagues, but I realize this is an external environment without context. No references since this is folklore (you can look at Hastie et al's Surprises in High-Dimensional Ridgeless Regression paper for the non-Bayesian version, Bruno Loureiro or Andrew Gordon Wilson probably have a paper with something similar).

Concentrating a density around a zero set means that I raise it to the power of 1/gamma (appropriately normalizing) and then take gamma to zero. If the likelihood was Gaussian, this would be equivalent to taking the variance to zero (yielding a point mass). But in overparameterized settings, this concentrates on a submanifold describing the set of interpolating solutions. In least-squares linear regression, that is the solution space. Reducing the variance on a Gaussian prior is treated as an asymptotic expansion by Laplace's method. If you choose the variance to decrease (inversely proportional to the parameter size, for example), then the marginal likelihood will increase monotonically with model size.

By any measure of model size, I mean that you can pick your favourite among the common ones, such as information metrics (e.g. mutual information / KL), statistical metrics (e.g. marginal likelihood), test error. You should be able to show the same phenomenon happening for all of them, so it isn't a quirk of marginal likelihood. It is concentration of measure working in your favor to reduce the variance in the estimator.

Re: Why don't machine learning research agents overfit?

#83

Post by hodgehog11 »

srean wrote: post #80
hodgehog11 wrote: post #62

Earlier quoted context omitted.

This is provably not true, and you can use the marginal likelihood / PAC-Bayes to prove it (or any other framework for measuring model quality). Increase the number of parameters in a linear model way beyond the point of interpolation, and concentrate the likelihood around the zero loss set. Then reduce the variance on a Gaussian prior. You can balance the two temperatures at exactly the right rate so that any measur…

> Then reduce the variance on a Gaussian prior. Are you sure that doing this after seeing the data is valid and does not suffer from the equivalent of peeking-into-the-test-set problem ? There are ways to address the peeking problem but that requires additional machinery. I don't dispute your broad claim but the first counterexample you quote seems problematic.

You can choose the prior according to any selection rule that does not see the data (actually, you can do more, but justifying this is the realm of empirical Bayes and requires some more precise arguments). In this case, you can choose it according to the model size and provided that your Jacobian is full rank, you will get increasing marginal likelihood.

Re: Why don't machine learning research agents overfit?

#84

Post by hodgehog11 »

srean wrote: post #81
hodgehog11 wrote: post #63

Earlier quoted context omitted.

Absolutely not. This link is a reference on PAC learning, which is thoroughly misleading in the land of deep learning and inevitably leads to vacuous bounds. This is common knowledge in deep learning. I would not recommend that any student learn any part of this theory at this point, since we have far better alternatives in terms of simplicity, accuracy, and generality. PAC-Bayes is genuinely superior, for example. I…

" Can be misleading" is an accurate characterization. It is an uniform bound and will have problems with very large hypothesis classes but its statement isnt wrong (just that the tool is a little heavy handed). For simpler models they are adequately useful. So I would suggest new users to learn if they want to use simpler models rather than deep learning methods. Real world isn't the adversarial/ worst-case that thes…

I agree that this is a good nuanced take. However, I find that students who have learned PAC (which usually takes quite some time) often have to unlearn certain principles to do PAC-Bayes, so my comments come from a fair amount of frustration with the topic. On the other hand, I find that teaching students PAC-Bayes from the get-go is easier, it still works for simpler models (you can derive the same PAC bounds, so you can't do any worse), and they get the full story. Obviously for those of us who learned both, it is good to know both. But I am skeptical for the next generation coming through whether it is worth teaching PAC at all.

Re: Why don't machine learning research agents overfit?

#85

Post by vrighter »

diddid wrote: post #8

I always get annoyed when people misinterpret Occam’s razor. It’s not that the simplest is more likely to be correct, it’s that you should prefer it, because it’s simple. It’s just like the Hopper quote. She said it’s better to ask for forgiveness during the fog of war, doing something you thought was right, not to do something you knew they were going to say no to and now you are trying to get away with something.

simpler is not the right word either. it's the one that makes the least assumptions, not the simplest. The simplest would be "god did it" pretty much everytime.

Re: Why don't machine learning research agents overfit?

#86

Post by srean »

hodgehog11 wrote: post #83
srean wrote: post #80

Earlier quoted context omitted.

> Then reduce the variance on a Gaussian prior. Are you sure that doing this after seeing the data is valid and does not suffer from the equivalent of peeking-into-the-test-set problem ? There are ways to address the peeking problem but that requires additional machinery. I don't dispute your broad claim but the first counterexample you quote seems problematic.

You can choose the prior according to any selection rule that does not see the data (actually, you can do more, but justifying this is the realm of empirical Bayes and requires some more precise arguments). In this case, you can choose it according to the model size and provided that your Jacobian is full rank, you will get increasing marginal likelihood.

What threw me off was the (possibly misunderstood) suggestion for minimizing the generalization bound over the prior after the data has been incorporated.

Re: Why don't machine learning research agents overfit?

#87

Post by srean »

hodgehog11 wrote: post #84
srean wrote: post #81

Earlier quoted context omitted.

" Can be misleading" is an accurate characterization. It is an uniform bound and will have problems with very large hypothesis classes but its statement isnt wrong (just that the tool is a little heavy handed). For simpler models they are adequately useful. So I would suggest new users to learn if they want to use simpler models rather than deep learning methods. Real world isn't the adversarial/ worst-case that thes…

I agree that this is a good nuanced take. However, I find that students who have learned PAC (which usually takes quite some time) often have to unlearn certain principles to do PAC-Bayes, so my comments come from a fair amount of frustration with the topic. On the other hand, I find that teaching students PAC-Bayes from the get-go is easier, it still works for simpler models (you can derive the same PAC bounds, so y…

You raise a good point.

I had not considered the pedagogical value of starting with PAC Bayesian bounds. It will be interesting to see ML courses that take that route.

Similar issues show up in traditional statics when considering multi-hypothesis testing. I wonder if betting/gambling over hypotheses might be a more accessible way to convey these ideas to new students.

Re: Why don't machine learning research agents overfit?

#89

Post by bopou »

BenoitEssiambre wrote: post #54
yorwba wrote: post #39

Earlier quoted context omitted.

The notion of "simplicity" can be completely arbitrary, though. It's enough that there are only finitely many hypotheses simpler than the best hypothesis (assuming there's such a thing as a best hypothesis). So as you eliminate hypotheses incompatible with the data, at some point you'll have eliminated all simpler hypotheses, and the simplest hypothesis left will also be the best hypothesis. If simpler hypotheses are…

It doesn't have to be arbitrary. The Kolmogorov/Solomonoff definition, that the simpler models are the shortest programs that can generate what we know about a domain, measured in bits, have a solid mathematical foundation, based on information entropy and even thermodynamic entropy if you cross over into computer hardware. Entropy is a metric targeted in LLM training which is likely why LLMs are overfitting less and…

How is the Kolmogorov definition not arbitrary? It depends on an arbitrary selection of the language you use to write the program.

Re: Why don't machine learning research agents overfit?

#90

Post by jmalicki »

hodgehog11 wrote: post #82
jmalicki wrote: post #74

Earlier quoted context omitted.

You are saying something interesting, but talking like Grok and skipping a lot of the details, without any references to common check-in points like terminology or specific studies. > and concentrate the likelihood around the zero loss set. Then reduce the variance on a Gaussian prior. Those phrases could mean a lot of different things. What are you proposing? > so that any measure of model quality will monotonically…

Apologies, I'm skipping details, because that's how I speak with my colleagues, but I realize this is an external environment without context. No references since this is folklore (you can look at Hastie et al's Surprises in High-Dimensional Ridgeless Regression paper for the non-Bayesian version, Bruno Loureiro or Andrew Gordon Wilson probably have a paper with something similar). Concentrating a density around a ze…

Okay, and that's all in-sample, which is the entire point, it won't necessarily hold out of sample.

E.g. over-fitting.

Post reply on HN