Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why don't machine learning research agents overfit?

amazon.science

Post reply on HN
Print view View original

51–60 of 71 posts

Re: Why don't machine learning research agents overfit?

#51

Post by yorwba »

srean wrote: post #42
yorwba wrote: post #39

Earlier quoted context omitted.

The notion of "simplicity" can be completely arbitrary, though. It's enough that there are only finitely many hypotheses simpler than the best hypothesis (assuming there's such a thing as a best hypothesis). So as you eliminate hypotheses incompatible with the data, at some point you'll have eliminated all simpler hypotheses, and the simplest hypothesis left will also be the best hypothesis. If simpler hypotheses are…

Except for the fact that eventually we are all dead. So it is kind of important to get there faster. For complicated hypotheses, where complicated is defined appropriately, it takes many many examples to realize that it was a wrong hypothesis all along. There lies the rub. For a particular instance of a learning problem we can't tell much, however using a Occams razor over many instances, one would be correct more of…

To be clear, I was doing a reductio ad absurdum. PAC is precisely the kind of theoretical framework that concerns itself with asymptotic long-run behavior. An appropriate definition of simplicity certainly gets you to the goal much faster than the worst-case bound. For example, you could order hypotheses from most to less likely. But that would render the claim that simpler hypotheses are more likely rather tautological.

More typical definitions of simplicity, e.g. using the bit length of some kind of natural encoding, aren't guaranteed to offer any special advantages in terms of likelihood. So if you have prior knowledge that a particular hypothesis is more likely than another, but the less likely one is simpler in an intuitive sense, you shouldn't let that override you, but still prefer the more likely option. (And you don't need to take the circuitous route of coming up with a new encoding where the more likely hypothesis has a shorter bit length, either.)

Re: Why don't machine learning research agents overfit?

#52

Post by Xcelerate »

sigbottle wrote: post #35

Compression in this modern day and age is so slop. Yes, I'm familiar with keystone results such as Solomonoff induction. It's a direct counterexample to compression - your intensional algorithm can completely outrun reality. I can literally specify a huge mega-algorithm that just searches over all possible Turing machines and evaluates them, and it's an optimal compressor. It's completely vacuous though. You can alwa…

I’m guessing your downvotes were for tone? You’re correct though regarding Solomonoff induction, as the choice of reference universal partial recursive function gives drastically different results for predictions based on finite data (even with access to a halting oracle). Asymptotically, any choice eventually converges to the same predictions, but that’s no help when there are infinitely many choices for U and no obvious natural prior over universal functions. And I don’t find the argument that our natural environment “implements some choice of U” particularly convincing. There’s definitely an open mystery there.

Re: Why don't machine learning research agents overfit?

#53

Post by 5555watch »

srean wrote: post #23
diddid wrote: post #8

I always get annoyed when people misinterpret Occam’s razor. It’s not that the simplest is more likely to be correct, it’s that you should prefer it, because it’s simple. It’s just like the Hopper quote. She said it’s better to ask for forgiveness during the fog of war, doing something you thought was right, not to do something you knew they were going to say no to and now you are trying to get away with something.

I think you should get less annoyed. > It’s not that the simplest is more likely to be correct, it’s that you should prefer it, because it’s simple. I don't know what Occam meant, but if you accept the formalism of PAC learning, it is more likely to be correct https://web.archive.org/web/20170428225156/http://www.cse.bu... https://web.archive.org/web/20130412062821/http://cs.ecs.bay...

Another useful link [0], page 91. Effectively, the more complex the solution, the heavier is the upper bound on true risk. It doesn't mean a simpler model is necessarily better. But the complexity brings its own larger support for mistakes to live in. The _likelihood_ of being _more correct_ is probably related to larger sample required to learn a more complex model.

[0]: https://www.cs.huji.ac.il/~shais/UnderstandingMachineLearnin...

Re: Why don't machine learning research agents overfit?

#54

Post by BenoitEssiambre »

yorwba wrote: post #39
srean wrote: post #23

Earlier quoted context omitted.

I think you should get less annoyed. > It’s not that the simplest is more likely to be correct, it’s that you should prefer it, because it’s simple. I don't know what Occam meant, but if you accept the formalism of PAC learning, it is more likely to be correct https://web.archive.org/web/20170428225156/http://www.cse.bu... https://web.archive.org/web/20130412062821/http://cs.ecs.bay...

The notion of "simplicity" can be completely arbitrary, though. It's enough that there are only finitely many hypotheses simpler than the best hypothesis (assuming there's such a thing as a best hypothesis). So as you eliminate hypotheses incompatible with the data, at some point you'll have eliminated all simpler hypotheses, and the simplest hypothesis left will also be the best hypothesis. If simpler hypotheses are…

It doesn't have to be arbitrary. The Kolmogorov/Solomonoff definition, that the simpler models are the shortest programs that can generate what we know about a domain, measured in bits, have a solid mathematical foundation, based on information entropy and even thermodynamic entropy if you cross over into computer hardware.

Entropy is a metric targeted in LLM training which is likely why LLMs are overfitting less and less as they improve and why they subjectively seem to obey Occam's razor.

There's lesson for software engineering in general behind that kind of simplicity. https://benoitessiambre.com/entropy.html

Re: Why don't machine learning research agents overfit?

#55

Post by fwip »

exit wrote: post #32
vatsachak wrote: post #25

No point in reading anything AI related anymore. It's all slop. We need to retvrn to rss feeds

what would returning to rss feeds achieve?

The pattern that RSS feeds best supports is subscribing to individual creators who produce things you value. Presumably, if you don't like AI word-vomit, you wouldn't subscribe to a blog that uses it heavily. This is in contrast to the social-media state of affairs (like HN, reddit, twitter, etc), wherein much of the posts you read are recommended to you by either a person or an algorithm, regardless of whether you believe that author will write something you like.

A shorter way of writing this is that RSS promotes writing stuff good enough to get into somebody's "favorite authors" list, and social media instead incentivizes "lots of people will share this."

Re: Why don't machine learning research agents overfit?

#56

Post by vatsachak »

fwip wrote: post #55
exit wrote: post #32

Earlier quoted context omitted.

what would returning to rss feeds achieve?

The pattern that RSS feeds best supports is subscribing to individual creators who produce things you value. Presumably, if you don't like AI word-vomit, you wouldn't subscribe to a blog that uses it heavily. This is in contrast to the social-media state of affairs (like HN, reddit, twitter, etc), wherein much of the posts you read are recommended to you by either a person or an algorithm, regardless of whether you b…

I actually couldn't have said it better. I'm brain rotted from LLM usage

Re: Why don't machine learning research agents overfit?

#57

Post by srean »

yorwba wrote: post #51
srean wrote: post #42

Earlier quoted context omitted.

Except for the fact that eventually we are all dead. So it is kind of important to get there faster. For complicated hypotheses, where complicated is defined appropriately, it takes many many examples to realize that it was a wrong hypothesis all along. There lies the rub. For a particular instance of a learning problem we can't tell much, however using a Occams razor over many instances, one would be correct more of…

To be clear, I was doing a reductio ad absurdum . PAC is precisely the kind of theoretical framework that concerns itself with asymptotic long-run behavior. An appropriate definition of simplicity certainly gets you to the goal much faster than the worst-case bound. For example, you could order hypotheses from most to less likely. But that would render the claim that simpler hypotheses are more likely rather tautolog…

PAC isn't asymptotic in general. It gives finite estimates for finite sizes of training data.

If you say, and I think you are indeed saying so, that PAC is ridiculously pessimistic, I would be in violently agreement with you. That's one reason why for practical training data sizes and practical (infinite version space) hypothesis classes PAC gives bound such as -- probability of error is less than 41K. This isn't exactly incorrect but not very useful or informative.

A far more useful formulation is PAC-Bayesian where you get error bounds less than 1 guaranteed and usually less than 0.5 on reasonable sized training data sets.

Re: Why don't machine learning research agents overfit?

#58

Post by fragmede »

diddid wrote: post #28
gowld wrote: post #13

Earlier quoted context omitted.

That's not true. It's pretty clear that she meant "do something you knew they were going to say no to and now you are trying to get away with something." https://youtu.be/wHdHCoeUbU4?t=861s > So I want to tell something to all the young people here on many many occasions you'll find it is much easier to apologize than it is to get permission. You do it then when somebody comes after you and say are you supposed to do…

But I still don’t think that means eat all the cookies in the cookie jar and then apologize after because nobody would have given permission. That’s still about doing what you believe to be right. She even frames the fallout as “where you supposed to do that?” and not “you shouldn’t have done that”.

Analogies only go so far, but I think eating the cookies in the cookie jar is the wrong one, because you're only feeding yourself with that one. If you're stealing flour from the baker to make a surprise cake for the whole troupe to enjoy, then you can apologize later, instead of asking first and ruining the surprise.

Re: Why don't machine learning research agents overfit?

#59

Post by ubutler »

If anything, the latest generation of AI models, Astra and Fable, are prime example of overfitting—whereas benchmarks suggest they’re AGI-tier, users (including myself) report the same old gaslighting, hallucination, context rot, cheating, incomprehensibility patterns as with prior models, sometimes even more pronounced.

Fable and Opus 5, I suspect, will become textbook examples of RL collapse.

Re: Why don't machine learning research agents overfit?

#60

Post by benlivengood »

sigbottle wrote: post #35

Compression in this modern day and age is so slop. Yes, I'm familiar with keystone results such as Solomonoff induction. It's a direct counterexample to compression - your intensional algorithm can completely outrun reality. I can literally specify a huge mega-algorithm that just searches over all possible Turing machines and evaluates them, and it's an optimal compressor. It's completely vacuous though. You can alwa…

I think that the ratio of work done by priors and search is the interesting question at this point. We aren't quite at the point where reading off the most likely hypothesis decompressed from a transformer is sufficient, but it's a lot closer than I originally suspected when we were ~solving chess. I think AlphaGo was kind of the watershed moment that prior-guided search was so much better than either alone.

I think the adversarial policies against Go AIs directly show the gap between intelligence and compression/priors.

Post reply on HN