Earlier quoted context omitted.
"The real trick is to see how well your model extrapolates from the data you have out into the future." That is the most common way to show the modeller is not shamelessly overfitting.:-| Another way, though, is less common but not vanishingly uncommon: the model may be so much simpler than the data it fits that overfitting is not a plausible explanation. (Roughly there are too many bits of entropy in the match to th…
Well - that's true apart from co-incidence. You can have a very simple theory which says "x is directly caused by y" and there is a lot of good data, and a great fit. But it's kist a co-incidence and breaks down immediately. Occam's razor is a rule of thumb and an aesthetic boon, but nothing more. The real test is that you have a theory that is meaningful and has explanatory power. If it grants insight on the mechani…
Occam's razor is a bit more than that. It isn't just that given a theory X and a theory Y = X + ε, both of which fit the facts, you should prefer X because it's "cleaner" or more aesthetically pleasing or whatever. You should prefer X because you can prove it is more likely to be true.
https://en.wikipedia.org/wiki/Solomonoff%27s_theory_of_induc...