Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Statistical challenges and misreadings of literature create unreplicable science [pdf]

stat.columbia.edu

Post reply on HN
Print view View original

51–57 of 57 posts

Re: Statistical challenges and misreadings of literature create unreplicable science [pdf]

#51

Post by mjburgess »

naasking wrote: post #50
mjburgess wrote: post #38

Earlier quoted context omitted.

They are highly distinct. Compare reporting the temperature tomorrow as a mode of all temperatures in November at your location, with a climate & weather simulation involving: cloud layers, the ocean, etc. The former is likely to be vastly more predictively accurate than the latter, but explains nothing. Explanatory models are often less predictively accurate than these (weakly inductive) predictive models. Their pur…

> The former is likely to be vastly more predictively accurate than the latter, but explains nothing. It actually explains quite a bit, most importantly that weather is cyclical with only statistically minor variations around the mode. This predictive model is also very specific, rather than general. There are plenty of "explanations" that are also not predictive, like that "Thor creates thunder". Explanations and pr…

As I've said, merely predictive models do not quantify over the features of reality.

A weakly inductive model is simply this: the next case will be some average of the prior cases because we assume unknown aspects of the environment will ensure it is so.

An explanatory model tells you why. It says: there are planets, gravity, molecules, mountains, germs, atoms, and so on.

The semantics of explanations are causal properties of reality, and their relationship.

Explanations do not have "parameters" to "minimise". The universal law of gravitation is not a curve fit to data. There was never a dataset of F,M,m,r; netwon did not fit any parameters -- there are no such parameters.

Re: Statistical challenges and misreadings of literature create unreplicable science [pdf]

#52

Post by naasking »

mjburgess wrote: post #51
naasking wrote: post #50

Earlier quoted context omitted.

> The former is likely to be vastly more predictively accurate than the latter, but explains nothing. It actually explains quite a bit, most importantly that weather is cyclical with only statistically minor variations around the mode. This predictive model is also very specific, rather than general. There are plenty of "explanations" that are also not predictive, like that "Thor creates thunder". Explanations and pr…

As I've said, merely predictive models do not quantify over the features of reality. A weakly inductive model is simply this: the next case will be some average of the prior cases because we assume unknown aspects of the environment will ensure it is so. An explanatory model tells you why . It says: there are planets, gravity, molecules, mountains, germs, atoms, and so on. The semantics of explanations are causal pro…

Explanations absolutely do have parameters, called premises, and we absolutely do try to minimize them. There's a whole principle about it called Occam's razor.

Re: Statistical challenges and misreadings of literature create unreplicable science [pdf]

#53

Post by light_hue_1 »

mjburgess wrote: post #26
light_hue_1 wrote: post #24

Earlier quoted context omitted.

And my PhD and dayjob is doing research on this. If a student told me they had this view of what ML is, I would tell them that we've failed to educate them. The thought that physics doesn't care about performance or approximation is silly. Just look at AlphaFold. Heck, I talk to climatologists and material scientists that want the equivalent all the time. Prediction is the heart of all science. Whether we're talking…

Prediction is not the heart of science, this is early 20th C. mumbojumbo and humean nonesense that gets repeated by curve-fitters because it's all they do. Explanation is the heart of science, not prediction. All predictions newton would have made of the orbits of the planets would have been wrong (and so on). And this goes for the vast majority of textbooks physics when its applied to very many ordinary situations:…

> There is no textbook of physics which models reality by saying, "Well, we suppose in the future, the positions and velocities will just follow the same distribution, but we've no idea why, and how dare you ask, and get out, and doesnt my Ideal-Gas-TransformerModel look pretty? It gets the pressure right for Argon at 20.0001 C in glass jars at about 2.002L"

That's precisely what physics textbooks say!

Put most famously by Feynman (originally by Mermin): "Shut up and calculate."

Entanglement is nonsense? Too bad it works. There's no explanation. Don't like the notion of virtual particles that violate the conservation of energy? Too bad, it just works. Don't like notion of wave function collapse and that we have no mathematical model of it. Too bad, it works. etc.

Explanations are models. Models can be mental models. They can be mathematical models. They can be computational models. But all that matters is that they make good predictions. Nothing else.

It's totally irrelevant whether you or anyone else personally feels they're provided with an explanation or not.

Re: Statistical challenges and misreadings of literature create unreplicable science [pdf]

#54

Post by mjburgess »

naasking wrote: post #52
mjburgess wrote: post #51

Earlier quoted context omitted.

As I've said, merely predictive models do not quantify over the features of reality. A weakly inductive model is simply this: the next case will be some average of the prior cases because we assume unknown aspects of the environment will ensure it is so. An explanatory model tells you why . It says: there are planets, gravity, molecules, mountains, germs, atoms, and so on. The semantics of explanations are causal pro…

Explanations absolutely do have parameters, called premises, and we absolutely do try to minimize them. There's a whole principle about it called Occam's razor.

Those are propositions with existential quantifications, not parameters, and we do not minimise them

We use comparisons over the richness of existential commitments in theory comparison, but NOT in the creation of theories nor in their specification

Re: Statistical challenges and misreadings of literature create unreplicable science [pdf]

#55

Post by naasking »

mjburgess wrote: post #54
naasking wrote: post #52

Earlier quoted context omitted.

Explanations absolutely do have parameters, called premises, and we absolutely do try to minimize them. There's a whole principle about it called Occam's razor.

Those are propositions with existential quantifications, not parameters, and we do not minimise them We use comparisons over the richness of existential commitments in theory comparison, but NOT in the creation of theories nor in their specification

These are all distinctions without a difference when analyzing theory descriptions under a unified framework, like ordering them by Kolmogorov complexity, eg. a Turing machine description is equivalent to a universal Turing machine+a suitably encoded tape.

Re: Statistical challenges and misreadings of literature create unreplicable science [pdf]

#56

Post by mjburgess »

naasking wrote: post #55
mjburgess wrote: post #54

Earlier quoted context omitted.

Those are propositions with existential quantifications, not parameters, and we do not minimise them We use comparisons over the richness of existential commitments in theory comparison, but NOT in the creation of theories nor in their specification

These are all distinctions without a difference when analyzing theory descriptions under a unified framework, like ordering them by Kolmogorov complexity, eg. a Turing machine description is equivalent to a universal Turing machine+a suitably encoded tape.

Yes, indeed, at the level of abstraction in which everything is syntax without any semantics, then everything is the same.

That's as content free as saying, "since the sun is ordered, and my CD collection is ordered, the sun and my CDs are the same".

This "unified framework" you're talking about is just an analysis of functions over whole numbers.

Re: Statistical challenges and misreadings of literature create unreplicable science [pdf]

#57

Post by naasking »

mjburgess wrote: post #56
naasking wrote: post #55

Earlier quoted context omitted.

These are all distinctions without a difference when analyzing theory descriptions under a unified framework, like ordering them by Kolmogorov complexity, eg. a Turing machine description is equivalent to a universal Turing machine+a suitably encoded tape.

Yes, indeed, at the level of abstraction in which everything is syntax without any semantics, then everything is the same. That's as content free as saying, "since the sun is ordered, and my CD collection is ordered, the sun and my CDs are the same". This "unified framework" you're talking about is just an analysis of functions over whole numbers.

Everything is not the same, but everything can be quantified and ordered by some metrics which may discard some irrelevant info, yes. I disagree that it's content-free, there is probably only syntax at a fundamental level so whatever the semantics you've created in your head, they also reduce to syntax in the end. There are little to no semantics in the standard model of particle physics after all.
Post reply on HN