Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Statistical challenges and misreadings of literature create unreplicable science [pdf]

stat.columbia.edu

Post reply on HN
Print view View original

41–50 of 57 posts

Re: Statistical challenges and misreadings of literature create unreplicable science [pdf]

#41

Post by renewiltord »

mjburgess wrote: post #8

We're increasingly aware today of how the media operates cycles of self-referential and self-justifying citations: a TV show will quote an article that reports "some people" taking an issue, which ends up being a quote from someone interviewed for another newspaper article.. and so on. This "legitimacy laundering" is rampant, and we're now getting towards media literacy levels which expose it for many people. However…

This is the danger with non-empirical government funded “science”. Empiricism allows verification. Private funding means that things that matter get studied.

If you trace the oft-cited claim from “studies” that cats kill so and so animals a year you will find it’s just someone’s Fermi estimate. And you’ll find that the person has a personal distaste for housecats.

Re: Statistical challenges and misreadings of literature create unreplicable science [pdf]

#42

Post by dash2 »

mrguyorama wrote: post #40
pacman001 wrote: post #14

Earlier quoted context omitted.

Every single time (some 4 times in my past life) when I’m familiar with/close to the background story of something that played out in the news I come to the same conclusion: news doesn’t report the facts. I’m in Western Europe, btw. I advise to run this experiment yourself.

No journalist will ever have the expertise required to accurately report scientific results. Instead we need to help average people understand that when the news says "Scientists say drinking red wine is healthy!" that No actual scientist ever said that! Instead, the journalist writing the segment cribbed notes from the University's PR page about the study, which was also written by someone with zero science backgrou…

I think science journalists often have some science background, and university press releases are in my experience collaborations between the scientist and the PR person.

Re: Statistical challenges and misreadings of literature create unreplicable science [pdf]

#43

Post by littlestymaar »

zorked wrote: post #25
littlestymaar wrote: post #19

Earlier quoted context omitted.

That's the problem when you have people responsible for news having the business model of ad-sposored entertainment and not fact reporting.

In Western Europe a lot of the media isn't ad-sponsored.

I live in France, and saying that 90% of the media are ad-sponsored is a conservative estimate.

So I don't know which part of western Europe you're talking about, but it clearly doesn't apply to all of it.

Re: Statistical challenges and misreadings of literature create unreplicable science [pdf]

#44

Post by rscho »

renewiltord wrote: post #41
mjburgess wrote: post #8

We're increasingly aware today of how the media operates cycles of self-referential and self-justifying citations: a TV show will quote an article that reports "some people" taking an issue, which ends up being a quote from someone interviewed for another newspaper article.. and so on. This "legitimacy laundering" is rampant, and we're now getting towards media literacy levels which expose it for many people. However…

This is the danger with non-empirical government funded “science”. Empiricism allows verification. Private funding means that things that matter get studied. If you trace the oft-cited claim from “studies” that cats kill so and so animals a year you will find it’s just someone’s Fermi estimate. And you’ll find that the person has a personal distaste for housecats.

The idea that private funding means that what is studied matters is highly dubious at best. Most private research pertains to application and product development, and often relies on fundamental discoveries coming from academic labs. Only academia can afford to let scientists run loose. It certainly results in a good amount of inapplicable theories, but is nevertheless very much essential.

Re: Statistical challenges and misreadings of literature create unreplicable science [pdf]

#45

Post by renewiltord »

rscho wrote: post #44
renewiltord wrote: post #41

Earlier quoted context omitted.

This is the danger with non-empirical government funded “science”. Empiricism allows verification. Private funding means that things that matter get studied. If you trace the oft-cited claim from “studies” that cats kill so and so animals a year you will find it’s just someone’s Fermi estimate. And you’ll find that the person has a personal distaste for housecats.

The idea that private funding means that what is studied matters is highly dubious at best. Most private research pertains to application and product development, and often relies on fundamental discoveries coming from academic labs. Only academia can afford to let scientists run loose. It certainly results in a good amount of inapplicable theories, but is nevertheless very much essential.

I doubt gender studies is essential.

Re: Statistical challenges and misreadings of literature create unreplicable science [pdf]

#46

Post by nialse »

mjburgess wrote: post #38
naasking wrote: post #37

Earlier quoted context omitted.

Prediction and explanation are basically equivalent IMO. A predictive model entails an explanation, and an explanation entails a predictive model. Predictive models that are more accurate are more accurate explanations, and predictive models that are more precise are more precise explanations, and vice versa. They are not that distinct.

They are highly distinct. Compare reporting the temperature tomorrow as a mode of all temperatures in November at your location, with a climate & weather simulation involving: cloud layers, the ocean, etc. The former is likely to be vastly more predictively accurate than the latter, but explains nothing. Explanatory models are often less predictively accurate than these (weakly inductive) predictive models. Their pur…

I enjoyed your conversation and just want to chip in that there are as many definitions of science and knowledge as there are philosophers. One don’t have to have only one definition, but usually one have to adhere to the ones within the realm of ones scientific paradigm to be accepted and to develop the science. Normal science as Kuhn called it.

Re: Statistical challenges and misreadings of literature create unreplicable science [pdf]

#47

Post by [deleted] »

nialse wrote: post #46
mjburgess wrote: post #38

Earlier quoted context omitted.

They are highly distinct. Compare reporting the temperature tomorrow as a mode of all temperatures in November at your location, with a climate & weather simulation involving: cloud layers, the ocean, etc. The former is likely to be vastly more predictively accurate than the latter, but explains nothing. Explanatory models are often less predictively accurate than these (weakly inductive) predictive models. Their pur…

I enjoyed your conversation and just want to chip in that there are as many definitions of science and knowledge as there are philosophers. One don’t have to have only one definition, but usually one have to adhere to the ones within the realm of ones scientific paradigm to be accepted and to develop the science. Normal science as Kuhn called it.

[deleted]

Re: Statistical challenges and misreadings of literature create unreplicable science [pdf]

#48

Post by mjburgess »

nialse wrote: post #46
mjburgess wrote: post #38

Earlier quoted context omitted.

They are highly distinct. Compare reporting the temperature tomorrow as a mode of all temperatures in November at your location, with a climate & weather simulation involving: cloud layers, the ocean, etc. The former is likely to be vastly more predictively accurate than the latter, but explains nothing. Explanatory models are often less predictively accurate than these (weakly inductive) predictive models. Their pur…

I enjoyed your conversation and just want to chip in that there are as many definitions of science and knowledge as there are philosophers. One don’t have to have only one definition, but usually one have to adhere to the ones within the realm of ones scientific paradigm to be accepted and to develop the science. Normal science as Kuhn called it.

If you look at the history of these accounts of science though, they are "pre modern" in a negative sence.

We didnt have formal methods of causal analysis to the 1920s, and it took til the 80s to have a real robust formalism and account of causal analysis.

Before, experimenters would "have in their heads" the causal knowledge of how to conduct and interpret these results -- but this was never formalised, or given explicitly.

So accounts of science before this "causal revolution" of the late 20th C. are broken, and based on a philosopher's literal reading of scientific experiments (and the like) with little undersatnding of how the experimenters actaully thought about them -- and philosophers often doctrinally opposed to entertaining these thoughts seriously.

Today the partition between science/engineering, prediction/explanation, etc. can be given on much more robust grounds, and there's few philosophers of science working that adopt positions wholly at odds with my account. What you find from ML-ists is humean conceptions which ideologically honour their curve-fitting methods with equivalent status to actual experimentation and explanation

Re: Statistical challenges and misreadings of literature create unreplicable science [pdf]

#49

Post by visres »

mrguyorama wrote: post #40
pacman001 wrote: post #14

Earlier quoted context omitted.

Every single time (some 4 times in my past life) when I’m familiar with/close to the background story of something that played out in the news I come to the same conclusion: news doesn’t report the facts. I’m in Western Europe, btw. I advise to run this experiment yourself.

No journalist will ever have the expertise required to accurately report scientific results. Instead we need to help average people understand that when the news says "Scientists say drinking red wine is healthy!" that No actual scientist ever said that! Instead, the journalist writing the segment cribbed notes from the University's PR page about the study, which was also written by someone with zero science backgrou…

You might like the BBC's Science in action https://www.bbc.co.uk/programmes/p002vsnb/episodes/downloads - the presenter Roland Pease has a scientific background and each episode he interviews the actual researchers on several interesting findings/publications for that week.

Re: Statistical challenges and misreadings of literature create unreplicable science [pdf]

#50

Post by naasking »

mjburgess wrote: post #38
naasking wrote: post #37

Earlier quoted context omitted.

Prediction and explanation are basically equivalent IMO. A predictive model entails an explanation, and an explanation entails a predictive model. Predictive models that are more accurate are more accurate explanations, and predictive models that are more precise are more precise explanations, and vice versa. They are not that distinct.

They are highly distinct. Compare reporting the temperature tomorrow as a mode of all temperatures in November at your location, with a climate & weather simulation involving: cloud layers, the ocean, etc. The former is likely to be vastly more predictively accurate than the latter, but explains nothing. Explanatory models are often less predictively accurate than these (weakly inductive) predictive models. Their pur…

> The former is likely to be vastly more predictively accurate than the latter, but explains nothing.

It actually explains quite a bit, most importantly that weather is cyclical with only statistically minor variations around the mode.

This predictive model is also very specific, rather than general. There are plenty of "explanations" that are also not predictive, like that "Thor creates thunder".

Explanations and predictive models that are both general and minimize parameters are better, and they are interchangeable.

Post reply on HN