Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We Should Not Accept Scientific Results That Have Not Been Repeated

nautil.us

Post reply on HN
Print view View original

181–190 of 282 posts

Re: We Should Not Accept Scientific Results That Have Not Been Repeated

#181

Post by unfamiliar »

Tloewald wrote: post #175
return0 wrote: post #79

Earlier quoted context omitted.

We don't need to 'define repetition', we need to foster a culture that a) accepts repetition and b) does not accept something for a fact just because it's in a journal. Right now, (a) is not even acceptable; nobody will publish a replication study. Ofc, LHC is impossible to replicate, but the vast majority of life science studies are. I should think this is mostly needed in life sciences. Other, more 'exact' sciences…

Does doing 50 studies and only publishing the 3 that produce favorable results count as repetition? Because, BTW, that's exactly what Big Pharma does, and also intrinsically what the bias towards publishing only significant results does.

> Because, BTW, that's exactly what Big Pharma does

That's exactly what almost everyone in academia does. Money is not the only corrupting factor.

Re: We Should Not Accept Scientific Results That Have Not Been Repeated

#185

Post by jerf »

bbctol wrote: post #92
jerf wrote: post #88

Earlier quoted context omitted.

"I don't think reproducibility should necessarily mean literally repeating the same study over again." Yes, it should. We have too many demonstrated instances of studies built on other studies that turned out to be flawed, but the second study, rather than showing the first one was flawed, was rationalized and massaged until it conformed to the first study because the first study already had the imprimatur of peer-re…

But there should also be replication of the concepts of the study under a different paradigm, otherwise you might just be finding out effects of the particular procedure. An original study is rarely "wrong" (fraud is a serious problem, but not a typical one), it's that it's merely one data point, from one procedure, and a lot of the time the effect that can be replicated is "real," it's just a side effect of the proc…

Yes. But replication is a starting point. It isn't just about fraud, it's also about error and failure to communicate the experiment. (It's also beyond time for paper journals to just contain the relevant summaries and conclusions for given studies and for "what fits in a journal" to cease being "what fits in publication". Authors ought to be able to publish as much stuff as they want with a paper, including full source code sets or data sets or whatever, with the paper limitations affecting only the top-level summary.)

Re: We Should Not Accept Scientific Results That Have Not Been Repeated

#186

Post by odbol_ »

mmierz wrote: post #120

I see a lot of people commenting here that there's no incentive to repeat previous research because it's not useful for getting grants, etc. This kind of true but I think it misses something important. At least in life sciences (can't comment on other fields), it's not that scientists don't repeat each other's results. After all, if you're going to invest a significant fraction of your tiny lab budget on a research p…

This is why I'm always skeptical of people zealously claiming "all GMOs are perfectly safe! It's been proven!"

Yeah, proven by expensive studies that were funded by the company making the GMO. Who is going to pay for another study to try to disprove it?

There's a reason we sprayed DET all over our vegetables for years before it was banned: there was no scientific studies proving that it was harmful, even though it clearly was harmful in hindsight.

Science is not instant, and there's no way someone can claim that some brand-new GMO is "perfectly safe", without any long-term studies on its effects over 10, 20, 30 years of exposure. That's just not possible. And yet you try to explain it to these science zealots and they just brush you off as being "anti-science".

Re: We Should Not Accept Scientific Results That Have Not Been Repeated

#187

Post by pc2g4d »

Replication isn't enough. It's also necessary to know how many non-replications have occurred but got swept under the rug. It's not the existence of replications that matter---it's the rate of replication relative to number of replication attempts.

So I agree with the title "We Should Not Accept Scientific Results That Have Not Been Repeated". But I would add to it "We Should Not Accept Scientific Results from Studies That Weren't Preregistered". Registration of studies forces negative results to be made public, allowing for the positive result rate / replication rate to be calculated.

Otherwise the existence of a "positive" result is more a function of the trendiness of a research area than it is of the properties of the underlying system being studied.

Re: We Should Not Accept Scientific Results That Have Not Been Repeated

#188

Post by yiyus »

> The inconvenient truth is that scientists can achieve fame and advance their careers through accomplishments that do not prioritize the quality of their work

An even more inconvenient truth is that scientists cannot even keep their jobs if they prioritize the quality of their work. The pressure to publish novel results is too strong and it is almost impossible to get any support for confirming previous ones.

Re: We Should Not Accept Scientific Results That Have Not Been Repeated

#189

Post by johncolanduoni »

anotherhacker wrote: post #163
misnome wrote: post #59

Define repetition. It's not as simple as that, for all sciences - once again an article on repeatability seems to have focused on medicinal drug research (it's usually that or psychology), and labelled the entire "Scientific community" as 'rampant' with " statistical, technical, and psychological biases". How about, Physics? The LHC has only been built once - it is the only accelerator we have that has seen the Higgs…

The difference is maths. E.g. Einstein proved his theory of gravity mathematically, THEN it was proved via empirical observation. We don't have a way to mathematically prove a drug for depression a psychological phenomena.

What does "prove mathematically" mean here? He constructed a self-consistent mathematical theory, and validated that it reproduced both the special relativistic and Newtonian limits, but I don't think "proved mathematically" is a good way to state that.

You can't "mathematically prove" a theory agrees with reality except by reconciling it with experiments (possibly indirectly through links to previous theories), so the only difference here is that a non-ad-hoc mathematical theory of depression medication doesn't appear to be feasible, not that physics has some magic alternate method of proof.

Re: We Should Not Accept Scientific Results That Have Not Been Repeated

#190

Post by Madmallard »

undergroundOps wrote: post #7

I'm a physician and I've been suggesting this to my colleagues for a few years, only to be met with alienated stares and labeled cynical. The doctors and doctors-in-training I work with have altruistic motives, but place too much stock in major medical studies. They also frequently apply single-study findings to patient care, even to patients that would've been excluded from that study (saw this a lot with the recent…

Can I have you as my PCP?

I had a severe reaction to fluoroquinolones and maybe some confounding comorbidities and have been pretty much unable to get effective help in our medical system so far :(

Post reply on HN