I have an alternative proposal: do a study right the first time. That means: A) Pre-registering the study design, including the statistical analysis. Otherwise, attaching a big label "Exploratory! Additional confirmation needed!" B) Properly powering the study. That means gathering a sample large enough that the chances of a false negative aren't just a coin flip. C) Making the data and analysis (scripts, etc.) publi…
Your proposal is comparable to saying that checks and balances are not needed in a democracy, politicians just need to govern "right". This is about incentivising scientists to do the right thing instead of merely demanding it, like you do.
We Should Not Accept Scientific Results That Have Not Been Repeated
281–282 of 282 posts
Re: We Should Not Accept Scientific Results That Have Not Been Repeated
#282Earlier quoted context omitted.
Does it not? If the experimental design is not preserved, don't you open yourself to inconsistent controls between the experiments? Where do you draw the line?
If you think about it, complete replication of any experiment is useless, and impossible. But that doesn't matter, because it is not the experiment you are trying to replicate, is the effect or the observation. Carefully changing the design of the experiment can allow to verify that your explanation of the effect or observation is accurate. Of course, if the change the experiment too much, a failure to replicated wil…
So I think what has to occur is a gradual "loosening up" of the controls from strict replication to weak replication, with both types of replication giving information about the effect you are testing: its validity and its generality, respectively.