Earlier quoted context omitted.
Hi, I'm the author. Just wanted to pop in and say: (1) I originally had the thing stuffed full of citations and links, but since they wanted to print it in the paper magazine, we stripped all those out. If you want a reference for any particular claim, I'm happy to provide. (2) First Things didn't pressure me to make any content or editorial modifications to the article. The sole exception was one or two more technic…
It might be helpful to compile a list of those citations online (your own website, perhaps?). This is the sort of essay that might get shared around a whole bunch....it would be a shame for its impact to be lessened because of the inevitable judgment people will have for the hosting publication. It may be a perfect example of the "secular bias" the publication references, but I too groaned inwardly when I went to che…
Scientific Regress
11–20 of 56 posts
Re: Scientific Regress
#12As much as I think the essay is great, this paragraph is a terrible ad-hominem argument for the existence of a "cult of science" or why it is dangerous. It doesn't matter if people wear xkcd shirts, Matt Damon uses profanity with "science", or if people like a facebook page. Laypeople having fun with "science" does not make a cult. A better argument would be to show how people cargo-cult science, and why it causes problems with science. The author fails to do both.
IFL is dangerous not because it is only pictures of scientific phenomena, but that bullshit is constantly posted on the page to be consumed by laypeople. The cult of science is dangerous because it has failed to teach people how to properly evaluate claims, and believe anything with the word "study" in it. Woo-pushers have appropriated the vocabulary of science indistinguishable to a layperson. Andrew Wakefield resurrected long dead pandemics by falsely publishing in the Lancet. This is what makes the cult of science dangerous, not the word "bitches".
Re: Scientific Regress
#13Re: Scientific Regress
#14Re: Scientific Regress
#15Certainly, the reproducibility crisis in social science and cancer biology is important, and deserves attention. But this article mostly strikes me as profoundly overstated. Yes, science is done by humans, and is thus imperfect. Yes, science's approach to truth is nonmonotonic. (I think most working scientists would consider this obvious.) Yes, scientists have various incentives to hype surprising results, cut corner…
Re: Scientific Regress
#16Certainly, the reproducibility crisis in social science and cancer biology is important, and deserves attention. But this article mostly strikes me as profoundly overstated. Yes, science is done by humans, and is thus imperfect. Yes, science's approach to truth is nonmonotonic. (I think most working scientists would consider this obvious.) Yes, scientists have various incentives to hype surprising results, cut corner…
It's academia that is broken, which is being referred to here as 'science'. What do we replace it with? Organizations that do not damage actual science in order perpetuate themselves, obviously.
Re: Scientific Regress
#17TLDR: some scientists are doing science by critiquing what some other scientists did, using statistics. The authors are a wee bit confused about the fact that science is needed to weed out the 'bad' science from the 'good' science.
Re: Scientific Regress
#18I think this is largely a case of 'moloch' as described by Scott Alexander: http://slatestarcodex.com/2014/07/30/meditations-on-moloch/ In that long (but excellent essay) Scott points out that self organizing systems like human society can sometimes arrange themselves in horrible local minimas which can be very difficult to escape. I think the way modern science is organized is decidedly suboptimal. Note that I'm tal…
1: http://slatestarcodex.com/2014/04/28/the-control-group-is-ou...
Re: Scientific Regress
#19First, I think that the most prominent studies are the most likely to have issues; generally they're doing something new, often with new methodology. It's here where extrapolations tend to be made that are the most dangerous. It's also not surprising that social sciences have such issues; they don't have rigorous tools (i.e. genetics) that can effectively ground their work. It makes compounding issues much harder to catch.
Related is the re-testing issue; in many fields, subsequent work will catch errors in previous work. If you work with a mutant and then someone else works with it, they'll see if it behaves differently than expected from the previous work. Germplasm travels, and it's the ultimate arbiter of truth. This usually does lead to further scrutiny and fixing the issue. The real problem that I've observed isn't that the errors aren't caught, but that a formal retraction isn't always done. Sometimes it just gets contradicted in a subsequent paper (and often with some relish) without ever resulting in a retraction. The editors of that journal clearly have a responsibility here that they are failing to uphold.
However, despite these faults, it's quite clear to those of us with 'boots on the ground' that you can't hide from the data; as long as you're using solid genetics and doing 'real' experiments (e.g. western blots, in situ/immuno-localization, simple gels/pcr, etc.) you can only hide for so long. The exception is if no one keeps working on it, in which case it's probably not that interesting to begin with. This also leads to a deep suspicion of bioinformatics among geneticists because we see how often things go wrong, and what is needed to make it right. Fortunately, genetics can still be used to great effect.
Ultimately, I'm not suspicious of the large body of work in my field. Most of it is based on extremely solid forward genetics that withstands lots of testing. Even now, great value is placed on these 'old school' methods because of how robust they are known to be.
Re: Scientific Regress
#20I am a post-doctoral research scientist that has wanted to be a scientist since early childhood (6-7 years old). The structure of how science is practiced seems to have changed substantially in that time frame, not for the better. Some of my childhood naivete plays a role in those perceptions, but many older scientists share that perspective. A lot of the problems in modern science revolve around the growing focus on…
One way to bypass this would be for larger organizations to internally incentivize their researchers to cite their own organizations papers in preference to, or in addition to, other organizations papers. Thus the larger the organization, the more it can press down on the PI scales.
Alternately, even if it's not a several large organizations, smaller organizations can incentivize citing partner papers in the same way.
Which is just restating the concept that if you measure something, and then base monetary rewards off that measurement, then that number will be hyper-optimized (even if it's not a good predictor of future performance).
I don't have a good alternative, but just mentioning that there are easy circumventions (like you mentioned with double blinding)