11–20 of 165 posts
From the article: Although Decety’s paper had reported that they had controlled for country, they had accidentally not controlled for each country, but just treated it as a single continuous variable so that, for example “Canada” (coded as 2) was twice the “United States” (coded as 1). I mean I don't even understand how this seemed like a normal thing to do?
From the article: Although Decety’s paper had reported that they had controlled for country, they had accidentally not controlled for each country, but just treated it as a single continuous variable so that, for example “Canada” (coded as 2) was twice the “United States” (coded as 1). I mean I don't even understand how this seemed like a normal thing to do?
You've never had a bug in your code that seemed insane after the fact?
"The paper received a great deal of attention, and was covered by over 80 media outlets including The Economist, the Boston Globe, the Los Angeles Times, and Scientific American." And how many of these will cover the retraction? A dozen at most? And all those articles will be sitting out there, getting cited and read on occasion.
From the article: Although Decety’s paper had reported that they had controlled for country, they had accidentally not controlled for each country, but just treated it as a single continuous variable so that, for example “Canada” (coded as 2) was twice the “United States” (coded as 1). I mean I don't even understand how this seemed like a normal thing to do?
From the article: Although Decety’s paper had reported that they had controlled for country, they had accidentally not controlled for each country, but just treated it as a single continuous variable so that, for example “Canada” (coded as 2) was twice the “United States” (coded as 1). I mean I don't even understand how this seemed like a normal thing to do?
Many algorithms work on both categorical and continuous variables, with different results depending on the variable's type.
Furthermore, this whole field was called cargo cult science by Feynman, using correlations, p-hacking, data dredging, and more
https://en.wikipedia.org/wiki/Multiple_comparisons_problem
The best analysis I have seen of the corner we have painted ourselves into
https://slatestarcodex.com/2014/04/28/the-control-group-is-o...
How did this not get caught immediately? If I did a study and found out that kids in Zambia are 47 more times as generous as American kids that'd make me instantly suspicious.
Or maybe the reviewers were all Canadian /s
Earlier quoted context omitted.
You've never had a bug in your code that seemed insane after the fact?
This seems like 101, 'another pair of experienced eyes would catch this' kind of malpractice.
You have:
- people with no coding experience and, in some cases (especially in social sciences), a strong aversion to math
- code that isn't unit tested, so answering the question, "Did it run correctly?" is often softened into, "Does this look plausible to me?"
- a strong incentive to end up with certain results
I dated a quantitative geneticist for a while, and her coding education was almost zero. She was writing code in R and essentially just changing lines until the output "looked right". It was insanely complicated math, so there was no way to make sure the output was good. The code had to be an exact match for the algorithm she had written out in mathematical notation, and there was essentially no chance of that.
It got worse. She'd write the algorithm in R and then end up with batches that would take, in some cases, years to finish running. Obviously she ended up with even more dubious hacks.
(For anyone curious, she's had a fairly decorated academic career under an acclaimed advisor who reviewed all of this code to some extent, and she's worked with most of the top genetics programs in the US.)
From the article: Although Decety’s paper had reported that they had controlled for country, they had accidentally not controlled for each country, but just treated it as a single continuous variable so that, for example “Canada” (coded as 2) was twice the “United States” (coded as 1). I mean I don't even understand how this seemed like a normal thing to do?
I think a major contributing factor to problems like this is people going into the soft/social sciences being more likely to be math/stats AND programming averse. Meanwhile, all sciences continue their long term trend towards applied math via programming. This leads to people using the math/stats via code without understanding very well what it is they are doing, and, naturally, the end result is lots and lots of mistakes.