I'm sorry to say that I've personally witnessed some people in the next lab commit fraud. It was investigated and someone was fired, but I can believe that it is all too common.
How many medical studies are faked or flawed?
141–150 of 257 posts
Re: How many medical studies are faked or flawed?
#142Earlier quoted context omitted.
Objectivity and honesty can be hard to find if all someone cares about is their reputation as a competent researcher or climbing the ladder. What do you think a potential solution would be for this? I feel like even in my own experience, trying something out and it not working feels like failure, when in fact to proclaim it a success or "fix" it is truly what harms both the endeavor for truth and the people reliant o…
> Objectivity and honesty can be hard to find if all someone cares about is their reputation as a competent researcher or climbing the ladder. It seems like destroying the reputation and career of people who fake science would be a great start. If you're willing to fake data and lie to get results, there will always be an industry who'd love to hire you no matter how tarnished your reputation is. We need a better mea…
> destroying the reputation and career of people who fake science would be a great start
When so many reports are faulty and fraudulent, that might instead be the great start of destroying the careers of those who would have revealed the fraudulent research?
I wonder what'd happen if researchers got compensated and funding based on other things, unrelated to papers published. But what would that be
Re: How many medical studies are faked or flawed?
#143Isn't this an issue with the reviews rather than the publication attempts? Wouldn't the normal (if there is such a thing) approach between people be to credit a reviewer with the quality of their reviews or lack thereof? Basically, who signed off on it?
That is my understanding. Papers can be wrong for different reasons, some nefarious, some honest mistakes. The problem I wouldn't say is with the paper itself. It more of people citing it with a lack of peer review or lack of replication. And also peer review being like a stereotypical "LGTM" approval on a pull request.
Re: How many medical studies are faked or flawed?
#144Earlier quoted context omitted.
"And that's when I believe people do have a somewhat best effort to maximize profits." Nope. I actually think that if you do scientific research as a company (profit) it may make you less bad/less likely to do fraud compared to academia (non-profit). Reason is that there are more ways to punish you, employees, board, investors, etc in a profit seeking vehicle, and as a profit seeking vehicle being caught must be part…
I have a simpler reason for the same belief. You can fake everything except a well designed A/B test. At FAANG scale, a statistically significant A/B test requirement will stop the worst fraud before it hits the user.
And also - you can somewhat take care of bugs by evenly distributing them to your test & control group.
Re: How many medical studies are faked or flawed?
#145While we're on the subject, how many studies period are faked or flawed to the point of being useless? It seems to me that the scientific community's reaction to the replication crisis has been to ignore it.
My internal filter is: Sociology/Psych/Economics are almost all junk. Their conclusions may or may not be correct. Medical studies are mostly junk. There's way too much financial incentive to show marginal improvement. Theraflu and anti-depressents come to mind. Both show a small effect in studies and launched billion dollar businesses. Hard science stuff tends to be pretty good. Mostly just outright fraud and they u…
Chemistry still has a ton of issues with people exaggerating numbers that can't be contested like yield and purity of reactions even among the top journals. Straight up fraud is rarer yes.
Re: How many medical studies are faked or flawed?
#146Earlier quoted context omitted.
> While we're on the subject, how many studies period are faked or flawed to the point of being useless? Or how many studies are useless, period? It's like publishing a memoir to Amazon. You can now say "author" on your resume, or when you're introduced or at cocktail parties but nobody finds any value in what you have to say. You can also use ChatGPT because people might not notice.
There is always value in expanding the breadth and depth of human knowledge, even if it doesn't seem useful to you, right now. That of course assumes that knowledge is true, which is the crux of the problem now.
Unfortunately, the preponderance of very low value research in the literature puts a significant burden on the scientists who have to sift through a lot of garbage to find what they're looking for. Even if the work is ostensibly correct (much of it is not), it really doesn't do anyone much good, except for the authors of course. But now every undergraduate and every parent's vanity project at Andover wants a first author contrib., so here we are.
Re: How many medical studies are faked or flawed?
#147Earlier quoted context omitted.
You don't need "big pharma" for this. Researchers at universities also do these kinds of things because it helps them advance their careers.
What's your point? We have "big pharma" for this. Do you think CocaCola and the Sacklers had their own unique ideas shared by no one else? That we've filtered all scrupulous people out of industry? Scruples are an abstraction at that scale.
Is the Coca-Cola mention about their campaign to influence CDC research and policy [0], and pro-obesity astroturf [1][2]?
[0]: Politico "Coca-Cola tried to influence CDC on research and policy, new report states" [https://www.politico.com/story/2019/01/29/coke-obesity-sugar...]
[1]: "Evaluating Coca-Cola’s attempts to influence public health ‘in their own words’: analysis of Coca-Cola emails with public health academics leading the Global Energy Balance Network" https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10200649/
[2]: Forbes: "Emails Reveal How Coca-Cola Shaped The Anti-Obesity Global Energy Balance Network" https://www.forbes.com/sites/nancyhuehnergarth/2015/11/24/em...
Re: How many medical studies are faked or flawed?
#148"For more than 150 trials, Carlisle got access to anonymized individual participant data (IPD). By studying the IPD spreadsheets, he judged that 44% of these trials contained at least some flawed data: impossible statistics, incorrect calculations or duplicated numbers or figures, for instance. And in 26% of the papers had problems that were so widespread that the trial was impossible to trust, he judged — either bec…
I'd like to point out that the study [1] reported 16 trials from the USA with just one "zombie/fatally flawed" trial. e.g. 6%
Most of the studies that were problematic came from China and Egypt.
In other words, nothing new here.
[1] https://associationofanaesthetists-publications.onlinelibrar...
Re: How many medical studies are faked or flawed?
#149Earlier quoted context omitted.
I think you misread the article slightly. The 26% of zombie papers are inside the 44% of papers that contained at least some flawed data. Look at the figure just below this quote where the blue bar, indicating papers he thought were ok, covers more than 50%.
Which, ironically, just shows how easy it is to make mistakes not just in complicated statistical methods, but in basic interpretation of what numbers mean in the first place.
not to mention that everyone here forgets to mention that the majority of submitted papers with suspected fraud came from countries with known issues with fraudulent science publishing. e.g. China.
The USA, for example, looked much better (although not perfect) at 6% zombie.
Re: How many medical studies are faked or flawed?
#150Earlier quoted context omitted.
Doing research has a lottery element. You explore something that might reveal an important discovery. And sometimes it just doesn't. That doesn't mean you're a bad scientist, just an unlucky one. But it does mean you can't get tenure. So it's easy to understand why people fake results to secure a career.
> That doesn't mean you're a bad scientist, just an unlucky one. But it does mean you can't get tenure. That sounds like a really easy problem to solve. Just treat valid science as important regardless of the results. The results shouldn't matter unless they've been replicated and verified anyway.
Too easy. Define "valid" though.