How many medical studies are faked or flawed? • Hacker News Board

Re: How many medical studies are faked or flawed?

#131

Post by ifyoubuildit » Tue, Sep 19, 2023, 8:54 PM UTC

paulddraper wrote: post #69

Solvency wrote: post #13

I often chuckle that same people who vehemently defend vaccine safety studies also decry studies that show processed seed oils, glyphosate, aspartame, and hundreds of other molecules or compounds are unsafe over time. That's the beauty of medical studies! Whether they're good or bad, your own biases mean you can simply ignore the ones that don't align with your opinions. Oh, before I forget! Red meat and saturated fa…

> Red meat and saturated fat is terrible for you! Red meat is a little bit bad for longevity. The majority of the reported effect is correlative. > it's actually sugar that's evil. Sugar is bad but mostly because it's easy to overeat, and obesity is all around terrible for health. > Vegetables are great for you! Oh wait, most vegetables contain oxalates and other defense, mechanisms and compounds that are actually ba…

This is all stated as if it is fact. It sounds believable to me, but how do you (and the rest of us) know that the sources you got this from aren't a part of the junk being called out by the article?

Re: How many medical studies are faked or flawed?

#132

Post by autoexec » Tue, Sep 19, 2023, 8:55 PM UTC

BurningFrog wrote: post #63

Ilverin wrote: post #47

Earlier quoted context omitted.

Well you need peer reviewed papers in highish impact journals to get tenure. For some people, the only way they're getting that is by cheating.

Doing research has a lottery element. You explore something that might reveal an important discovery. And sometimes it just doesn't. That doesn't mean you're a bad scientist, just an unlucky one. But it does mean you can't get tenure. So it's easy to understand why people fake results to secure a career.

> That doesn't mean you're a bad scientist, just an unlucky one. But it does mean you can't get tenure.

That sounds like a really easy problem to solve. Just treat valid science as important regardless of the results. The results shouldn't matter unless they've been replicated and verified anyway.

Re: How many medical studies are faked or flawed?

#133

Post by paulddraper » Tue, Sep 19, 2023, 8:57 PM UTC

ifyoubuildit wrote: post #131

paulddraper wrote: post #69

Earlier quoted context omitted.

> Red meat and saturated fat is terrible for you! Red meat is a little bit bad for longevity. The majority of the reported effect is correlative. > it's actually sugar that's evil. Sugar is bad but mostly because it's easy to overeat, and obesity is all around terrible for health. > Vegetables are great for you! Oh wait, most vegetables contain oxalates and other defense, mechanisms and compounds that are actually ba…

This is all stated as if it is fact. It sounds believable to me, but how do you (and the rest of us) know that the sources you got this from aren't a part of the junk being called out by the article?

Preponderance of the evidence of multiple studies.

Skepticism is reasonable.

Re: How many medical studies are faked or flawed?

#134

Post by MockObject » Tue, Sep 19, 2023, 8:57 PM UTC

When I first grasped how many fake papers are floating around, I gnashed my teeth and called for heinous penalties upon the fraudsters.

After a moment, I thought better, and decided that we should actually offer incentive for fraudulent papers! Liars will always have incentive to lie. When their lies are accepted by those who ought to be more skeptical, it's an indictment of the system, which should be more robust in detecting fraud.

Re: How many medical studies are faked or flawed?

#135

Post by hellotheretoday » Tue, Sep 19, 2023, 8:59 PM UTC

PaulHoule wrote: post #12

function_seven wrote: post #8

Earlier quoted context omitted.

That cynicism you have about those studies is probably because you don't eat enough eggs. Or too many. I'm not sure...

At least I get eggs from a neighbor and not “big egg”; I keep thinking about getting my own chickens but that’s not the way I want to feed Mr. Fox who lives in my neighborhood too.

My neighbor had 7 chickens and a rooster and lost all of them to mr. Fox. It was a tragedy, the eggs were so good.

Re: How many medical studies are faked or flawed?

#136

Post by screye » Tue, Sep 19, 2023, 8:59 PM UTC

ancorevard wrote: post #114

onlyrealcuzzo wrote: post #96

Earlier quoted context omitted.

As someone who works in Data Science at FAANG - if you look hard enough - there is something questionably wrong in every step of the data funnel. And that's when I believe people do have a somewhat best effort to maximize profits. There are plenty of people that only care about career progression and think they can get away with lying and cheating their way to the top. They wouldn't believe that if it didn't work som…

"And that's when I believe people do have a somewhat best effort to maximize profits." Nope. I actually think that if you do scientific research as a company (profit) it may make you less bad/less likely to do fraud compared to academia (non-profit). Reason is that there are more ways to punish you, employees, board, investors, etc in a profit seeking vehicle, and as a profit seeking vehicle being caught must be part…

I have a simpler reason for the same belief.

You can fake everything except a well designed A/B test. At FAANG scale, a statistically significant A/B test requirement will stop the worst fraud before it hits the user.

Re: How many medical studies are faked or flawed?

#137

Post by noslenwerdna » Tue, Sep 19, 2023, 8:59 PM UTC

bjourne wrote: post #22

We all know it is super-easy to cheat in science, but not much can be done about it. Short of requiring replication for every result published which isn't feasible if the study was costly to produce. And the problem isn't confined to medicine either. How many studies in hpc of the type "we setup this benchmark and our novel algorithm/implementation won!" aren't also faked or flawed?

Require experimental analysis to be pre-registered? That is how it's done in particle physics, and it works well.

Re: How many medical studies are faked or flawed?

#138

Post by MockObject » Tue, Sep 19, 2023, 9:01 PM UTC

gmd63 wrote: post #74

gustavus wrote: post #50

Earlier quoted context omitted.

This seems overly simplistic, I mean part of the reason so many studies are so difficult is because they can happen over the period of months or years. This isn't like your High School chemistry experiment. These studies can be massive longitudinal studies that take years to conclude and work on. If the answers were easy I'm sure someone would've implemented it already, it turns out those kind of things are hard. I'd…

I'm interested more in why it specifically would be hard to just show people what you are doing (or did) instead of telling them a story at the end of months or years. Critical details and oversights can be lost in translation between reality and LaTeX that could be easily pointed out by third party observers.

Maybe experiments can take thousands of hours.

Re: How many medical studies are faked or flawed?

#139

Post by wizzwizz4 » Tue, Sep 19, 2023, 9:02 PM UTC

rqtwteye wrote: post #71

adamsb6 wrote: post #66

Earlier quoted context omitted.

The replication crisis is even a thing because of very strong career incentives. You might think that publishing about the replication crisis itself would be great for your career, but perhaps not. Maybe the incentives to be able to bullshit your way to a professorship are so great that no one wants to rock the boat.

"Maybe the incentives to be able to bullshit your way to a professorship are so great that no one wants to rock the boat." Our whole economy is fueled by people bullshitting each other.

Our economy is fuelled by people doing actual work. Large-ish, visible parts are shaped by bullshitters, but that's a different thing.

Re: How many medical studies are faked or flawed?

#140

Post by ifyoubuildit » Tue, Sep 19, 2023, 9:03 PM UTC

Solvency wrote: post #13

I often chuckle that same people who vehemently defend vaccine safety studies also decry studies that show processed seed oils, glyphosate, aspartame, and hundreds of other molecules or compounds are unsafe over time. That's the beauty of medical studies! Whether they're good or bad, your own biases mean you can simply ignore the ones that don't align with your opinions. Oh, before I forget! Red meat and saturated fa…

I dislike "the same people who" comments, but I agree with the sentiment. A lot of us have very little ability to determine the validity of this study over that, but will confidently voice an opinion anyway.

The only way (imo) to stay on firm ground is to acknowledge that someone published a thing saying xyz, and maybe that you are x% convinced by it. Can't get too far out over your skis going that route.