Live data from Hacker News

How many medical studies are faked or flawed?

web.archive.org

211–220 of 257 posts

Re: How many medical studies are faked or flawed?

#211

While we're on the subject, how many studies period are faked or flawed to the point of being useless? It seems to me that the scientific community's reaction to the replication crisis has been to ignore it.

Check out https://retractionwatch.com/ , a continuous stream of exposés on authors publishing flawed or fraudulent works.

Re: How many medical studies are faked or flawed?

#212
post #177

Earlier quoted context omitted.

"Trust the science" is dumb, but "trust that the single-most reviewed medical studies of the past decade with 30k+ participants aren't outright bullshit" is enough for me. The risk for those studies is probably closer to what happened with Astrazeneca, where a very rare side effect got missed because 30k people weren't enough to detect it. I think skepticism is healthy to a degree, but ironically, the more adversaria…

> "Trust the science" is dumb, but "trust that the single-most reviewed medical studies of the past decade with 30k+ participants aren't outright bullshit" is enough for me. What does "aren't outright bullshit" translate into in quantitative (%) terms?

I'd say bullshit means data that's worthless, because it's fabricated, collected by P-hacking, or otherwise means nothing. There's no useful quantitative measurement that tells you more than "Professor Bullshit just filled out random junk in Excel."

Re: How many medical studies are faked or flawed?

#213
post #136

Earlier quoted context omitted.

"And that's when I believe people do have a somewhat best effort to maximize profits." Nope. I actually think that if you do scientific research as a company (profit) it may make you less bad/less likely to do fraud compared to academia (non-profit). Reason is that there are more ways to punish you, employees, board, investors, etc in a profit seeking vehicle, and as a profit seeking vehicle being caught must be part…

I have a simpler reason for the same belief. You can fake everything except a well designed A/B test. At FAANG scale, a statistically significant A/B test requirement will stop the worst fraud before it hits the user.

FAANG engineers: “hold my beer”

Seriously though, as a person who has built related systems at FAANG, yes this problem exists there. Your beautiful cathedral of an A/B testing framework is covered in knobs that are just perfect for p-hacking.

Re: How many medical studies are faked or flawed?

#214
post #177
post #173

Earlier quoted context omitted.

Ah, but don’t worry folks, those vaccines which generated $40B in profits for Pfizer and Moderna, those are just A-ok. Trust the science, which in this case is perfect, unassailable and uncorruptible (unlike everything else the pharma industry does). Don’t question the faith.

"Trust the science" is dumb, but "trust that the single-most reviewed medical studies of the past decade with 30k+ participants aren't outright bullshit" is enough for me. The risk for those studies is probably closer to what happened with Astrazeneca, where a very rare side effect got missed because 30k people weren't enough to detect it. I think skepticism is healthy to a degree, but ironically, the more adversaria…

Severe side effects or death cases were excluded from the Pfizer phase 3 study by conveniently shifting blame on patients, inventing diagnoses or claiming they had a Covid infection.

Even if all that were not intentional, it would certainly raise the question why this should be the only case of fabricating data for studies. Since the RNA-vaccines are obviously not as positively effective as claimed, what else is there that might be not talked about? And lastly why is the topic still that controversial when the safety and effectivity claims have not stood up to the real world test?

Re: How many medical studies are faked or flawed?

#215
post #185

Earlier quoted context omitted.

Serving as editor of a major journal in any discipline can make one cynical. I read many years ago a quote from a mathematics editor that a large fraction of published proofs were faulty, and from a computer science editor that a large fraction of published theorems were false. (My foggy memory gives 1/2 and 1/3 for the fractions, but I don't remember which was which.)

Almost nobody has the throughput to check equations at all

You mean the 10 pages of the proofs located in the Appendix aren't read? I hate CS academia...

Re: How many medical studies are faked or flawed?

#216

Earlier quoted context omitted.

I mean I’m as vaccinated as anyone, but the originally reported 95% effectiveness against symptomatic COVID-19 with the monovalent vaccine just doesn’t seem realistic in hindsight. Who knows what complex set of incentives was at play. If you’re running a trial site and your business relies on continued contracts from Pfizer, are you reporting every cough, fever, or headache?

I don't think the claim was 95% against symptomatic covid, it was against hospitalization or death. The vaccine was tested against the original virus but Omicron was almost a different disease with much more immune escape. We're lucky the vaccine held up as well as it did. This is also why the alternative doctors that wanted everyone to catch covid to build up herd immunity were wrong, but nobody seems to bring them…

Whether the original study said so or not, that is absolutely how it was universally sold to the public. This [1] is a collection of various high profile individuals talking about the efficacy of the vaccines, why that means you won't carry the virus, you won't get sick, and and how that will completely stop the spread of COVID.

It's only as this very obviously failed to be the case that the metric was completely shifted to hospitalization/death. I'd also add this is about the time that the 'public messaging' swapped from talking about efficacy and other topics to outright vitriol and attacks on unvaccinated individuals, which is probably where the politicization of the topic began.

[1] - https://twitter.com/i/status/1472327161352798212

Re: How many medical studies are faked or flawed?

#217
post #196
post #177

Earlier quoted context omitted.

"Trust the science" is dumb, but "trust that the single-most reviewed medical studies of the past decade with 30k+ participants aren't outright bullshit" is enough for me. The risk for those studies is probably closer to what happened with Astrazeneca, where a very rare side effect got missed because 30k people weren't enough to detect it. I think skepticism is healthy to a degree, but ironically, the more adversaria…

Peer review is not a magic spell that reveals the truth. “Most-reviewed” is meaningless here if the underlying data was not collected or reported properly.

Peer review, no. Review by a substantial percentage of world-wide immunologists along with everyone who's even remotely interested and a bunch of people who believe your work is killing people, still not a magic spell, but it's a hell of a lot better.

Re: How many medical studies are faked or flawed?

#218

Earlier quoted context omitted.

I mean I’m as vaccinated as anyone, but the originally reported 95% effectiveness against symptomatic COVID-19 with the monovalent vaccine just doesn’t seem realistic in hindsight. Who knows what complex set of incentives was at play. If you’re running a trial site and your business relies on continued contracts from Pfizer, are you reporting every cough, fever, or headache?

I don't think the claim was 95% against symptomatic covid, it was against hospitalization or death. The vaccine was tested against the original virus but Omicron was almost a different disease with much more immune escape. We're lucky the vaccine held up as well as it did. This is also why the alternative doctors that wanted everyone to catch covid to build up herd immunity were wrong, but nobody seems to bring them…

There is no need to think, you can go read it. There was no indication that it was any help against severe disease, hospitalization, or death. The touted 95% was indeed against some definition of “symptomatic Covid” (a subjective one). In the all cause mortality section, there were more deaths in the vaccine arm than in the placebo arm. (But that was not statistically significant). At any time before 2020, the study would have been laughed at. But scientific standards were thrown out the door in 2020, and have not yet returned. (And … as it turns out from this and similar studies e.g. Ioannidis’ seminal paper showing most research is wrong from 2005 - it wasn’t as good as most people thought in 2020 already)

Re: How many medical studies are faked or flawed?

#219
post #196
post #177

Earlier quoted context omitted.

"Trust the science" is dumb, but "trust that the single-most reviewed medical studies of the past decade with 30k+ participants aren't outright bullshit" is enough for me. The risk for those studies is probably closer to what happened with Astrazeneca, where a very rare side effect got missed because 30k people weren't enough to detect it. I think skepticism is healthy to a degree, but ironically, the more adversaria…

Peer review is not a magic spell that reveals the truth. “Most-reviewed” is meaningless here if the underlying data was not collected or reported properly.

Yeah, because hundreds of health authorities simultaneously collecting their own data (often with larger groups of each cohort) wouldn't be able to replicate or disprove the outcome of the original research.

Re: How many medical studies are faked or flawed?

#220

Earlier quoted context omitted.

I don't think the claim was 95% against symptomatic covid, it was against hospitalization or death. The vaccine was tested against the original virus but Omicron was almost a different disease with much more immune escape. We're lucky the vaccine held up as well as it did. This is also why the alternative doctors that wanted everyone to catch covid to build up herd immunity were wrong, but nobody seems to bring them…

Whether the original study said so or not, that is absolutely how it was universally sold to the public. This [1] is a collection of various high profile individuals talking about the efficacy of the vaccines, why that means you won't carry the virus, you won't get sick, and and how that will completely stop the spread of COVID. It's only as this very obviously failed to be the case that the metric was completely shi…

> Whether the original study said so or not, that is absolutely how it was universally sold to the public.

Trust the politics, blame the science?

Post reply on HN