Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How many medical studies are faked or flawed?

web.archive.org

Post reply on HN
Print view View original

231–240 of 257 posts

Re: How many medical studies are faked or flawed?

#231

Post by mike_hearn »

treis wrote: post #103
some_random wrote: post #18

While we're on the subject, how many studies period are faked or flawed to the point of being useless? It seems to me that the scientific community's reaction to the replication crisis has been to ignore it.

My internal filter is: Sociology/Psych/Economics are almost all junk. Their conclusions may or may not be correct. Medical studies are mostly junk. There's way too much financial incentive to show marginal improvement. Theraflu and anti-depressents come to mind. Both show a small effect in studies and launched billion dollar businesses. Hard science stuff tends to be pretty good. Mostly just outright fraud and they u…

I used to think like that. Soft science = dubious, hard science = ok. Not any more though. Just recently there was LK99 in materials science though, which was pretty much the textbook definition of an exciting but non-replicable claim.

Also a lot of stuff is tough to classify. Are epidemiology or climatology "hard" sciences?

Re: How many medical studies are faked or flawed?

#232

Post by mharig »

fer wrote: post #219
avalys wrote: post #196

Earlier quoted context omitted.

Peer review is not a magic spell that reveals the truth. “Most-reviewed” is meaningless here if the underlying data was not collected or reported properly.

Yeah, because hundreds of health authorities simultaneously collecting their own data (often with larger groups of each cohort) wouldn't be able to replicate or disprove the outcome of the original research.

You trust health authorities? Come on.

Re: How many medical studies are faked or flawed?

#233

Post by throwoutway »

strken wrote: post #217
avalys wrote: post #196

Earlier quoted context omitted.

Peer review is not a magic spell that reveals the truth. “Most-reviewed” is meaningless here if the underlying data was not collected or reported properly.

Peer review, no. Review by a substantial percentage of world-wide immunologists along with everyone who's even remotely interested and a bunch of people who believe your work is killing people, still not a magic spell, but it's a hell of a lot better.

Did they review the raw data or did they just read the paper? AFAIK, Pfizer did not release the "summaries" of the data until March 2022 (forced by the FDA if i recall), long after these debates reached critical mass. I don't believe they released the data itself still. Google makes this hard to search for because it's still censoring...

Re: How many medical studies are faked or flawed?

#234

Post by RoyalHenOil »

jonlucc wrote: post #191
ancorevard wrote: post #114

Earlier quoted context omitted.

"And that's when I believe people do have a somewhat best effort to maximize profits." Nope. I actually think that if you do scientific research as a company (profit) it may make you less bad/less likely to do fraud compared to academia (non-profit). Reason is that there are more ways to punish you, employees, board, investors, etc in a profit seeking vehicle, and as a profit seeking vehicle being caught must be part…

Completely anecdotally, I find this to be true. I do pre-clinical research (disclaimer: for a pharma company), and if you hand me a paper describing some research result in my field, it’s much more likely to reproduce in my hands if it’s from industry. I know from talking to my academic research friends that there are a ton of shortcuts, rotating inexperienced staff, and misaligned incentives. I’m not saying pharma i…

I used to work in agricultural R&D in the private sector, and we collaborated with several universities.

The biggest difference I saw was that universities were very short-term focused, while we were more long-term focused.

The universities had a constant churn of personnel, as new PhD candidates appeared and old ones left, whereas our own researchers and technicians stuck around for far longer.

Additionally, they were so hyperfocused on grants and papers that they tended to not put as much effort into replication, since that didn't pay their bills. By comparison, we typically repeated our experiments ad nauseum; it was common for us to perform the same experiment twice a year (once in the northern hemisphere and then again in the southern hemisphere) for a decade or more, gradually iterating and refining our processes along the way.

Even if we initially got negative results, we'd beat that dead horse for a few years to make sure. Occasionally it turned out not to be so dead after all.

Re: How many medical studies are faked or flawed?

#235

Post by stefantalpalaru »

Llamamoe wrote: post #221
riperoni wrote: post #214

Earlier quoted context omitted.

Severe side effects or death cases were excluded from the Pfizer phase 3 study by conveniently shifting blame on patients, inventing diagnoses or claiming they had a Covid infection. Even if all that were not intentional, it would certainly raise the question why this should be the only case of fabricating data for studies. Since the RNA-vaccines are obviously not as positively effective as claimed, what else is ther…

Early into the pandemic, the original COVID strains had like 40% rate of permanent aftereffects, and the hospitalisation and death rates were insane. At the time, the vaccines have definitely saved lives, and the "covered up severe side effects or death" you talk about never came up in the real world, so...

[dead]

Re: How many medical studies are faked or flawed?

#236

Post by PaulHoule »

the_af wrote: post #102
PaulHoule wrote: post #93

Earlier quoted context omitted.

The animals we're mostly concerned about at our farm are horses and cats. I don't think there is anything left in North America that can trouble a horse, unless you count mosquito-transmitted infectious diseases. I hear foxes are not dangerous to cats but I believe we've lost some to coyotes. Mostly we've had people around, either tenants or neighbors, who keep chickens so we don't have to. I'll say the eggs from a s…

> I believe we've lost some to coyotes Am I out of place if I say I also like coyotes? (Remember: city boy, so nothing is at stake for me here. I also like wolves!) And yeah, Japanese folklore has taught me that it's best to avoid kitsune. Though they sometimes turn into magical women who help you?

Really I don't take predation personally.

I don't tell the SPCA, however, that I am getting a cat there because it's a replacement for one lost to predators. They die of old age, get hyperthyroidism, high blood pressure, become blind, walk around mindlessly and crash into things, and one morning you find them dead at the bottom of the stairs. I think they would find dying at the claws of a predator an honorable death.

As for kitsune they are the Japanese version of a myth that is widespread in East and South Asia, for instance Daji

https://en.wikipedia.org/wiki/Daji

is probably isomorphic to Tamamo-no-mae

https://en.wikipedia.org/wiki/Tamamo-no-Mae

and is sometimes believed to be the same entity. I was into anime for a long time (Urusei Yatsura was a watershed but it really goes back to seeing Star Blazers on TV) but lately I have been into Chinese pop culture like Three Kingdoms, Nezha and Wolf Warrior 2 and that's gotten me reading about Chinese mythology and one clear thing is a lot of Japanese mythology comes from China, for instance it seems there is a "world tree" in almost every JRPG that they climb to get to heaven and even if they call it Yggdrasil it is not from Norse mythology it is from Chinese mythology. (Turns out also a lot of Chinese mythology as well as neopagan ideas comes from India as well.)

Re: How many medical studies are faked or flawed?

#237

Post by fer »

mharig wrote: post #232
fer wrote: post #219

Earlier quoted context omitted.

Yeah, because hundreds of health authorities simultaneously collecting their own data (often with larger groups of each cohort) wouldn't be able to replicate or disprove the outcome of the original research.

You trust health authorities? Come on.

You on the other hand believe that, somehow, absolutely independent (and often confronted) authorities, both governmental and private, would magically agree on something false instead of calling each other out.

That's the-moon-landing-was-fake tier denial-ism.

Re: How many medical studies are faked or flawed?

#238

Post by jjslocum3 »

queuebert wrote: post #181
jjslocum3 wrote: post #151

> ...it would help if journals routinely asked authors to share their IPD Why couldn't a bad actor just fake the raw data? Isn't that what Climategate was all about?

That's hard to do such that it is undetectable to competent biostatisticians. Edit: See the Dan Ariely drama for an example.

So perhaps a competent biostatistician could get away with it?

Re: How many medical studies are faked or flawed?

#239

Post by mattkrause »

gmd63 wrote: post #74
gustavus wrote: post #50

Earlier quoted context omitted.

This seems overly simplistic, I mean part of the reason so many studies are so difficult is because they can happen over the period of months or years. This isn't like your High School chemistry experiment. These studies can be massive longitudinal studies that take years to conclude and work on. If the answers were easy I'm sure someone would've implemented it already, it turns out those kind of things are hard. I'd…

I'm interested more in why it specifically would be hard to just show people what you are doing (or did) instead of telling them a story at the end of months or years. Critical details and oversights can be lost in translation between reality and LaTeX that could be easily pointed out by third party observers.

Who is going to watch thousands[0] of hours of video, most of which will show a scientist fiddling around with setscrews or hunting down a loose ground wire?

And what scientist is going to be happy with constant surveillance that's intended to be shared (so no picking your nose, losing your temper, or making mistakes)?

[0] Literally, thousands. I just finished a paper about a project that we started three years ago. This was the main project for two people, plus a tech: 2000 hrs/yr * 3 * 80% effort = 4800 hours of footage.

Re: How many medical studies are faked or flawed?

#240

Post by the_af »

PaulHoule wrote: post #236
the_af wrote: post #102

Earlier quoted context omitted.

> I believe we've lost some to coyotes Am I out of place if I say I also like coyotes? (Remember: city boy, so nothing is at stake for me here. I also like wolves!) And yeah, Japanese folklore has taught me that it's best to avoid kitsune. Though they sometimes turn into magical women who help you?

Really I don't take predation personally. I don't tell the SPCA, however, that I am getting a cat there because it's a replacement for one lost to predators. They die of old age, get hyperthyroidism, high blood pressure, become blind, walk around mindlessly and crash into things, and one morning you find them dead at the bottom of the stairs. I think they would find dying at the claws of a predator an honorable death…

Indeed, a klingon-like death for a cat would be honorable.

The Japanese do derive a lot of their culture from the Chinese, don't they? I would like to one day read some of the stuff around the Three Kingdoms. Alas! So much to read and do, and so little time!

Anyway, thanks for this conversation, I enjoy it.

Post reply on HN