Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Time to assume that health research is fraudulent until proven otherwise?

blogs.bmj.com

Post reply on HN
Print view View original

271–280 of 491 posts

Re: Time to assume that health research is fraudulent until proven otherwise?

#271

Post by caddemon »

aabaker99 wrote: post #209
sjwalter wrote: post #203

What I want to know is how does this issue impact the notion that we all seem to buy into that we should "follow the science". Scientists themselves have a hard time "following the science". Add to it the observation that when an issue is getting lots of attention outside of academia, then there are usually some really strong incentives (profit, prestige) associated with doing the science and applying it (e.g., epide…

I think this is a very important question. This is something that I struggle with. I have read a lot of papers. I generally think science can be a force for good. I understand analytic methods developed by or used in papers from my field of interest. I generally believe that those methods are capable of answering important and interesting questions. In my view, the problem is that you can't know if an article is good…

It's also the norm in some fields to provide only high level info in the methods section, often without supplemental method details accompanying. This makes it even harder to tell if they did the work correctly, because usually half the methods are intermediate steps which don't have any results directly reported in the paper. In a perfect world those methods would be uniform across labs, but in practice they definitely are not, and it makes tracing down the source of honest replication differences a nightmare.

There's also no way to really know if the researcher entirely left out 10 other tests they tried that failed. Sometimes you can guess that it's a stretch because of the stupid categories they use (I'm reminded of those ESPN graphics that say things like "most home runs on a rainy Tuesday in June"). But it's harder to detect if someone straight up removes data points, repeats tests and reports only the nicest, etc.

So at some point you basically need to be an insider in the field so you hear the gossip about what doesn't replicate. Or if you have access to thousands of dollars to blow you could try a dozen different variations to try getting it to replicate yourself.

I think for something like COVID that is actively affecting many people, there should be funding explicitly for replicating studies, and some slots reserved in a prestigious journal for the findings of the replications. I get that it is not feasible to be replicating everything in science, but I don't see why we can't have ~one lab per relevant university department that specializes in replicating important studies. If you make that a path towards becoming a tenured prof I think that could change the culture surrounding replication studies in general.

Re: Time to assume that health research is fraudulent until proven otherwise?

#272

Post by tgsovlerkhgsel »

dqpb wrote: post #84
injidup wrote: post #20

I recently had the privilege of trying to do the right thing when I identified fraudulent research carried out by an institution in Austria. The initial response of the institution was positive but when I pressed for further details on how such things could happen suddenly nobody anymore would take my call. The research was paid for by a private company to pimp a nonsense product. The research was never published in…

Can you be sued for asking questions?

You can be sued for anything, or nothing. You might be able to "easily" win the case, which may still cost you resources that you may not want to expend, and you won't be reimbursed for the stress and time used, only the financial expenses (and even that isn't true in all jurisdictions).

That said, if you know the legal situation, demonstrating that you're not afraid of the legal threats is often the way to make them go away.

For security research, if the goal of the legal threat is to prevent you from publishing your results for the first time... publishing the results can significantly reduce the incentive to cause you further trouble because the goal is no longer achievable.

Re: Time to assume that health research is fraudulent until proven otherwise?

#273

Post by michaelmrose »

nixpulvis wrote: post #248
menzoic wrote: post #174

Earlier quoted context omitted.

It seems well intentioned, but I'm not surprised at all about the outcome. I don't think you can expect the institution to implicate themselves like that. You have to realize that preventing legal and financial liabilities is the #1 priority for institutions. Unfortunately this includes when they are wrong. Even if the employees are "good" their lawyers will advise them against providing any details like that and mos…

So what should be done? Hire your own lawyers to go after them pro bono? Write an article and send it to the papers? This kind of thing needs to be shamed and punished. While I agree that the institutions can be expected to cover there ass, there must be a course of action.

Monies paid for students education should be conditioned on not being skeevy and unethical. Basically just arrange the penalties such that outright fraud is effectively the death penalty for the institution and lesser malfeasance is still financially disastrous.

Wait for one really good case and let the papers instead discuss how a century old institution collapsed.

Re: Time to assume that health research is fraudulent until proven otherwise?

#274

Post by caddemon »

tshaddox wrote: post #253
sjwalter wrote: post #203

What I want to know is how does this issue impact the notion that we all seem to buy into that we should "follow the science". Scientists themselves have a hard time "following the science". Add to it the observation that when an issue is getting lots of attention outside of academia, then there are usually some really strong incentives (profit, prestige) associated with doing the science and applying it (e.g., epide…

What is the alternative to following the science? Following people who are not scientists and who are explicitly making things up? This sounds a lot like "most plane crashes are due to pilot error, so maybe we should give non-pilots control of the planes."

I think it depends on the risk/reward of not following the science - as an example, there was debate about whether ibuprofen worsened the effects of COVID early in the pandemic. MGH basically came out and said that while the research was leaning towards ibuprofen being okay (and the WHO at that point had announced it was), they had plenty of acetaminophen available and no reason to doubt its safety for COVID patients. So they were going to stick with acetaminophen only for most patients.

Another COVID-related example is masks. At one point early in the pandemic there was messaging that masks were not useful in preventing the spread of COVID in the general public. There were supply chain reasons that probably motivated that message, but I also know virology PhD students that were insistent it was the truth, with papers in hand. At the time there were also papers to suggest the opposite, so one could have a healthy debate on the subject - but at the end of the day I gave my grandparents a small box of masks, because wearing a mask is so low effort I don't see why you wouldn't unless you are truly extremely certain that it's useless.

Re: Time to assume that health research is fraudulent until proven otherwise?

#275

Post by logicalmonster »

nabla9 wrote: post #195
tarere wrote: post #191

I will address the elephant in the room on this one. 168 comments so far, I did a research for word "vaccine" in the page, not a single result. So I assume this has not been discussed despite the current situation we are living right now. I've just read 2 articles about Moderna in general and Bancel in particular : https://www.bloomberg.com/news/features/2021-07-14/moderna-m... And Vanity Fair, unfortunatly in french…

No results because vaccines are not accepted based on publications and peer review on scientific papers. They go trough completely different and extremely rigorous testing process where everything is documented carefully, documents are examined and double checked. It's great to see 168 comments before first anti-vaxxer comment.

> They go trough completely different and extremely rigorous testing process where everything is documented carefully, documents are examined and double checked.

Rigorous testing is all well and good, but we do know how many errors crop up in scientific papers, right? A lot.

The gold standard of testing is double-blind, randomized, placebo controlled studies and this is only done sometimes for vaccines, as far as I aware.

I believe there are certain circumstances such as no safe and effective already existing vaccine, or where there's a certain kind of benefit to the injected population, where vaccines are generally allowed to go through this double-blind, randomized, placebo controlled studies (the gold standard of testing) and certain circumstances where they do not go through such rigorous testing.

I believe this ethic stems from Jonas Salk's decision during the development of the Polio vaccine where the ethical call was made to not do double-blind, placebo controlled testing due to the desire to prevent damage to human lives that could be prevented by not using a placebo.

I invite fact-checking on everything stated.

Re: Time to assume that health research is fraudulent until proven otherwise?

#276

Post by tarere »

adolph wrote: post #102
faeriechangling wrote: post #88

Earlier quoted context omitted.

>failing to believe science Believing in science means believing in falsification and sceptism. This was published during the ongoing replication crisis in medicine where we're finding that more than half of cancer studies don't replicate [1]. This replication crisis hasn't been put on hold and all science is now deemed irrefutable or you are VACCINE HESISTANT and DANGEROUS. I'm mostly confident in the COVID vaccines…

don't replicate != fraudulent I'm mostly confident in the COVID vaccines <- Is this "mostly" qualifier not the dangerous front end of misinformation and vaccine hesitancy?

>"Is this "mostly" qualifier not the dangerous front end of misinformation and vaccine hesitancy? "

I do not understand. HN forum are places where "question everything" should be the norm, especially about science, and more especially about health when so much money is at stake.

It seems we are in a religious quest, with virtue signaling signs everywhere, to laudate the vaccine COVID-19 miracle. Everything is "magic". Somehow one could make a parallel with the state of the economy and the stock market. We are printing our way to prosperity as we are saving the world with "methode Coué" and of course brutal coercition (mandatory vaccination at state level).

What part of "you can not buy time" don't you understand ?

You may throw billions, no, trillions, quadrillons, bazillions to all the Silicon Valleys startups and brains to developp a vaccine in a matter of months ; but you can not buy a 3, 5, 10 years study with this money. You can't even buy one month or one second, you just have to wait, this is life and bio-trans-human thinking still can't and never will overcome nature, space and time.

So you might be "mostly" confident about the vaccine, I would say that this is the absolute maximum optimism best case scenario adjective you can use. Moderna, Pfizer sure are, "risk big, win big".

But we'll have to wait a few years more to completely assert something like "safe and effective" for the COVID vaccines, that is not a fact anybody can deny.

Re: Time to assume that health research is fraudulent until proven otherwise?

#277

Post by h0l0cube »

nixpulvis wrote: post #248
menzoic wrote: post #174

Earlier quoted context omitted.

It seems well intentioned, but I'm not surprised at all about the outcome. I don't think you can expect the institution to implicate themselves like that. You have to realize that preventing legal and financial liabilities is the #1 priority for institutions. Unfortunately this includes when they are wrong. Even if the employees are "good" their lawyers will advise them against providing any details like that and mos…

So what should be done? Hire your own lawyers to go after them pro bono? Write an article and send it to the papers? This kind of thing needs to be shamed and punished. While I agree that the institutions can be expected to cover there ass, there must be a course of action.

We just need a 'Yelp for research institutions'. Perhaps there already exists a global register tracking reputation like this? If so it needs to be visible (particularly to the press) and be properly vetted to have real consequence

Re: Time to assume that health research is fraudulent until proven otherwise?

#278

Post by Tabular-Iceberg »

jahnu wrote: post #219
OriginalNebula wrote: post #134

Earlier quoted context omitted.

No, thank you! Austria has a weird thing with pseudoscience scams [0][1][2] and it needs to be dealt with. [0] https://de.wikipedia.org/wiki/Belebtes_Wasser [1] The current Minister of Economy was working as an "energizer" https://www.diepresse.com/5395317/wirtschaftsministerin-schr... [2] During the construction of a hospital in Vienna, an "energetic" ring was built around the construction site for 95000 Euro. https…

Not to mention almost all pharmacies selling homeopathic rubbish. I find this particularly irritating given the ridiculously regulations and control around things like buying paracetamol or ibuprofen.

Medical and ethical standards are in complete free fall in European pharmacies. I was told by my vet to rinse my dog’s eyes in sterile isotonic saline solution. When I went to the pharmacy to ask for it they didn’t know what it was. They suggested using chlorhexidine wound disinfectant instead.

My poor dog would have probably been blind or worse if I had taken the advice. But they don’t care, they just want to move products, whatever the cost.

Re: Time to assume that health research is fraudulent until proven otherwise?

#279

Post by caddemon »

version_five wrote: post #234
sjwalter wrote: post #203

What I want to know is how does this issue impact the notion that we all seem to buy into that we should "follow the science". Scientists themselves have a hard time "following the science". Add to it the observation that when an issue is getting lots of attention outside of academia, then there are usually some really strong incentives (profit, prestige) associated with doing the science and applying it (e.g., epide…

Follow the science is only used as a rhetorical device outside of science to try and convince people of something political. You would never hear an actual researcher say that. There is a realistic, weaker statement about the best available information we have, that a specialist could use to explain to a non specialist why they are making a recommendation about something emerging or theoretical. But what we are heari…

Totally agree. I do think most times I hear people saying "follow the science" they are saying it to someone that is being anti-intellectual. But at the same time the "follow the science" people usually give horrible counterarguments that contain straight up inaccuracies too. I don't know what annoys me more.

A fun one that circulated for awhile when the vaccines were first launching was "all vaccines are the same". They are decidedly not, and I don't think people are as stupid as that kind of messaging implies. It was weirdly taboo to say that you'd prefer a particular vaccine, even once we got to the point that there was enough stock to choose. It's true the clinical trial efficacy numbers shouldn't be literally compared to one another, but that doesn't mean we should pretend the vaccines are actually the same either. Somebody deciding to drive an extra hour so they can get an mRNA vaccine is not anti-science lmao.

Re: Time to assume that health research is fraudulent until proven otherwise?

#280

Post by bjornsing »

eloff wrote: post #111
ta988 wrote: post #22

When you want to do a proper work, your grants and papers get rejected because they are not innovatove enough or don't go far enough. So it is not a surprise that people that lied in their applications about what they can realistically do also lie when it comes to reporting results. Unfortunately there is no way out. I stopped counting how many reviewers of my grants disagreed on what was proposed, one saying that it…

Peer review is shit. It's elitist, it's actively anti innovation and enforces the status quo. Peer review should happen out in the open and not just be limited to academics.

Many see peer review as an integral part of the scientific method, but it’s actually a quite recent custom (ca 1970 IIRC). And I agree it’s not without problems. Whenever you give a collective power over the individual it creates room for politics.
Post reply on HN