Time to assume that health research is fraudulent until proven otherwise? • Hacker News Board

Re: Time to assume that health research is fraudulent until proven otherwise?

#61

Post by topspin » Mon, Jul 19, 2021, 6:53 PM UTC

caseysoftware wrote: post #24

peytn wrote: post #2

No, but you need to be able to ask people who’re “in the know,” e.g. > When Roberts contacted one of the journals the editor responded that “I wouldn’t trust the data.” People already know what the bullshit results are.

If the editor didn't trust the data, why did they publish? The people who keep informed on their field (aka "in the know") would then be tainted because they would likely believe a journal would vet the data, process, and researchers before publication. Unless you mean "in the know" in that they know the entire publishing system is a scam... and the ripples from that are huge.

"If the editor didn't trust the data, why did they publish?"

They're in on it. Without material the journals have nothing to publish. So their inclination is to accept and publish with as little friction as possible.

Publish or Die. Remember? That applies to the whole supply chain, not just poor put upon individual researchers.

Re: Time to assume that health research is fraudulent until proven otherwise?

#62

Post by ajsnigrutin » Mon, Jul 19, 2021, 6:55 PM UTC

Another problem with studies is, that negative results are rarely published unless it's something really really "interesting".

"we tried treating X with Y, and it didn't help (even though in theory it should have some effect)" is harder to get published than "we treated X with Z in vitro and it killed all the cancer cells (and noncancer ones too, whoops)".

Re: Time to assume that health research is fraudulent until proven otherwise?

#63

Post by timy2shoes » Mon, Jul 19, 2021, 6:57 PM UTC

function_seven wrote: post #21

east2west wrote: post #14

Already reached the conclusion when I saw a genetic researcher presenting his p-value < 10^-40 as better than < 10^-10. I kept my mouth shut because I didn't want to ruin the poor guy's moment in the sun, but I knew it was time to get out.

My naive understanding is that "smaller p-value" == "more likely result is true". I know there's always more nuance in statistical reasoning, but the first number is vastly smaller than the second one, right? Is it just that both are hilariously tiny and not credible? Or is there no additional value after you get into the one-in-billions territory?

> My naive understanding is that "smaller p-value" == "more likely result is true".

I think you're making the classic Prosecutor's fallacy: https://en.wikipedia.org/wiki/Prosecutor%27s_fallacy. In my experience, smaller p-value tends to be more of a measure of sample size than anything else, or an overly restrictive null distribution that is almost certain to be rejected.

Re: Time to assume that health research is fraudulent until proven otherwise?

#64

Post by sharikone » Mon, Jul 19, 2021, 6:57 PM UTC

I was terrified approximately 14 months ago when the big pharma companies started to get billions to develop the vaccine. My fear was that their research was fraudulent and they would be exposed and discredit science as a whole. I am very happy that this did not happen.

I guess we are not in a total catastrophe situation but there is significant rot.

Re: Time to assume that health research is fraudulent until proven otherwise?

#65

Post by dleslie » Mon, Jul 19, 2021, 6:58 PM UTC

Shouldn't all research be considered _tenuous_ until it is corroborated by a third-party without financial or social connections to the original researchers?

Re: Time to assume that health research is fraudulent until proven otherwise?

#66

Post by heisenbit » Mon, Jul 19, 2021, 7:02 PM UTC

ta988 wrote: post #22

When you want to do a proper work, your grants and papers get rejected because they are not innovatove enough or don't go far enough. So it is not a surprise that people that lied in their applications about what they can realistically do also lie when it comes to reporting results. Unfortunately there is no way out. I stopped counting how many reviewers of my grants disagreed on what was proposed, one saying that it…

The problem is not that people lie on their application but that these people are now being judged by people who lied on their applications some time back. The lying has been institutionalized and leaves little resources for small but meaningful progress.

Re: Time to assume that health research is fraudulent until proven otherwise?

#67

Post by WalterBright » Mon, Jul 19, 2021, 7:02 PM UTC

ta988 wrote: post #22

When you want to do a proper work, your grants and papers get rejected because they are not innovatove enough or don't go far enough. So it is not a surprise that people that lied in their applications about what they can realistically do also lie when it comes to reporting results. Unfortunately there is no way out. I stopped counting how many reviewers of my grants disagreed on what was proposed, one saying that it…

I'm sadly amused by all this. The complaint I hear about privately done research is it's all tainted by the profit motive, and so research should be funded by the government, as then it'll be pure and untainted by selfish motives.

Of course, government funded motives are just as tainted by selfish motives, if not more. Even worse, the people who make the funding decisions aren't spending their own money, so they have little reason to care.

At least with privately funded research, the people providing the money aren't going to fund bullshit fake research. This is why market systems work better than government systems.

Re: Time to assume that health research is fraudulent until proven otherwise?

#68

Post by vadansky » Mon, Jul 19, 2021, 7:03 PM UTC

dleslie wrote: post #65

Shouldn't all research be considered _tenuous_ until it is corroborated by a third-party without financial or social connections to the original researchers?

Not even that is enough because you might end up with the filling drawer problem. For example you might end up with 50 people trying to replicate it, 49 failing and 1 "succeeding" by chance. The 49 won't publish because they probably got the "wrong" answer by chance, and they don't want to publish negative results, so they will leave it in the filing drawers.

The one that did "replicate" it will publish and then in round 2 there is even more pressure not to publish negative results because "hey, this was replicated before already"

Re: Time to assume that health research is fraudulent until proven otherwise?

#69

Post by tasogare » Mon, Jul 19, 2021, 7:04 PM UTC

nextos wrote: post #41

ta988 wrote: post #22

When you want to do a proper work, your grants and papers get rejected because they are not innovatove enough or don't go far enough. So it is not a surprise that people that lied in their applications about what they can realistically do also lie when it comes to reporting results. Unfortunately there is no way out. I stopped counting how many reviewers of my grants disagreed on what was proposed, one saying that it…

You are absolutely right. Furthermore, universities tend to require tons of publications to promote you. Things are spinning out of control. I know a few EU countries where the written norm is to need > 100 publications to qualify for a full professorship, with equally ridiculous requirements for associate and assistant positions. Obviously, this encourages and rewards completely broken practices. Many associate and…

> Many associate and full professors in my area only care about stamping their names into as many journal articles as humanly possible. [...] Obviously, they hardly ever contribute anything. Their serfs do all the work.

This describe my lab's head perfectly. At first I found strange he was so angry about a side project paper I wrote alone quickly on my free time and asked to publish at a conference. Then I understood why: in his view, every minute I spend on my projects is one I don't spend on his projects. The guy approved my first journal paper submission, which had his name on it without even reading it. It was obvious by the lack of comments and when he asked a few days later during a lab meeting to change half the content of the paper...

I'm not against putting name of people contributing to the research, even slightly and informally, but at this point this is pure leeching and exploitation. Then he wonders why my thesis isn't progressing (hint: because when I chose nothing about the topic, method and experiment setting I'm not really motivated to work on that).

Re: Time to assume that health research is fraudulent until proven otherwise?

#70

Post by wizzwizz4 » Mon, Jul 19, 2021, 7:04 PM UTC

adolph wrote: post #52

zucker42 wrote: post #26

Earlier quoted context omitted.

What? How do you figure that? Some person points out actual instances of poor quality research and you imply he is anti-vaccine? I can find no record of this person discouraging vaccination.

Ok, maybe its just a coincidence that it was published right when the current dangers of misinformation are so high and so many are failing to believe science.

The solution to “people aren't believing academia” is to make academia more trustworthy (probably by filtering for trustworthiness rather than making individuals more trustworthy), not to encourage scientism. Seeking the truth is still important, after all.