Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Time to assume that health research is fraudulent until proven otherwise?

blogs.bmj.com

Post reply on HN
Print view View original

81–90 of 491 posts

Re: Time to assume that health research is fraudulent until proven otherwise?

#81

Post by east2west »

function_seven wrote: post #21
east2west wrote: post #14

Already reached the conclusion when I saw a genetic researcher presenting his p-value < 10^-40 as better than < 10^-10. I kept my mouth shut because I didn't want to ruin the poor guy's moment in the sun, but I knew it was time to get out.

My naive understanding is that "smaller p-value" == "more likely result is true". I know there's always more nuance in statistical reasoning, but the first number is vastly smaller than the second one, right? Is it just that both are hilariously tiny and not credible? Or is there no additional value after you get into the one-in-billions territory?

Exactly, numerical errors could easily have accounted for the difference between already tiny p-values. The point isn't that the smaller p-value isn't better than the bigger one, it is, but that small significance should have been attached to the difference.

This example is a gnome-wide genetic association study. Every genetic variations are tested, so at least 500K or more linear regressions were performed. This many statistical tests could lead to many false positives just by chance, so one must do multiple-testing corrections. The end result of multiple-testing correction is much bigger and therefore worse p-values. Hence the drive toward ridiculously tiny p-values.

Re: Time to assume that health research is fraudulent until proven otherwise?

#82

Post by derbOac »

ModernMech wrote: post #50
ta988 wrote: post #22

When you want to do a proper work, your grants and papers get rejected because they are not innovatove enough or don't go far enough. So it is not a surprise that people that lied in their applications about what they can realistically do also lie when it comes to reporting results. Unfortunately there is no way out. I stopped counting how many reviewers of my grants disagreed on what was proposed, one saying that it…

> When you want to do a proper work, your grants and papers get rejected because they are not innovatove enough or don't go far enough. Not being innovative enough isn't the root cause though. The real issue is there isn't enough funding to go around, and so the bar is higher than it needs to be. Available research funding in the US is a paltry sum considering the aggregate ROI of discoveries and technologies that or…

I get the sense from colleagues and visiting different universities that this varies across the US, Canada, the UK, and the EU, but grants are now the bread and butter at most US universities. It's not really enough to publish 100s of articles, or have a high h index, it's to bring in money even if it's not strictly necessary for your research.

Part of the reason we have the problem you're mentioning is not that there isn't enough money to go around, it's that universities (at least in the US) now depend on inflated costs to function. The costs of research are kicked down the road to the federal government, and the research itself is seen in terms of profits rather than discovery. So if you have all these universities essentially telling researchers their jobs are on the line if they don't bring in profits, you're going to have everyone scrambling to bring in as much money as they can. It's not just postdocs or untenured research professor lines, it's tenured professors as well, whose income can be brought down below some necessary standard of living, or who can have salaries frozen or resources cut.

I was thinking about this the other morning. I had a grant proposal that the program officer was really excited about. This program of research could probably be conducted for almost nothing because it involved archival data analysis. However if you put a dollar amount on the time, it might realistically actually cost around 250k USD, maybe 500k max, pretty generously in terms of staff effort. However, the university managed to inflate the budget ask to around 2 million for the sole purpose of indirect funds.

When you have that kind of monetary incentive (carrot or stick), of course you're going to have thousands of persons applying for each opportunity. It's what led to the graduate student ponzi scheme, inflated numbers of surplus graduates, etc and so forth.

It all trickles down too, in terms of research claims, p-hacking, etc and so forth.

There's a place for profit, but there's also some realms where it does nothing but corrupt.

Re: Time to assume that health research is fraudulent until proven otherwise?

#83

Post by gotoeleven »

onlyrealcuzzo wrote: post #7
gotoeleven wrote: post #3

The war on standards is creating a low trust society.

I don't know. The amount of people that take one poorly measured data point and use that as a north star seems more troubling to me - than a society with low trust. I'm not convinced a low-trust society is inherently bad. I am convinced that a society that has high trust in garbage data is bad.

I meant more that society's attitude toward standards of any kind--honesty, proper behavior, civility--has become increasingly suspicious. These standards are, some argue, simply power structures to promulgate the various -isms that plague society.

If you've got scientists that believe that the standard of honesty is an artificially constructed power structure then obviously you can't trust them.

Re: Time to assume that health research is fraudulent until proven otherwise?

#84

Post by dqpb »

injidup wrote: post #20

I recently had the privilege of trying to do the right thing when I identified fraudulent research carried out by an institution in Austria. The initial response of the institution was positive but when I pressed for further details on how such things could happen suddenly nobody anymore would take my call. The research was paid for by a private company to pimp a nonsense product. The research was never published in…

Can you be sued for asking questions?

Re: Time to assume that health research is fraudulent until proven otherwise?

#85

Post by adolph »

zucker42 wrote: post #77
adolph wrote: post #52

Earlier quoted context omitted.

Ok, maybe its just a coincidence that it was published right when the current dangers of misinformation are so high and so many are failing to believe science.

Well of course I'm not going to believe blatantly fraudulent "science". That has nothing to do whether or not I would take a COVID vaccine (I have). Trying to suppress the truth that there is shoddy or false research to keep up appearances is ridiculous. There's something between ignoring scientific evidence you dislike and credulously taking everything a scientist says as fact. And the "dangers of misinformation" ha…

What is your plan for identifying the "blatantly fraudulent?"

How do you know your medical provider agrees with you about what is "blatantly fraudulent?"

Re: Time to assume that health research is fraudulent until proven otherwise?

#86

Post by derbOac »

nextos wrote: post #41
ta988 wrote: post #22

When you want to do a proper work, your grants and papers get rejected because they are not innovatove enough or don't go far enough. So it is not a surprise that people that lied in their applications about what they can realistically do also lie when it comes to reporting results. Unfortunately there is no way out. I stopped counting how many reviewers of my grants disagreed on what was proposed, one saying that it…

You are absolutely right. Furthermore, universities tend to require tons of publications to promote you. Things are spinning out of control. I know a few EU countries where the written norm is to need > 100 publications to qualify for a full professorship, with equally ridiculous requirements for associate and assistant positions. Obviously, this encourages and rewards completely broken practices. Many associate and…

One of the clearest examples of the publishing problem to me was the shift in meaning of last authors on papers over the course of my career. When I first started, last author meant the person who had contributed the least to the paper (in cases where ordering of effort can actually be determined -- often there's genuinely equal contributions). Often this was the senior faculty member, as they did little but sort of read over a paper or maybe supervise someone independently functioning.

Over time though the last author came to mean "the more senior person" and then "the person whose idea it really is". So being last went from this thing that no one wanted, to this thing that people would kind of argue over. In the process the more manipulative cases, people would kind of casually say "oh I can be last author" realizing the gains from that position.

It seems when a more junior person is doing all the work and is first author, an unscrupulous senior researcher will claim that "it's the idea that counts"; when that senior researcher is first author, it's "ideas are a dime a dozen, it's getting it done that matters."

Re: Time to assume that health research is fraudulent until proven otherwise?

#87

Post by AnimalMuppet »

adolph wrote: post #39
AnimalMuppet wrote: post #38

Earlier quoted context omitted.

No, this is about all of medecine , not about vaccines for a specific illness.

Are "vaccines for a specific illness" not in the set "all of medicine?"

Yes. But "all of medecine" is not in the set "vaccines for a specific illness".

Re: Time to assume that health research is fraudulent until proven otherwise?

#88

Post by faeriechangling »

adolph wrote: post #52
zucker42 wrote: post #26

Earlier quoted context omitted.

What? How do you figure that? Some person points out actual instances of poor quality research and you imply he is anti-vaccine? I can find no record of this person discouraging vaccination.

Ok, maybe its just a coincidence that it was published right when the current dangers of misinformation are so high and so many are failing to believe science.

>failing to believe science

Believing in science means believing in falsification and sceptism. This was published during the ongoing replication crisis in medicine where we're finding that more than half of cancer studies don't replicate [1]. This replication crisis hasn't been put on hold and all science is now deemed irrefutable or you are VACCINE HESISTANT and DANGEROUS.

I'm mostly confident in the COVID vaccines because we're at billions of doses and there is so much uncorrelated data about the vaccines, and they're so controversial and their safety/efficacy is being looked at by so many people that one can be relatively confident about their efficacy and safety (so long as you look at uncensored information to avoid systemic bias). We don't need to be science denialists and start denouncing scepticism on safety grounds, we can point out there are special reasons to be confident in the vaccines.

[1] https://www.npr.org/sections/health-shots/2017/01/18/5103048...

Re: Time to assume that health research is fraudulent until proven otherwise?

#89

Post by tasogare »

argvargc wrote: post #17

Yes, though it's not much of a problem for those unwilling to take the leap based on only one or two emergent studies, who might prefer relying on decades of safety data. Oh, but then that entirely sensible and understandable action might get one censored, deleted, fired, fined and/or discriminated against as a third class citizen. Cui bono?

Exactly. I've argued on social media with "trust the science" people: none of them were scientists while I am one. People in general have no idea how the sausage is made and how broken the system is. A year or more ago I was joking with a coworker "I hope medical studies are done more seriously than what we do". Today I wouldn't make the same joke given how the situation went out of control with states enforcing authoritarian policies based on poor understanding of how science work.

Re: Time to assume that health research is fraudulent until proven otherwise?

#90

Post by [deleted] »

WalterBright wrote: post #67
ta988 wrote: post #22

When you want to do a proper work, your grants and papers get rejected because they are not innovatove enough or don't go far enough. So it is not a surprise that people that lied in their applications about what they can realistically do also lie when it comes to reporting results. Unfortunately there is no way out. I stopped counting how many reviewers of my grants disagreed on what was proposed, one saying that it…

I'm sadly amused by all this. The complaint I hear about privately done research is it's all tainted by the profit motive, and so research should be funded by the government, as then it'll be pure and untainted by selfish motives. Of course, government funded motives are just as tainted by selfish motives, if not more. Even worse, the people who make the funding decisions aren't spending their own money, so they have…

[deleted]
Post reply on HN