Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cracking down on research fraud

undark.org

Post reply on HN
Print view View original

131–140 of 227 posts

Re: Cracking down on research fraud

#131

Post by aaron695 »

Gatsky wrote: post #78
lettergram wrote: post #40

I think there is outright fraud and subtle fraud. For instance, most mouse studies specifically as it relates to aging, drug safety, and cancer research should be thrown out. This is well covered here: https://m.youtube.com/watch?v=pRCzZp1J0v0 That’s not the only issue, but it’s a known issue, everyone is ignoring as it relates to all studies with the most common mice

This Weinstein mouse model thing is blown completely out of proportion, and is only so prominent because his own brother Eric put it on The Portal. The point is that everyone knows mouse models are imperfect, but they are still useful. Pointing out one of the ways they are imperfect is fine. Pretending this is a nobel prize winning earth shattering discovery that is suppressed by the establishment is just annoying. R…

He specifically mentions comments like yours are not helpful. This is the researcher fraud that is the problem in science that we are talking about, issues at hand are ignored or twisted so the paper mill can continue.

The concept is all mice in the USA come from one laboratory and have a common attribute that's different from wild mice and European suppliers.

And it's spelt out how in theory this different attribute could change medical results.

To come back with we don't care, without proof it's not a issue, is a deeply broken system, but is exactly how current universities work.

Here's the actual segment (18 mins) - https://www.youtube.com/watch?v=ve4q-1D_Ajo

Related (12 mins) - https://www.youtube.com/watch?v=8ygLNOt43So

Re: Cracking down on research fraud

#132

Post by lettergram »

Gatsky wrote: post #78
lettergram wrote: post #40

I think there is outright fraud and subtle fraud. For instance, most mouse studies specifically as it relates to aging, drug safety, and cancer research should be thrown out. This is well covered here: https://m.youtube.com/watch?v=pRCzZp1J0v0 That’s not the only issue, but it’s a known issue, everyone is ignoring as it relates to all studies with the most common mice

This Weinstein mouse model thing is blown completely out of proportion, and is only so prominent because his own brother Eric put it on The Portal. The point is that everyone knows mouse models are imperfect, but they are still useful. Pointing out one of the ways they are imperfect is fine. Pretending this is a nobel prize winning earth shattering discovery that is suppressed by the establishment is just annoying. R…

I generally agree people understand mouse models are imperfect.

The issue here, is that you have to understand how they are imperfect if they have any value at all. In this case, doing drug research on a mouse model is fine. Unless the goal of that study is to see damage to organs or dna. Unfortunately, that’s what was not being controlled for or understood. Effectively, the DNA has more protection from damage in the lab micr used for models than normal mice or humans.

This implies most of our drug evaluations look better on paper than reality. Huge deal if true And there’s evidence (long term) to support this.

The reason it’s on the portal and Joe Rogan is no one in the scientific community is evaluating it (on the surface). What’s more baffling is working in and around this space, I can see the effects first hand (which I won’t dive into, for the same reason no one is saying anything about this in the scientific community).

Re: Cracking down on research fraud

#133

Post by dodobirdlord »

wjn0 wrote: post #12
woofie11 wrote: post #4

Basic set of checks-and-balances: * Preregistration and adoption of open science practices * Public access to research results, methods, and data, with some exceptions (such as PII) * Federally-funded universities can't use NDAs or non-disparage agreements * Federally-funded universities must respond to records requests under terms similar to FOIA (note that FOIA has requestor pay costs) * Federally-funded universiti…

> * Public access to research results, methods, and data, with some exceptions (such as PII) You mentioned PII, so I'm assuming some familiarity with the health field. I'm curious about your thoughts on the position that one should not be required to immediately publicize their data, because there needs to be an expectation that a researcher can translate the capital (both time and money) they expend to acquire quali…

At the very least it seems like it would be fraud not to include the original collector of the data as a co-author on the analysis paper.

Re: Cracking down on research fraud

#134

Post by tridentlead »

vkou wrote: post #119
dasudasu wrote: post #114

Earlier quoted context omitted.

There are no ways around that. You cannot feasibly make funding infinite nor do anything about the lack of political will for increasing research funding. Even if funding was to increase dramatically, you'd get people coming up with even more expensive research projects (new particle accelerator anyone?) and there would still be winners and losers of that funding game. There's also the fact that a PI minting dozens o…

You don't need to make funding infinite, in order to meet the current demand for it. And increasing funding will not increase demand in lockstep. There are indeed more people who want to do science then there is money allocated to pay them, but the number of those people is not going to vastly increase if 'researcher' continues to be a job that requires years of post-secondary education, has long, shitty hours, and p…

Well if the goal is to fund everyone, the pressures that ensure it “ requires years of post-secondary education, has long, shitty hours, and pays peanuts” will likely reduce.

Re: Cracking down on research fraud

#135

Post by wisty »

dasudasu wrote: post #97
gwerbret wrote: post #77

These issues of research fraud come up often, but the root of the problem is a bit more subtle. In North America at least, biomedical research labs operate largely as fiefdoms of the individual principal investigators (PIs). The actual research work falls almost entirely on the backs of grad students and postdoctoral fellows. The grad students need to generate "good" data in order to graduate, the postdocs need the s…

The topic of funding is a recurrent one in these types of discussions. There are always more people wanting to do science than what is available in funding. There needs to be a process to allocate resources that takes into account competency for the job. Just granting it at random or equally is probably very very far from optimal if it means that an exceptional PI cannot even do the things to push his/her research to…

> Just granting it at random or equally is probably very very far from optimal if it means that an exceptional PI cannot even do the things to push his/her research to the next level.

What does "far from optimal" mean? Let's pretend the allocators are not god-like decision makers (if they are, let's fire all the scientists because our god-like decision makers can do the research for them).

Let's just accept that if we have a large bunch of diverse potential researchers, and limited funding, the best way to get a breakthrough is by funding a random sample of these researchers.

Maybe this sample should be stratified, but it should not be stratified in such a way that it incentives chasing grants (effectively making the grant-allocators the real PIs - with all their flaws and biases systematically driving the process, but without any accountability or management effort on their behalf).

We can't get optimal without unlimited wisdom, and if we had unlimited wisdom we wouldn't need scientists.

Re: Cracking down on research fraud

#136

Post by Gatsky »

aaron695 wrote: post #131
Gatsky wrote: post #78

Earlier quoted context omitted.

This Weinstein mouse model thing is blown completely out of proportion, and is only so prominent because his own brother Eric put it on The Portal. The point is that everyone knows mouse models are imperfect, but they are still useful. Pointing out one of the ways they are imperfect is fine. Pretending this is a nobel prize winning earth shattering discovery that is suppressed by the establishment is just annoying. R…

He specifically mentions comments like yours are not helpful. This is the researcher fraud that is the problem in science that we are talking about, issues at hand are ignored or twisted so the paper mill can continue. The concept is all mice in the USA come from one laboratory and have a common attribute that's different from wild mice and European suppliers. And it's spelt out how in theory this different attribute…

I've heard everything he said. I'm well aware of the concept. Again, everyone knows that all mice come from the same place (because it's written in the papers), and that they are very different to wild mice or mice from elsewhere.

I also didn't say we don't have proof that it is a problem. Again, all models are imperfect, but some are useful.

I am telling you what I see as the significance of Weinstein's findings as someone that works in the field.

It is nonsesnical to me that this particular concept is suddenly so prominent because of Joe Rogan, whereas a multitude of other issues with research and human translation are not talked about at all. To me what this is really about is Weinstein's perception of the significance of his findings. He, like a lot of scientists, thinks his work is hugely important and he deserves more credit. Others do not. This happens all the time. The only difference is that he has found a way to disseminate his findings to a non-specialist audience.

Re: Cracking down on research fraud

#137

Post by galimaufry »

ketzo wrote: post #66
tlb wrote: post #35

Estimating from my own reading, 1% of research is fraud and 80% is worthless for other reasons. Numbers vary between fields. If that's the case, what's the argument for why we should spend time doing something about the 1%? Solving 100% of the 1% wouldn't change the overall situation much. Possible arguments include: - Fixing the 80% is hard, but fixing the 1% is satisfying (to the aggressively conventional-minded, a…

I think the disproportionate harm argument is a big deal. A single high-profile case of fraud undermines trust in the entire field, and that’s critical to maintain.

I don't think that's true. There have been frauds in materials science, but people still trust it. The fields that are untrusted are the ones that are so hard that it's plausible that the conventional wisdom might be wrong even if everyone is honest.

Re: Cracking down on research fraud

#138

Post by voxl »

dasudasu wrote: post #97
gwerbret wrote: post #77

These issues of research fraud come up often, but the root of the problem is a bit more subtle. In North America at least, biomedical research labs operate largely as fiefdoms of the individual principal investigators (PIs). The actual research work falls almost entirely on the backs of grad students and postdoctoral fellows. The grad students need to generate "good" data in order to graduate, the postdocs need the s…

The topic of funding is a recurrent one in these types of discussions. There are always more people wanting to do science than what is available in funding. There needs to be a process to allocate resources that takes into account competency for the job. Just granting it at random or equally is probably very very far from optimal if it means that an exceptional PI cannot even do the things to push his/her research to…

I think there should be two pools of money, one that is granted at random to anyone who has obtained a professorship at a university, let that be the first filter for competency. The second pool of money can be given to projects that have immediate social interest and impact, and the government can, in this case, be very selective as there should be some social weight to succeeding here.

The first pool is for idea generation, failure is an option. The second pool is to direct invention for social benefit.

Re: Cracking down on research fraud

#139

Post by prionassembly »

canjobear wrote: post #115
gwerbret wrote: post #77

These issues of research fraud come up often, but the root of the problem is a bit more subtle. In North America at least, biomedical research labs operate largely as fiefdoms of the individual principal investigators (PIs). The actual research work falls almost entirely on the backs of grad students and postdoctoral fellows. The grad students need to generate "good" data in order to graduate, the postdocs need the s…

Another variant of the incentive problem is more insidious. Many academics are driven by ego, and specifically a desire to be influential. One way to be influential is to have great, original, and correct ideas. This can be a good incentive, but coupled with our human ability to deceive ourselves, it can become pathological. I’ve seen researchers get convinced that their ideas are so beautiful and right-feeling that…

> I’ve seen researchers get convinced

Where have you seen them?

Re: Cracking down on research fraud

#140

Post by voxl »

gwerbret wrote: post #107
martincmartin wrote: post #91

Earlier quoted context omitted.

What do you suggest as an alternative?

1) I think the idea of funding "people, not projects" (the Howard Hughes Research Institute's byline; I'm not affiliated with them in any way) is a great one. It would probably involve changing the structure of research programs in other fundamental ways, such as by 2) Limiting the number of graduate students. The current relatively-low barrier of entry into grad school provides cheap, motivated labor for PIs who are…

Your second and third suggestions will literally kill all research at anything but the most prestigious universities. It is elitism. Graduate students are not always RAs, they're usually not. They often teach, so you're not giving an accurate depiction of the incentives placed on them.

Becoming a graduate student is not easy, the bar is not relatively low, and making it through the program is even harder. It's true that academic positions are not large enough but you're ignoring the private sector.

Research scientists could not possibly be paid enough by universities. These would be graduate students minus the mentorship. A worse of all worlds.

Post reply on HN