Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cracking down on research fraud

undark.org

Post reply on HN
Print view View original

201–210 of 227 posts

Re: Cracking down on research fraud

#201

Post by aaron695 »

Gatsky wrote: post #187
aaron695 wrote: post #169

Earlier quoted context omitted.

You are still saying what Weinstein said you would say. Weinstein has laid out a case. It theoretically makes sense. It is quantifiable. He makes a good argument it is massively significant. If confirmed it is fixable. If confirmed it means things in other fields. This is in pop culture, anyone researching with mice should have heard of it, so the fact no one can whip out an article disputing it says a lot. Science i…

I don’t think you are listening to me. I am not disputing his point. I think it could well be valid and is certainly plausible. The idea is pretty clever. Attempts should be made to understand it more and minimise the effect. Great, add it to the list of problems with mouse models. It certainly isn’t at the top.

Honestly, the video only 18 mins (9 mins at 2X) long. Just give it a watch.

He says it's the top. He explains why it is the top. It is logical and well thought out with theoretical and practical evidence it is the top.

I'm happy to consider it's not the top.

But you have to actually say what is at the top.

Re: Cracking down on research fraud

#202

Post by roel_v »

webmaven wrote: post #200
roel_v wrote: post #183

Earlier quoted context omitted.

Thank you for writing this. Your examples were lethargic. I know I should stay away from the 'academia is broken duh duh' threads here but I get sucked into them anyway. They're always full of armchair experts who don't know anything about universities or research except from being students or at best junior researchers and then think this makes them experts on the matter. But the article itself had that same vibe fo…

> But not being included as an author on a paper, or being included when you didn't contribute that much? I've been in both situations and in some cases, I was completely fine with it; in others I was a bit miffed (mostly because of the same interpersonal frictions that happen everywhere where people work together) - but in none of those I would call it anywhere near 'fraud' or even 'dishonest'. As I understand it, g…

shrug Sure, theoretically, but someone who wants to ensure a positive review can just as easily collude with someone who didn't do any of the work. And then still - yes it's possible that a completely bogus paper gets signed off on by a dishonest conspirator, but the editor should see the discrepancy between that and the reviews of the other authors, and dig deeper. But the real problems start in the grey zone; like when you're on the fence between 'reject' and 'major revisions'. There is no 'objective' truth there and quite honestly as an author it's a crap shoot and always has been. It sucks the first few times it happens but none of leads to 'peer review being broken'.

Re: Cracking down on research fraud

#204

Post by secretasiandan »

shuckles wrote: post #161
logicslave wrote: post #96

Earlier quoted context omitted.

You got me, poor wording. My father worked at a very high level in influential investment funds, I grew up around the industry and understand how it works. I will delete this later because its not something I want to advertise

It sounds like you have a background and exposure to people who might have many enabling success factors. Shiny credentials might be one but social networks are another. Collectively, they might be contributing to the success you suggest is not deserved, but I’m not sure that’s an obvious argument. People of these backgrounds could probably game any gating ritual. Perhaps we should focus on drawing more actual talent…

I think it is correct that on the surface it is credentials. I also think it is correct that the underlying property is social networks and a particular culture / mindset.

Ultimately, I think the "phonies" need to be addressed directly because I believe they are effectively a cabal.

I also think it requires the broader culture to take active steps to help make this happen: work harder to think about what bad behavior is and take action to avoid / penalize it.

FWIW, I have traveled in the finance / startup circles and kept looking for "better places" but have come to the conclusion that they are few and far between and the issue is the business culture and the broader culture that celebrates it.

Re: Cracking down on research fraud

#205

Post by brownbat »

You can't really improve this until the lead author doesn't have a final say on how to treat outliers.

In any given study, there are going to be hundreds of special cases in the data that you didn't anticipate, and you have to decide whether to include or exclude them.

Any researcher will subconsciously be more sympathetic to arguments to exclude subjects that go against the principal theory, and less so to subjects that confirm it.

And it's a battle of reasonable arguments, most of the cases aren't bright line fraud or misconduct, they're just humans finding some arguments more compelling and the impossibility of escaping our own biases. (And yes, sometimes it's fraud, but fraud is just the tip of the iceberg if we're talking about genuinely improving the reliability of scientific findings.)

Prepublication is helpful, but more and more I'm convinced that the only way to do proper science would be to completely disaggregate study design from study execution.

Re: Cracking down on research fraud

#206

Post by obmelvin »

jrochkind1 wrote: post #198
obmelvin wrote: post #160

Earlier quoted context omitted.

HHMI is crazy, it has definitely had a positive impact on my girlfriends PhD experience and allowed her to take on a 'post doc killer' project. It's taken her almost 3 years, but she's gotten great quality data, and just got a great review back from Nature Chem with proposed experiments as expected. Also, the period can increase as time goes on. Her lab doesn't have to renew for 9 years I think (they renewed last sum…

> That said, it's obviously a model that really requires them to select the best of the best labs as you mention... It seems like it's also a model that makes labs better . What labs will be the "best of the best labs" is not independent from the way their funding sources provide pressure/rewards/punishments to do work in certain ways or do certain kinds of work.

Yes, you are very much correct. Unfortunately, if HHMI and the like are unwilling* to do these sorts of funding experiments, I'm not really sure who will.

* I can't claim to be up-to-date on the distribution of labs funded so for all I know someone is already doing this.

Re: Cracking down on research fraud

#207

Post by weehoo »

alexashka wrote: post #47
logicslave wrote: post #6

The root of the problem is that we allow people with high esteemed credentials to run society. They jumped through a hoop. "You got straight As at 14 - 18 years old and got into an ivy league school as a result? Here run this venture fund." "You got a PhD in Economics with a good publication by P-hacking your secret data? Here take a run at the FED with power over the US Economy. Your Phd shows that you are the man f…

I'll trade my credential for a few million any day of the week. Never have any takers, I wonder why.

Credentials are non transferable and non-voidable unless you are later discovered having cheated. I don’t get what point you’re trying to make here.

Re: Cracking down on research fraud

#209

Post by mturmon »

adminprof wrote: post #143
martincmartin wrote: post #91

Earlier quoted context omitted.

What do you suggest as an alternative?

This entire thread is really frustrating for me to read. A lot of these ideas are one-liners from people who have had very little experience with different parts of academia, except maybe being a student. Every single idea here are debated to death by academics already (not just university academics, but funding agencies, academic societies, award panels, journal editors and conference organizers), with changes happe…

You're right. Love your example.

HN does not have much actual collective expertise in this area (say: governance structures for technical research), but HN is solution-oriented, so ideas will be proposed. They just tend to be not very good. (See also: HN climate-studies threads.)

I've spent 10 or 15 minutes on this thread, and didn't read any comments actually building on what the OP said. OP's author does have expertise in this - he has been writing a series of good stories for Science magazine in this general area. Sigh.

Re: Cracking down on research fraud

#210

Post by etrautmann »

obmelvin wrote: post #206
jrochkind1 wrote: post #198

Earlier quoted context omitted.

> That said, it's obviously a model that really requires them to select the best of the best labs as you mention... It seems like it's also a model that makes labs better . What labs will be the "best of the best labs" is not independent from the way their funding sources provide pressure/rewards/punishments to do work in certain ways or do certain kinds of work.

Yes, you are very much correct. Unfortunately, if HHMI and the like are unwilling* to do these sorts of funding experiments, I'm not really sure who will. * I can't claim to be up-to-date on the distribution of labs funded so for all I know someone is already doing this.

HHMI is not the only one - certainly DARPA is willing to provide large funds for specific areas of interest (though a VERY different funding model). Wellcome trust, the Allen Institute, Broad institute, and others also work with slightly different models that can complement the NIH funding lines.
Post reply on HN