Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cracking down on research fraud

undark.org

Post reply on HN
Print view View original

21–30 of 227 posts

Re: Cracking down on research fraud

#21

Post by throwaway2048 »

ktaylora wrote: post #10
Something1234 wrote: post #8

Earlier quoted context omitted.

Thanks for pointing out the problem with credentials. Name a better indicator then for being competent.

Industry kinda already has. Products launched. Teams managed. A candidate's publication record is a conversation starter during interviews, but I feel like years of experience at a company working on launching a successful project is a much more valuable currency.

These metrics can just as easily be gamed, see how all the glory at Google goes towards launching exciting projects, that then immediately get put into stasis, then canceled.

Re: Cracking down on research fraud

#22

Post by OminousWeapons »

RcouF1uZ4gsC wrote: post #3

I think one of the big things that you can do is split up the data gathering and data analysis. Kind of like we don’t trust the companies to audit themselves, instead we have an outside firm. In this model, a researcher would create a hypothesis and collect the data. Their team would write the background and methods sections of the paper. Then the entirety of raw data would be sent off to a third party for data analy…

You would just get people massaging or generating the data to fit a conclusion prior to sending it out for statistical analysis. If people want to cheat they can find a way. The only way to really do what you are getting at is to construct core labs or CROs to run all experiments on behalf of the investigators. This is not infeasible in many cases (and is already done in narrow ways) but it requires hiring staff scientists to run every experiment rather than grad students or post docs and costs / complexity will explode.

The real way to defend against a lot of fraud is to force people to submit actually detailed methods sections so experiments are legitimately reproducible (they largely aren't now). This would catch a lot more fraud quickly, although even this won't fully work as some experiments are simply too costly to reproduce for validation purposes (eg animal studies).

Re: Cracking down on research fraud

#23

Post by chrisseaton »

ktaylora wrote: post #10
Something1234 wrote: post #8

Earlier quoted context omitted.

Thanks for pointing out the problem with credentials. Name a better indicator then for being competent.

Industry kinda already has. Products launched. Teams managed. A candidate's publication record is a conversation starter during interviews, but I feel like years of experience at a company working on launching a successful project is a much more valuable currency.

A good PhD is launching a successful project.

Re: Cracking down on research fraud

#24

Post by cblconfederate »

Equalize the field, radically remove research hierarchies. Liberate research funding from pointless bureaucracies and 4-year grants and instead pay scientists to work on whatever they like most. Let science be fun again

I always return to this article from David Hubel (of Hubel&Wiesel): https://www.cell.com/neuron/fulltext/S0896-6273(09)00733-8

I arrived at his lab around noon and found him working alone, recording from an anesthetized macaque monkey. I asked him when he had started the experiment, and he answered “in the morning”, which I finally realized was the morning of the day before. So he had worked, by himself, all the day before, all that night, and that day until noon. What was typical, in that era, was not only the long hours but the fact that the project was done by one person, single handedly. The major papers were either by Mountcastle alone or in partnership with one other person. The leader of the physiology department was Philip Bard, but the idea that Bard should have asked to have his name on any of Vernon's papers surely never occurred to anyone.

Re: Cracking down on research fraud

#25

Post by ramraj07 »

kovac wrote: post #20
viburnum wrote: post #19

A friend of mine was doing a chemistry phd when he discovered his supervisor was falsifying data. If he had blown the whistle it would have ended his career, but if he had played along his dissertation would have been based on false data. Both options were bad so he just quit. https://www.statnews.com/2016/11/25/postdocs-grad-students-f...

That's some guts to do the right thing. Respect.

I was not faced with such a shitty situation, and I probably would have done the same as well, but the right thing would have been to report it.

Re: Cracking down on research fraud

#26

Post by Ericson2314 »

woofie11 wrote: post #4

Basic set of checks-and-balances: * Preregistration and adoption of open science practices * Public access to research results, methods, and data, with some exceptions (such as PII) * Federally-funded universities can't use NDAs or non-disparage agreements * Federally-funded universities must respond to records requests under terms similar to FOIA (note that FOIA has requestor pay costs) * Federally-funded universiti…

let's add

* Portion of faculty which must be tenured.

Re: Cracking down on research fraud

#27

Post by opportune »

logicslave wrote: post #6

The root of the problem is that we allow people with high esteemed credentials to run society. They jumped through a hoop. "You got straight As at 14 - 18 years old and got into an ivy league school as a result? Here run this venture fund." "You got a PhD in Economics with a good publication by P-hacking your secret data? Here take a run at the FED with power over the US Economy. Your Phd shows that you are the man f…

All three of those examples are missing a step of like 5-20 years in the middle. Nobody is running a venture fund as a 22 year old ivy grad (barring someone who's taking over for their family member, but in that case it has nothing to do with being an ivy grad), nobody is running the FED right out of a PhD, nobody is a director at a big corporation right out of a PhD.

Presumably once those people are in such high-power positions, they also have a track record of real accomplishments behind them; it's certainly possible they've lied and cheated their entire career but it's definitely less likely they'll make it that far that way.

Re: Cracking down on research fraud

#28

Post by nomel »

saagarjha wrote: post #14
ktaylora wrote: post #10

Earlier quoted context omitted.

Industry kinda already has. Products launched. Teams managed. A candidate's publication record is a conversation starter during interviews, but I feel like years of experience at a company working on launching a successful project is a much more valuable currency.

For getting a job, maybe…industry isn't everything.

I think his point is that practical experience and a track record of proven, successful, outcomes are more useful for predicting future outcome.

Re: Cracking down on research fraud

#29

Post by cosmodisk »

logicslave wrote: post #6

The root of the problem is that we allow people with high esteemed credentials to run society. They jumped through a hoop. "You got straight As at 14 - 18 years old and got into an ivy league school as a result? Here run this venture fund." "You got a PhD in Economics with a good publication by P-hacking your secret data? Here take a run at the FED with power over the US Economy. Your Phd shows that you are the man f…

Can't agree more. And it's getting worse. In certain fields people are almost untouchable. Couldn't count anymore the number of discussions I had about doctors and how absolutely incompetent some of them are: the usual reaction is like 'No, you don't say! But the selection process ensures only the smartest get selected,so what are you on about?'. The second one is PhDs, and in general all sorts of professors almost elevated to god like level. You almost get a free pass to spread whatever shit you want if the credentials are right.

The corporate world is exactly the same. Since my last promotion,I now have a nice title. People now listen to what I have to say. They even take me seriously. I can now go to some guy who's got 20 times more experience and start selling my consulting services(I'm not a consultant).

Re: Cracking down on research fraud

#30

Post by DanielleMolloy »

I've read that before 2003, the whole of humanity has published as many scientific papers as from 2003 to 2016.

So what happened around 2000? Who has turned the scientific mission into a blind competition for superficial metrics? So many people in science I meet (apart from the few who benefited from this system, and therefore were selected by it) are frustrated by publishing for the sake of publishing (not science) and the bad incentives this system creates.

Who has thought that these superficial metrics would improve anything about science and why?

Post reply on HN