Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Researchers taking on fraudulent science

analystnews.org

Post reply on HN
Print view View original

41–50 of 87 posts

Re: Researchers taking on fraudulent science

#41

Post by armchairhacker »

scherlock wrote: post #35
armchairhacker wrote: post #23

Earlier quoted context omitted.

But how do we “solve” those incentives? There will always be winners and losers, and since successful fraud is by definition indistinguishable from genuine success, there will always be people trying to game the system. The only effective solution I see is making fraud harder to hide. (There’s also making the penalty for fraud higher and trying to instill integrity, but the problem is some people will still take the…

The major issue with research is how only successful results are viewed as worth while, and study results without a "positive" out come is viewed as a failure. So this means researchers are incentized to p-hack or falsifying results since they need more funding. The reality is these negative results are just as useful and can either help other researchers avoid dead ends or they might have a not great result but with…

You are 100% correct. The problem is, even without the incentive of money and employment there's still the incentive of fame, and negative results simply won't get the same attention and impact as positive results. We can give people with negative results PhDs and publications, we can continue funding them. But at the end of the day, people remember and researchers cite the team who invented the LLM, not the many teams who tried and invented the less-successful models.

Re: Researchers taking on fraudulent science

#42

Post by bluenose69 »

freework wrote: post #34

In my opinion, the solution to all of this is to get rid of the whitepaper system all together. From now on, all scientists who want to publish science, they publish their work in the form of a video. They film themselves doing the science and then put that video on YouTube, or maybe some science specific version of youtube. Other people can then watch the video and determine for themselves if the science is valid or…

It takes of order 6 months to do the work described in a single paper. Even if we imagine someone watching the videos at double speed, it would take 3 months per paper. And that's for an expert who can understand the material.

Folks don't have the time to read the literature thoroughly as it is. That's one of the reasons the junk gets published. So this is not practical.

Plus, imagine reporting to your funding agency, Dean, or boss, that all you accomplished this year was to watch a few videos. No new work of your own. I guess you would make a video of you watching a video. And then someone would watch a video of you watching a video ...

Re: Researchers taking on fraudulent science

#43

Post by nwhitehead »

armchairhacker wrote: post #23
derbOac wrote: post #14

I love these people's work. At the same time I don't think any of this really addresses the root causes of the problems, which are career, university administrative, and field (i.e., recognition and attention) incentives. The whole paradigm we use, and the way we treat scientists and fund research, is broken. It feels a little bit like bailing water out of your boat with teacup on one side while ignoring the gaping h…

But how do we “solve” those incentives? There will always be winners and losers, and since successful fraud is by definition indistinguishable from genuine success, there will always be people trying to game the system. The only effective solution I see is making fraud harder to hide. (There’s also making the penalty for fraud higher and trying to instill integrity, but the problem is some people will still take the…

One idea is to put more pressure on writers of letters of recommendation. There is obviously already some rather weak "reputational incentive" to not give letters of recommendation to fraudsters. But it could be made a more formal policy. Letters could be required to include a statement about the candidate adhering to various formalized "good behavior" policies. Then when fraud happens there could be actual consequences for the letter writer. This would be like the culture in software development where you don't blame people for writing bad code, you blame reviewers for letting in bad code. (Idea comes from Angela Collier, in relation to other forms of antisocial behavior).

Re: Researchers taking on fraudulent science

#44

Post by NiloCK »

neilv wrote: post #38

One of the many troubling things about recently exposed patterns of fraudulent science: every dishonest scientist or professor is taking up a position and funding that could've instead gone to any of surely numerous people with much more integrity. Permit all the cronyism and gaming around university hiring and admissions, and don't hold universities responsible for unethical behavior of their faculty and graduates,…

> every dishonest scientist or professor is taking up a position and funding that could've instead gone to any of surely numerous people with much more integrity.

Maybe, maybe not. It feels possible at this point that the entire top of the competitive curve is actively juicing. Ie, with such fierce competition, there could be a vanishingly small cohort of researchers who both operate with integrity and whose CVs are competitive for faculty positions.

At best, we might expect a removed fraud to be replaced with someone marginally more honest.

Re: Researchers taking on fraudulent science

#45

Post by ChrisMarshallNY »

neilv wrote: post #38

One of the many troubling things about recently exposed patterns of fraudulent science: every dishonest scientist or professor is taking up a position and funding that could've instead gone to any of surely numerous people with much more integrity. Permit all the cronyism and gaming around university hiring and admissions, and don't hold universities responsible for unethical behavior of their faculty and graduates,…

> growth pump&exit investment scams

I'd go so far as to say that the mere existence of an "exit plan" is an indicator that the purported product may be flawed (the business may not be flawed, from an investor's PoV, but to them, the "product" is actually the business, itself, as opposed to what the business says is their product).

IMO: Real businesses are meant to be ongoing enterprises. Being sold to others, is supposed to be an unplanned (possibly fortunate and hoped-for) event. It shouldn't be an inevitable business plan milestone.

Re: Researchers taking on fraudulent science

#46

Post by tkgally »

derbOac wrote: post #14

I love these people's work. At the same time I don't think any of this really addresses the root causes of the problems, which are career, university administrative, and field (i.e., recognition and attention) incentives. The whole paradigm we use, and the way we treat scientists and fund research, is broken. It feels a little bit like bailing water out of your boat with teacup on one side while ignoring the gaping h…

One positive move would be for more institutions to publicly endorse the recommendations of the Declaration on Research Assessment. It won’t solve all of the incentive problems, but it should be a step in the right direction. I wrote about it briefly here a couple of weeks ago [1].

[1] https://news.ycombinator.com/item?id=39064322

Re: Researchers taking on fraudulent science

#47

Post by medstrom »

NiloCK wrote: post #44
neilv wrote: post #38

One of the many troubling things about recently exposed patterns of fraudulent science: every dishonest scientist or professor is taking up a position and funding that could've instead gone to any of surely numerous people with much more integrity. Permit all the cronyism and gaming around university hiring and admissions, and don't hold universities responsible for unethical behavior of their faculty and graduates,…

> every dishonest scientist or professor is taking up a position and funding that could've instead gone to any of surely numerous people with much more integrity. Maybe, maybe not. It feels possible at this point that the entire top of the competitive curve is actively juicing. Ie, with such fierce competition, there could be a vanishingly small cohort of researchers who both operate with integrity and whose CVs are…

To get more good research done, there are at least two different fronts we could fight on. The straightforward one is to reduce incentive to fake at the "top of the competitive curve", somehow. But the total population of researchers is still so small, it might not be the most important front.

The other front is to let the fakers keep faking, but change society so that everyone who wants to research out of personal interest has the free time to do so. They'll have less incentive to fake.

Re: Researchers taking on fraudulent science

#48

Post by jltsiren »

nwhitehead wrote: post #43
armchairhacker wrote: post #23

Earlier quoted context omitted.

But how do we “solve” those incentives? There will always be winners and losers, and since successful fraud is by definition indistinguishable from genuine success, there will always be people trying to game the system. The only effective solution I see is making fraud harder to hide. (There’s also making the penalty for fraud higher and trying to instill integrity, but the problem is some people will still take the…

One idea is to put more pressure on writers of letters of recommendation. There is obviously already some rather weak "reputational incentive" to not give letters of recommendation to fraudsters. But it could be made a more formal policy. Letters could be required to include a statement about the candidate adhering to various formalized "good behavior" policies. Then when fraud happens there could be actual consequen…

1. There is already a lot of administrative bloat in the academia, and ideas like this are responsible for most of it. Every time you replace an informal process with a formal one, you add to the bloat. And the formal process rarely works as intended.

2. Letter-writers have no control over the work of the person they recommend. They cannot prevent the submission of a manuscript or a grant. They would be taking a lot of risk for no gain.

3. Letters of recommendation are a form of corruption and nepotism. If you would have a conflict of interest for reviewing a manuscript or a grant, you should not be allowed to write a letter of recommendation either.

Re: Researchers taking on fraudulent science

#49

Post by gustavus »

derbOac wrote: post #14

I love these people's work. At the same time I don't think any of this really addresses the root causes of the problems, which are career, university administrative, and field (i.e., recognition and attention) incentives. The whole paradigm we use, and the way we treat scientists and fund research, is broken. It feels a little bit like bailing water out of your boat with teacup on one side while ignoring the gaping h…

We've had these incentives since we started science. Remember Albert Einstein wasn't able to get a teaching position until after he was winning the Nobel. It turns out the people that really love science will do it even if it doesn't pay anything and brings them nothing.

We just need to make sure the salesman scientists and parasitic beuracrats don't squash the efforts of those doing real work.

Re: Researchers taking on fraudulent science

#50

Post by gmd63 »

derbOac wrote: post #14

I love these people's work. At the same time I don't think any of this really addresses the root causes of the problems, which are career, university administrative, and field (i.e., recognition and attention) incentives. The whole paradigm we use, and the way we treat scientists and fund research, is broken. It feels a little bit like bailing water out of your boat with teacup on one side while ignoring the gaping h…

The solution is unfortunately for everyone, more transparency in research. When you conduct an experiment, why on earth is the report composed months, sometimes years later, instead of being streamed live and recorded?

Just the same as in business. We need a new more prestigious stock exchange for companies that are willing to open their books up to continual audit.

The government can step in to aid businesses and researchers that act openly, in the form of grants to researchers or tax breaks for members of the more restrictive stock exchange, and shift the cost of policing destructive fraud to rewards for constructive conduct.

Post reply on HN