Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cracking down on research fraud

undark.org

Post reply on HN
Print view View original

211–220 of 227 posts

Re: Cracking down on research fraud

#211

Post by Chris2048 »

adminprof wrote: post #143
martincmartin wrote: post #91

Earlier quoted context omitted.

What do you suggest as an alternative?

This entire thread is really frustrating for me to read. A lot of these ideas are one-liners from people who have had very little experience with different parts of academia, except maybe being a student. Every single idea here are debated to death by academics already (not just university academics, but funding agencies, academic societies, award panels, journal editors and conference organizers), with changes happe…

> If it was this simple that a random person here could come up with how to "solve" academia, we'd have already done it decades ago.

Just because an idea is simple to come up with, doesn't mean it's also easy to implement.

Also, can you explain to me how your programming analogy works? Because I agree with a lot of it, though it seems I'm not supposed to.

Re: Cracking down on research fraud

#213

Post by moralestapia »

rscho wrote: post #177
moralestapia wrote: post #121

Earlier quoted context omitted.

>labs operate largely as fiefdoms of the individual principal investigators To add to the injury, you wouldn't believe the extent of the actions to which some PIs become engaged in order to attain/preserve their 'power'. Illegal, unethical and pathetic. The core problem is that PIs are human, and humans are flawed. The solution (if there is one) should take this into account, and somehow try to reduce it systematical…

> The core problem is that PIs are human Actually, the problematic behaviours you describe are encouraged at the institutional level, because institutions also get more funding if they output more/better research. The fact that postdocs don't want to commit scientific suicide for the sake of morals in that context should surprise no one. The ground truth is that scientific research has been turned into a poorly regul…

>[...] the problematic behaviours you describe are encouraged at the institutional level

Yes, which is why I wrote,

>[...] and somehow try to reduce it systematically, at an institutional level.

Re: Cracking down on research fraud

#214

Post by adminprof »

marcus_holmes wrote: post #173
adminprof wrote: post #143

Earlier quoted context omitted.

This entire thread is really frustrating for me to read. A lot of these ideas are one-liners from people who have had very little experience with different parts of academia, except maybe being a student. Every single idea here are debated to death by academics already (not just university academics, but funding agencies, academic societies, award panels, journal editors and conference organizers), with changes happe…

So, what's the best proposal or line of thinking coming from all this deep thought about how to solve it? Because you can't possibly be saying that the current system is the best we can do, or that the problem is intractable. (I agree with most of your proposals about software engineering btw. We should do more of this).

It'd be a process: participant in a community in a subfield, listen to ideas that have been tried and outcomes, come up with an idea using those lessons learned and success metrics, build consensus around a new idea, test in a single-instance (like one conference, grant review panel, tenure committee at one university), share lessons to the field, do this for a couple of years to show clear success, expand to multiple events in the field, become an exemplar field for that idea and "infect" other fields

Sounds slow but there's thousands of such experiments happening simultaneously right now. This is how a long of major field-sized changes have happened, like the transition to conferences from journals (which had many initial problems like during tenure review or a lack of quality in reviews), etc. Ideas will lose traction at various stages (for example, there was a movement some time ago to use alpha=0.001 instead of alpha=0.05 for null hypothesis testing, which has been limited to that field or subfield).

Re: Cracking down on research fraud

#215

Post by adminprof »

Chris2048 wrote: post #211
adminprof wrote: post #143

Earlier quoted context omitted.

This entire thread is really frustrating for me to read. A lot of these ideas are one-liners from people who have had very little experience with different parts of academia, except maybe being a student. Every single idea here are debated to death by academics already (not just university academics, but funding agencies, academic societies, award panels, journal editors and conference organizers), with changes happe…

> If it was this simple that a random person here could come up with how to "solve" academia, we'd have already done it decades ago. Just because an idea is simple to come up with, doesn't mean it's also easy to implement. Also, can you explain to me how your programming analogy works? Because I agree with a lot of it, though it seems I'm not supposed to.

Sure I guess I didn't make that very clear. Just going down the line:

1) stop releasing before the bugs are fixed. Software and games are rushed out. Companies need to take their time to fix the bugs so the users don't have to encounter them.

- This is a tradeoff between shipping time and bugs. You will never fix all bugs, so it's unrealistic. And it's a business decision in many cases. Even the definition of a bug is tricky, like is a usability issue a bug? So it's just a naive idea.

2) no more technical debt. Programmers are sloppy and introduce technical debt because they are not incentivized to do high quality programs.

- No one wants to create technical debt. Obviously it slows down development later on. Again this could be a business decision. Some programs like one-off data science scripts don't need to fix all their technical debt. Technical debt also accrues naturally (like just changing environment, platform, standards) so it's not possible to aim to not have debt in the very beginning. Hindsight is 20-20 and all that. Saying programmers are not incentivized to do high quality programs is just a blanket naive statement, and depends on the definition of high quality programs.

3) cap team sizes. Everyone knows that large teams fail more spectacularly. Gmail and Napster and the original version of Google were made by a group of 4 people. Software teams need to be 4-5 people max.

- Depends on the type of software. Can't just generalize given a few token examples. Expectations also change over the course of the product.

4) programmers must use a transparency scorecard. Software companies like Oracle and IBM charge ridiculous amounts for their work. They hide costs and cut corners. Programmers should be transparent about the work they are doing each day, what data they access, and which functions they are writing.

- This uses one subsection of the software economy to make a point (as a fallacy). But also some of these measures don't make sense, like some programmers read a lot of code or delete lines, and so the metrics are not generalizable.

In summary, these are ideas that someone who has not done long-term software development would say, or someone who has only had experience with one type of software or company would say. They're not well defined, not generalizable, and don't account for the complex and varied sociotechnical process that software development is.

Re: Cracking down on research fraud

#216

Post by marcus_holmes »

adminprof wrote: post #214
marcus_holmes wrote: post #173

Earlier quoted context omitted.

So, what's the best proposal or line of thinking coming from all this deep thought about how to solve it? Because you can't possibly be saying that the current system is the best we can do, or that the problem is intractable. (I agree with most of your proposals about software engineering btw. We should do more of this).

It'd be a process: participant in a community in a subfield, listen to ideas that have been tried and outcomes, come up with an idea using those lessons learned and success metrics, build consensus around a new idea, test in a single-instance (like one conference, grant review panel, tenure committee at one university), share lessons to the field, do this for a couple of years to show clear success, expand to multipl…

Interesting, thanks.

Is the move to pre-publishing servers (like arxiv) a part of this? How does SciHub (and similar) figure into it?

It does sound slow, and a bit trivial, if I'm honest. Are there any examples you can share of successful experiments that have travelled across field boundaries?

Re: Cracking down on research fraud

#217

Post by moralestapia »

MaxBarraclough wrote: post #199
moralestapia wrote: post #164

Earlier quoted context omitted.

Perhaps I was in an extremely corrupt environment (which eventually made me leave), but I would put the "intentional fraud" rate as high as 30% or so. I pointed out one of such instances once and that was the beginning of the end for me in that place. More: My field was Biology. No one double checks anyone or anything. A measurement comes back and you can just pretty much ignore it, in the end you just write whatever…

> I pointed out one of such instances once and that was the beginning of the end for me in that place. How does that occur? Did they invent some reason you couldn't remain? Were you intending to remain anonymous when reported it? Lastly, were there consequences for the fraudsters?

Power games.

In my specific scenario, she said I was underperforming and didn't want me anymore as her student, there's nothing much one could do after that.

Of course I tried to defend my stance, even provided solid evidence of her wrongdoings and overall attitude towards me and other students. In the end it was "just better if you just leave".

>Were there consequences for the fraudsters?

Zero.

Re: Cracking down on research fraud

#218

Post by dodobirdlord »

adamsea wrote: post #154
dodobirdlord wrote: post #126

Earlier quoted context omitted.

It would be far more fair to say that we have no problem with infinite funding for non-discretionary entitlement spending, since that's most of that the Federal Government spends money on. Military spending in the US is not wildly outside of the norm for the developed world as a percentage of GDP, and money spent to "bail out banks" is a rounding error in terms of spending, and has historically turned a profit for th…

What is entitlement spending? These? "Social Security and Medicare are sometimes called "entitlements," because people meeting relevant eligibility requirements are legally entitled to benefits, although most pay taxes into these programs throughout their working lives." https://en.wikipedia.org/wiki/Expenditures_in_the_United_Sta...

Pretty much, “entitlement” isn’t a technical term but it refers to government spending on pre-defined benefit programs. A better term above might have been Non-discretionary Spending, but most of it is entitlement spending. And yes, Social Security and Medicare are the two most significant components, followed by Medicaid as the third. Those three plus interest on debt constitute the majority of the federal budget.

Re: Cracking down on research fraud

#219

Post by Balgair »

gwerbret wrote: post #77

These issues of research fraud come up often, but the root of the problem is a bit more subtle. In North America at least, biomedical research labs operate largely as fiefdoms of the individual principal investigators (PIs). The actual research work falls almost entirely on the backs of grad students and postdoctoral fellows. The grad students need to generate "good" data in order to graduate, the postdocs need the s…

Aside: To give readers an idea of the financial pressures I have an example. One of my former professors told me that the lab spaces costs ~$45/sqft/mo from the university, 300sqft minimum. That's ~$160k/year just for the floor space in just the lab, not the office space either. Now, electrical, heat, vivarium, elevators, janitors, security, etc. are all rolled up into that. Bit that's not specific equipment costs, reagents, grad students, and the profssor's own salary. Granted, every university is different (just look at patent ownership clauses).

It's not cheap and you must win grants just to pay rent for your own apartment. Especially early on in your career, it's a real struggle.

Re: Cracking down on research fraud

#220

Post by TuringNYC »

VHRanger wrote: post #197
TuringNYC wrote: post #194

Earlier quoted context omitted.

https://en.wikipedia.org/wiki/Peter_R._Orszag (NOTE: NOT suggesting he cheated, just noting that lots of people have fast-tracks pre-planned for them w/o the hassle of working their way up.) Not quite running the FED, but look at the years, pretty close. Orszag earned an A.B. summa cum laude in economics from Princeton University in 1991 after completing an 80-page long senior thesis titled "Congressional Oversight o…

Assistants to the president are often these fresh out of school types. Obamas speechwriter were in their early 20s as well for instance. The Fed is way more stringent - - a fresh PhD would be an analyst or research associate, nothing more. State departments and organizations have a lot more hierarchy for better or worse.

He was "Senior Advisor" before even graduating. He was a WH Director within 10yrs. Three years later, he was Vice Chairman of a global bank (Citigroup).
Post reply on HN