Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Please Commit More Blatant Academic Fraud (2021)

jacobbuckman.com

Post reply on HN
Print view View original

91–100 of 121 posts

Re: Please Commit More Blatant Academic Fraud (2021)

#91

Post by tensor »

SirHumphrey wrote: post #38
tonyg wrote: post #24

> Undermining the credibility of computer science research is the best possible outcome for the field, since the institution in its current form does not deserve the credibility that it has. Horseshit. This might be true for AI research (and even there that's an awfully broad brush you're using, mate), but it's certainly not true for other areas of computer science.

Maybe your lab is different (if you work in a research setting), but most of the researchers will readily admit that most of their research output is at least somewhat bull**. It's something that is trained in to people from high school research projects onwards - people judging your results usually do not have time or ability to research you work and even if they have they usually have much more importing things to…

Excuse me? Based on what? In my time in academia exactly zero researchers would claim that their work is somewhat bullshit.

As a society, there is laughably little support for science, instead the majority of policy and business decisions are based on fairy tales and snake oil. We need more trust in science.

Re: Please Commit More Blatant Academic Fraud (2021)

#92

Post by throw4847285 »

tovej wrote: post #83
daveguy wrote: post #73

Earlier quoted context omitted.

I'm wondering how someone knows the size of the field of evolutionary psychology and what it should be instead of what it is so well that they feel no evidence is necessary. Are you in the field yourself? Or do you have questions about evolution in general? Or none of the above?

Evolutionary psychology is highly speculative and is widely known to be full of questionable results. So much so that there's a wikipedia page for how bad it is: https://en.m.wikipedia.org/wiki/Criticism_of_evolutionary_ps...

The SEP page is a slightly more rigorous take, though unlike Wikipedia it is clearly primarily authored by one person, a philosopher of biology who I don't know much about. Still, I think it's a fair and detailed overview of the field, from a methodological perspective.

https://plato.stanford.edu/entries/evolutionary-psychology/

Re: Please Commit More Blatant Academic Fraud (2021)

#93

Post by throw4847285 »

daveguy wrote: post #73
throw4847285 wrote: post #71

Unfortunately, I think this article is not pessimistic enough. If it were true that academic fraud is important for establishing the boundaries of respectable scholarship in a field, then behavioral economics wouldn't exist anymore. And evolutionary psychology would be a fraction of its current size. Trendiness trumps all notions of academic rigor, and as long as a field "feels like" it's on the cutting edge it can g…

I'm wondering how someone knows the size of the field of evolutionary psychology and what it should be instead of what it is so well that they feel no evidence is necessary. Are you in the field yourself? Or do you have questions about evolution in general? Or none of the above?

Just cut out everybody who is studying "sexual selection" and I think the field is way healthier. Weird that so many evo psych scholars of a certain persuasion are fixated on that. I have a theory, but it veers into armchair non-evolutionary psychology, and it's not very nice.

Re: Please Commit More Blatant Academic Fraud (2021)

#94

Post by huijzer »

pbronez wrote: post #66
huijzer wrote: post #48

Earlier quoted context omitted.

> I've become so skeptic over the years that I assume almost all papers to be lies until proven otherwise. I couldn't agree more. I have read a lot of psychology papers during my PhD and I think there is very little signal in the papers. Many empirical papers for example use basically the same "gold standard" analysis, which is fundamentally flawed in many ways. One problem for example is that if you would use anothe…

How could this be corrected? The scientific community has lost a lot of credibility with the public, and the backlash is obvious in recent policy changes. Fast forward four years. Assume Trump and RFK jr have successfully destroyed the current system. What should replace it? How could the Federal government ensure that public monies only fund high quality research? Could policy re-shape the incentives and unlock a he…

I don’t know but these are exactly the right questions to ask!

Re: Please Commit More Blatant Academic Fraud (2021)

#95

Post by gopher_space »

wtallis wrote: post #84
rors wrote: post #78

Earlier quoted context omitted.

I remember attending ACL one year, where the conference organisers ran an experiment to test the effectiveness of double blind reviews. They asked reviewers to identify the institution that submitted the anonymised paper. Roughly 50% of reviewers were able to correctly identify the institutions. I think there was a double digit percentage of being able to predict authors. The organisers then made the argument that do…

You're saying "not working" when you only have presented evidence for "not perfect". That experiment showed that even when asked to put effort into identifying the source of an anonymized paper—something that most reviewers probably don't put any conscious effort into normally—the anonymization was having a substantial effect compared to not anonymizing the papers. Am I missing some obvious reason why double-blind re…

The subtext to this whole comment chain is that you need to have hands-on experience with qualitative to quantitative conversions if you want to reason about the scientific process.

> Am I missing some obvious reason why double-blind reviews should only be attempted if the blinding can be achieved with a near-perfect success rate, or are you just setting the bar unreasonably high?

OP thinks you are looking at either signal or noise, instead of determining where the signal begins for yourself.

Re: Please Commit More Blatant Academic Fraud (2021)

#96

Post by cess11 »

ninalanyon wrote: post #88
cess11 wrote: post #30

Earlier quoted context omitted.

Sounds like economics. Research fraud is common pretty much everywhere in academia, especially where there's money, i.e. adjacent to industry.

It does rather depend on the industry. Research in fields relevant to electrical engineering are much less likely to be fraudulent because the industry actually uses the results to make the products and the customers depend on those products working as specified.. If you discover a better and cheaper ceramic insulator you can be confident that transformer manufacturers will take it up but the big companies are well s…

Graphene in electrical engineering is a staple of every (dis)reputable papermill.

Re: Please Commit More Blatant Academic Fraud (2021)

#97

Post by ramblenode »

smallmancontrov wrote: post #82
ploynog wrote: post #45

Double-blind review is a mirage that does not hold up. While I was in academia I reviewed a paper that turned out to be a blatant case of plagiarism. It was a clear Level 1 copy according to the IEEE plagiarism levels (Uncredited Verbatim Copy of more than 50% of a single paper). I submitted all of these findings with the original paper and what parts were copied (essentially all of it) as my review. A few days later…

When we criticize without proposing a fix or alternative, we promote the implicit alternative of tearing something down without fixing it. This is often much worse than letting the imperfect thing stand. So here's a proposal: do what we do in software. No, really: we have the same problem in software. Software developers under high pressure to move tickets will often resort to the minor fraud of converting unfinished…

> This is very similar to the minor fraud of an academic publishing an overstated / incorrect result to stay competitive with others doing the same.

I completely disagree.

For one, academic standards of publishing are not at all the same as the standards for in-house software development. In academia, a published result is typically regarded as a finished product, even if the result is not exhaustive. You cannot push a fix to the paper later; an entirely new paper has to be written and accepted. And this is for good reason: the paper represents a time-stamp of progress in the field that others can build off of. In the sciences, projects can range from 6 months to years, so a literature polluted with half-baked results is a big impediment to planning and resource allocation.

A better comparison for academic publishing would be a major collaborative open source project like the Linux kernel. Any change has to be thoroughly justified and vetted before it is merged because mistakes cause other people problems and wasted time/effort. Do whatever you like with your own hobbyist project, but if you plan for it to be adopted and integrated into the wider software ecosystem, your code quality needs to be higher and you need to have your interfaces speced out. That's the analogy for academic publishing.

The problems in modern academic publishing are almost entirely caused by the perverse incentives of measuring academic status by publication record (number of publications and impact factor). Lowering publishing standards so academics can play this game better is solving the wrong problem. Standards should be even higher.

Re: Please Commit More Blatant Academic Fraud (2021)

#98

Post by [deleted] »

smallmancontrov wrote: post #82
ploynog wrote: post #45

Double-blind review is a mirage that does not hold up. While I was in academia I reviewed a paper that turned out to be a blatant case of plagiarism. It was a clear Level 1 copy according to the IEEE plagiarism levels (Uncredited Verbatim Copy of more than 50% of a single paper). I submitted all of these findings with the original paper and what parts were copied (essentially all of it) as my review. A few days later…

When we criticize without proposing a fix or alternative, we promote the implicit alternative of tearing something down without fixing it. This is often much worse than letting the imperfect thing stand. So here's a proposal: do what we do in software. No, really: we have the same problem in software. Software developers under high pressure to move tickets will often resort to the minor fraud of converting unfinished…

[deleted]

Re: Please Commit More Blatant Academic Fraud (2021)

#99

Post by BeetleB »

araes wrote: post #87
BeetleB wrote: post #81

Earlier quoted context omitted.

Let me introduce you to theoretical condensed matter physics, where no one cares if the data confirms the hypothesis, because they are writing papers about topics that very likely can never be tested. At least in the social sciences there is an expectation of having some data!

That's actually the part about people constantly negging on social sciences [1] that I often find confusing. There's huge amounts of data available (geography, lots and lots of maps; history, huge amount of historical documentation; economics, vast amounts of public datasets produced every month by most governments; political science, censuses, voting records, driver registrations, political contest results all over…

A lot of psychology research involves data not from these datasets, though.

The complaint is that their data often doesn't strongly support the hypothesis, and dubious statistical techniques are performed to make it appear otherwise. And just poor statistics abilities (not malicious intent).

Physicists get away with it because they often just don't do any statistics. Often the data aligns so well with the hypothesis that you don't need any sophisticated techniques, or their work doesn't involve any data (like my example in my prior comment).

Most US trained physicists have never taken a course in statistics. It's not in the curriculum in most universities. When I was in school and would point it out, the response was always "Why do we need a whole course in statistics? We learn it in quantum mechanics."

No. That's probability you learn. Not statistics.

In social sciences (and medicine) people take a lot more statistics courses because the systems are much more complex than typical physics systems. A lot more confounding variables, etc. They simply need more statistics.

(Yes, yes. I know. There's probably some experimental branch in physics where people actually do use statistics. But most don't).

Re: Please Commit More Blatant Academic Fraud (2021)

#100

Post by autoexec »

GuestFAUniverse wrote: post #7

I am a co-author on a paper I never asked for, but my supervisor insisted, because the petty idea upgrading his desperate try to the point of "considerable at all" came from me. It was a chair which normally prouded itself of only publishing in the most highly regarded journals of its field (internally graded A, B, C). They had a few years without viable paper. Desperate to publish. From my POV this was a D. The pape…

Maybe we need an "Alan Smithee" (https://en.wikipedia.org/wiki/Alan_Smithee) for research papers.
Post reply on HN