Please Commit More Blatant Academic Fraud (2021) • Hacker News Board

Re: Please Commit More Blatant Academic Fraud (2021)

#81

Post by BeetleB » Fri, Feb 21, 2025, 5:11 PM UTC

archi42 wrote: post #29

cgcrob wrote: post #12

Earlier quoted context omitted.

If it’s any consolation I split up with an ex partner after she wanted to put me as a co-author on a pseudoscience bullshit paper that she was working on to try and hit her quota. Her entire field, in the social sciences, is inventing a wild idea and using meta analysis to give it credibility. Then flying to conferences and submitting expenses. I contributed nothing other than a statistical framework which was discar…

I treat all social science degrees as "likely bullshit" these days. Could as well be astrology. A few computer science friends of mine worked at a social science department during university. Their tasks included maintaining the computers, but also support the researchers with experiment design (if computers were involved) and statistical analysis. They got into trouble because they didn't want to use unsound or inco…

Let me introduce you to theoretical condensed matter physics, where no one cares if the data confirms the hypothesis, because they are writing papers about topics that very likely can never be tested.

At least in the social sciences there is an expectation of having some data!

Re: Please Commit More Blatant Academic Fraud (2021)

#82

Post by smallmancontrov » Fri, Feb 21, 2025, 5:14 PM UTC

ploynog wrote: post #45

Double-blind review is a mirage that does not hold up. While I was in academia I reviewed a paper that turned out to be a blatant case of plagiarism. It was a clear Level 1 copy according to the IEEE plagiarism levels (Uncredited Verbatim Copy of more than 50% of a single paper). I submitted all of these findings with the original paper and what parts were copied (essentially all of it) as my review. A few days later…

When we criticize without proposing a fix or alternative, we promote the implicit alternative of tearing something down without fixing it. This is often much worse than letting the imperfect thing stand. So here's a proposal: do what we do in software.

No, really: we have the same problem in software. Software developers under high pressure to move tickets will often resort to the minor fraud of converting unfinished features into bugs by marking them complete when they are not in fact complete. This is very similar to the minor fraud of an academic publishing an overstated / incorrect result to stay competitive with others doing the same. Often it's more efficient in both cases to just ignore the problem, which will generally self-correct with time. If not, we have to think about intervention -- but in software this story has played out a thousand times in a thousand organizations, so we know what intervention looks like.

Acceptance testing. That's the solution. Nobody likes it. Companies don't like to pay for the extra workers and developers don't like the added bureaucracy. But it works. Maybe it's time for some fraction of grant money to go to replication, and for replication to play a bigger role in gating the prestige indicators.

Re: Please Commit More Blatant Academic Fraud (2021)

#83

Post by tovej » Fri, Feb 21, 2025, 5:14 PM UTC

daveguy wrote: post #73

throw4847285 wrote: post #71

Unfortunately, I think this article is not pessimistic enough. If it were true that academic fraud is important for establishing the boundaries of respectable scholarship in a field, then behavioral economics wouldn't exist anymore. And evolutionary psychology would be a fraction of its current size. Trendiness trumps all notions of academic rigor, and as long as a field "feels like" it's on the cutting edge it can g…

I'm wondering how someone knows the size of the field of evolutionary psychology and what it should be instead of what it is so well that they feel no evidence is necessary. Are you in the field yourself? Or do you have questions about evolution in general? Or none of the above?

Evolutionary psychology is highly speculative and is widely known to be full of questionable results.

So much so that there's a wikipedia page for how bad it is: https://en.m.wikipedia.org/wiki/Criticism_of_evolutionary_ps...

Re: Please Commit More Blatant Academic Fraud (2021)

#84

Post by wtallis » Fri, Feb 21, 2025, 5:17 PM UTC

rors wrote: post #78

ploynog wrote: post #45

Double-blind review is a mirage that does not hold up. While I was in academia I reviewed a paper that turned out to be a blatant case of plagiarism. It was a clear Level 1 copy according to the IEEE plagiarism levels (Uncredited Verbatim Copy of more than 50% of a single paper). I submitted all of these findings with the original paper and what parts were copied (essentially all of it) as my review. A few days later…

I remember attending ACL one year, where the conference organisers ran an experiment to test the effectiveness of double blind reviews. They asked reviewers to identify the institution that submitted the anonymised paper. Roughly 50% of reviewers were able to correctly identify the institutions. I think there was a double digit percentage of being able to predict authors. The organisers then made the argument that do…

You're saying "not working" when you only have presented evidence for "not perfect".

That experiment showed that even when asked to put effort into identifying the source of an anonymized paper—something that most reviewers probably don't put any conscious effort into normally—the anonymization was having a substantial effect compared to not anonymizing the papers.

Am I missing some obvious reason why double-blind reviews should only be attempted if the blinding can be achieved with a near-perfect success rate, or are you just setting the bar unreasonably high?

Re: Please Commit More Blatant Academic Fraud (2021)

#85

Post by BeetleB » Fri, Feb 21, 2025, 5:18 PM UTC

jimbokun wrote: post #72

> Most researchers are to some extent “career researchers”, motivated by the power and prestige that rewards those who excel in the academic system, rather than idealistic pursuit of scientific truth. This is the funny part. There is little to no power and prestige to be had in the academic system. To a first approximation no one outside academia cares. I was just working as a staff programmer and taking grad courses…

In the social sciences, there is a lot of prestige to be had. Do some ground breaking work, engage with the public via bestselling books, and then get invited by the president to work on policy.

Even if they don't work for the administration, there are plenty of other bodies that will value them and pay large sums of money (or let them have large influence).

Very common amongst economists, and more and more common amongst disciplines like psychology.

Even in technical fields, if you can manage to become a big name, you can do consulting work and get paid quite well.

> Then got a job in industry making far more money

This is not a healthy way to look at it.

The average mechanical engineer isn't making tons of money in industry. A ME professor at a top university likely makes more. A biology major with just a BS degree will make less than the average biology associate professor.

But more importantly, there's a simpler reason why money is a poor metric to measure: You can always get more money in finance or medicine than as a mechanical engineer. Does it make sense to denigrate a whole profession just because one can make more money elsewhere?

Re: Please Commit More Blatant Academic Fraud (2021)

#86

Post by proto-n » Fri, Feb 21, 2025, 5:37 PM UTC

michaelt wrote: post #56

proto-n wrote: post #52

Earlier quoted context omitted.

It's really not that hard to come up with ways to commit fraud if you want to. On the other hand, it's very easy to make such mistakes if you don't know to avoid them. This characterization is very misguided IMO.

Ah, perhaps I wasn't clear about what I'm trying to say. I don't think we should stop training researchers in common mistakes and fraudulent methods to watch out for. I'm just saying: I don't believe anyone actually tells budding researchers that they should commit fraud. Instead I think the process is probably more like this: Year 1: Statistics/research training. Here are a load of subtle mistakes to watch out for a…

I still disagree. It's more like "omg I should have published at least a few papers by now, what am I doing" and then you start frantically looking for provable things in the dataset. You find one that you can also support with a nice story. Now either a) You were not tought about how or why this is wrong and you publish the paper b) You were, and know that you should collect a separate dataset to test the hypothesis. But also, there is a huge existential pressure to just close your eyes and roll with it.

It's not that you need to be tought how to cheat, it's that you need to be tought how to avoid unintentionally cheating.

Re: Please Commit More Blatant Academic Fraud (2021)

#87

Post by araes » Fri, Feb 21, 2025, 5:39 PM UTC

BeetleB wrote: post #81

archi42 wrote: post #29

Earlier quoted context omitted.

I treat all social science degrees as "likely bullshit" these days. Could as well be astrology. A few computer science friends of mine worked at a social science department during university. Their tasks included maintaining the computers, but also support the researchers with experiment design (if computers were involved) and statistical analysis. They got into trouble because they didn't want to use unsound or inco…

Let me introduce you to theoretical condensed matter physics, where no one cares if the data confirms the hypothesis, because they are writing papers about topics that very likely can never be tested. At least in the social sciences there is an expectation of having some data!

That's actually the part about people constantly negging on social sciences [1] that I often find confusing.

There's huge amounts of data available (geography, lots and lots of maps; history, huge amount of historical documentation; economics, vast amounts of public datasets produced every month by most governments; political science, censuses, voting records, driver registrations, political contest results all over the Earth - often for decades if not centuries).

Most is relatively well verified, and often tells you how it was verified [2]. Often it's obtainable in publicly available datasets that numerous other researchers can verify was obtained from a legitimate source. [3][4][5][6][7][8][9][10][11][12]

There's lots of data available. Much is also verifiable in a very personal way simply by walking somewhere and looking. In many ways, social sciences should be one of the most rigorous disciplines in most of academia.

[1] Using Wikipedia's grouping on "social sciences" (anthropology, archaeology, economics, geography, history, linguistics, management, communication studies, psychology, culturology and political science): https://en.wikipedia.org/wiki/Social_science

[2] Census 2020, Data Quality: https://www.census.gov/programs-surveys/decennial-census/dec...

[3] Economic Indicators by Country: https://tradingeconomics.com/indicators

[4] Our World in Data (with Demographics, Health, Poverty, Education, Innovation, Community Wellbeing, Democracy): https://ourworldindata.org/

[5] Observatory of Economic Complexity: https://oec.world/en

[6] iNaturalist (at least from a biological history perspective): https://www.inaturalist.org/taxa/43577-Pan-troglodytes

[7] Coalition for Archaeological Synthesis, Data Sources: https://www.archsynth.org/resources/data-sources/

[8] Language Goldmine (linguistics datasets): http://languagegoldmine.com/

[9] Pew Research (regular surveys on economics, political science, religion, communication, psychology - usually 10,000 respondents United States, 1000 respondents international): https://www.pewresearch.org/

[10] Marinetraffic (worldwide cargo shipping): https://www.marinetraffic.com/en/ais/home/centerx:-12.0/cent...

[11] Flightradar Aviation Data (people movement): https://www.flightradar24.com/data

[12] Windy Worldwide Web Cameras: https://www.windy.com/?42.892,-104.326,5,p:cams

Re: Please Commit More Blatant Academic Fraud (2021)

#88

Post by ninalanyon » Fri, Feb 21, 2025, 5:48 PM UTC

cess11 wrote: post #30

archi42 wrote: post #29

Earlier quoted context omitted.

I treat all social science degrees as "likely bullshit" these days. Could as well be astrology. A few computer science friends of mine worked at a social science department during university. Their tasks included maintaining the computers, but also support the researchers with experiment design (if computers were involved) and statistical analysis. They got into trouble because they didn't want to use unsound or inco…

Sounds like economics. Research fraud is common pretty much everywhere in academia, especially where there's money, i.e. adjacent to industry.

It does rather depend on the industry. Research in fields relevant to electrical engineering are much less likely to be fraudulent because the industry actually uses the results to make the products and the customers depend on those products working as specified.. If you discover a better and cheaper ceramic insulator you can be confident that transformer manufacturers will take it up but the big companies are well stocked with experts in the field so a fraudulent paper will quickly be spotted.

Re: Please Commit More Blatant Academic Fraud (2021)

#89

Post by porridgeraisin » Fri, Feb 21, 2025, 6:08 PM UTC

bjackman wrote: post #19

wucke13 wrote: post #10

Tangentially relevant: Gernots list of benchmarking crimes. https://gernot-heiser.org/benchmarking-crimes.html

Ha, I thought this would be a really useful resource but I think the people the author is complaining about do much better than most benchmarking I see in the industry Almost all the benchmarking results I see is just a percentage difference between two algebraic means, no statistical analysis whatsoever. Very common interaction: QA folks say "your change degraded some of our metrics and improved some others". I know…

Yep. The most recent example that's stuck in my head is actually much worse: they didn't even take the mean! One sample!

https://github.com/denoland/pm-benchmark

Check the run bench shell script (there's not much else in the repo anyways)

Re: Please Commit More Blatant Academic Fraud (2021)

#90

Post by tensor » Fri, Feb 21, 2025, 6:28 PM UTC

ploynog wrote: post #45

Double-blind review is a mirage that does not hold up. While I was in academia I reviewed a paper that turned out to be a blatant case of plagiarism. It was a clear Level 1 copy according to the IEEE plagiarism levels (Uncredited Verbatim Copy of more than 50% of a single paper). I submitted all of these findings with the original paper and what parts were copied (essentially all of it) as my review. A few days later…

Seems to me that the review worked, you caught the plagiarism, even though the other two missed it. It's disturbing that somehow the paper author found your contact information though!