I tried to report scientific misconduct. How did it go? • Hacker News Board

Re: I tried to report scientific misconduct. How did it go?

#281

Post by auganov » Wed, Jan 27, 2021, 4:14 PM UTC

MaxBarraclough wrote: post #142

DarkWiiPlayer wrote: post #137

Earlier quoted context omitted.

Every now and then I go through some of wikipedias sources on certain social topics because I don't trust them at all. The amount of BS I've found in papers, even though I don't eve have any research background at all is impressive. My favourite was probably this one paper where the author essentially made a reddit-post asking a community about themselves, then cherry-picked (the post is still up, with timestamps and…

Did you edit the Wikipedia page or challenge the interpretation in the talk page?

I wanted to edit an article about an obscure religious group that included some blatantly wrong statements about the espoused ideology. They had an academic tertiary source making these claims extrapolated from reaearch by the same author that made both plausible claims but also included similar inferences. Being a very obscure group there aren't many other academic sources discussing it. All literature that could disprove these claims comes from non-academics affiliated with the group which are a no-go.

As per Wikipedia rules (which took hours to figure out), there's not much one can do short of getting some impartial or friendly academic to publish a more reasonable article.

Re: I tried to report scientific misconduct. How did it go?

#282

Post by antonvs » Wed, Jan 27, 2021, 4:16 PM UTC

analog31 wrote: post #234

xkde wrote: post #198

Earlier quoted context omitted.

I think people have to get more serious about separating science as a procedure from scientism (that is, philosophical issues that are often discussed in tandem). When one uses the phrase, “science denier”, it often means, “you don’t agree with my philosophy/metaphysics/economic policy” rather than “you deny these particular facts”, which causes people to be rightly concerned. I’m not optimistic that this is going to…

I tend to limit my use of "science denier" when an organization or its followers systematically deny scientific knowledge on multiple unrelated fronts. Interestingly, I have read that in the 1920s and 30s, there was actually an organized relativity denialist movement, that wrote articles and held public protests.

Relativity was a huge philosophical shift from the comparative simplicity of Newton's laws. It's not surprising that there was resistance to it.

Tesla was famously against relativity, telling the New York Times, "Einstein’s relativity work is a magnificent mathematical garb which fascinates, dazzles and makes people blind to the underlying errors. The theory is like a beggar clothed in purple whom ignorant people take for a king".

Re: I tried to report scientific misconduct. How did it go?

#283

Post by SubiculumCode » Wed, Jan 27, 2021, 4:16 PM UTC

nxpnsv wrote: post #98

ALittleLight wrote: post #61

Earlier quoted context omitted.

That doesn't seem to be the case in the story in the OP. The problematic author is still publishing at a great pace.

Which is weird... I guess reviewers are lacking in stats education.

I mean, those are pretty low tier journals but...

Re: I tried to report scientific misconduct. How did it go?

#284

Post by kenjackson » Wed, Jan 27, 2021, 4:16 PM UTC

peytn wrote: post #154

kenjackson wrote: post #140

Earlier quoted context omitted.

This still don’t make sense. For example, I want to paint my house with a less toxic paint. I can’t trust any academic research. I have to now research what is toxic in paint? Then I have to find ways to measure various chemicals and gases? Etc... This seems like a complete utter waste of time. In real life most life impacting academic research is much more right than wrong. You are far better served assuming so. Unl…

I think what you’re missing is that academic research focuses on novelty, not basic facts. Ultimately not trusting novelty can save time. Basic facts can be found in reference material. So it’s more like suppose you want to paint your house green, and you read that somebody says you can mix red and blue paint to make a really cool green paint. Instead of immediately going out and buying enough red and blue paint to c…

It’s not as simple as buying paint. You’re not going to use any treatment where research came from a medical school or associated institute without personally proving it works first? Good luck!

If making green paint is impossible I think that it will eventually self correct, or is simply inconsequential. In some instances it may take a while, but if the alternative is to reprove a result before using it — that seems like something only a fool would do or someone with infinite time.

Re: I tried to report scientific misconduct. How did it go?

#285

Post by lumost » Wed, Jan 27, 2021, 4:19 PM UTC

foobarian wrote: post #248

lumost wrote: post #243

Earlier quoted context omitted.

The number of incentives driving this kind of activity in science is disheartening. So many papers get published, few are read widely, and even fewer are replicated, they'll still get citations if the talk circuit is played right. Citations are what advance a scientist in their career, and anything that could be tossed off as an unfortunate statistical anomaly or error is unlikely to end a career. In such a world, "o…

The thing I find disheartening is that if fraudulent results are being cited, it must mean that the mechanism of "standing on the shoulders of giants" is not working. One would expect that these papers would be contributions that citing scientists could benefit from and use in their own work with impact. For example if scientist A truly developed an O(N) sorting algorithm, then a scientist B might use it in their wor…

> And the citations don't actually reflect the true dependencies, but some political/social graph instead. Too bad.

I think this gets to the major concern with Academia today, as it becomes somewhat of a self-reinforcing feedback loop. Curry citations with political savvy, get awarded grants due to citations and political savvy, show that you are productive due to citations, grants, and political savvy - earning yet more political capital.

This will probably become my go to explanation for why Academic CS research has largely become decoupled from industrial application and industrial research. While political savvy is important in a large corporation, eventually you need to produce results.

Re: I tried to report scientific misconduct. How did it go?

#286

Post by throw998 » Wed, Jan 27, 2021, 4:20 PM UTC

qwantim1 wrote: post #182

renewiltord wrote: post #113

Earlier quoted context omitted.

My cousin was a student at a lab where a sketchy grad student doctored results too. She was majorly sketched out by the whole thing and that the PI supported the whole op. It was very painful and set her back a bit but she managed to switch to a different lab and do proper work, defend her thesis, graduate, and get far away from those people. Now we just shake our head in disbelief at them but at that point it was fa…

Attention and effort will go to “well-presented fake”. Marie Curie believed that radioactivity might have been caused by ghosts or the paranormal because of such things.[1] While there may actually be ghosts or other things paranormal, I’d bet that Marie Curie was fooled. The good part is that Curie’s work persists, and we think we have more understanding about radioactive substances. I’m not sure whether she had to…

I find this fascinating. If we allow ourselves to entertain the idea of quantum time paradoxes, could it be that the radioactivity was in fact caused by the ghost of Marie Curie herself? She would have a very strong and obvious reason to haunt the science.

Re: I tried to report scientific misconduct. How did it go?

#287

Post by searine » Wed, Jan 27, 2021, 4:29 PM UTC

LockAndLol wrote: post #187

How are we, the laymen, supposed to trust published, peer-reviewed papers? They seem to be just a bunch of words now with little meaning. Before, I used to consider scientific papers... well, scientific and would try to base an informed opinion on the abstracts and conclusions. After reading blog entries like these and actually reading some papers (especially soft science papers which are easier to understand), even…

>How are we, the laymen, supposed to trust published, peer-reviewed papers?

You are not supposed to.

The laymen really isn't the intended audience of academic publications. The literature is always in flux and inherently unreliable as discoveries are claimed and over decades, proven true or false. Taking a snapshot of the literature at any one time is to accept that a proportion of the claimed truth will be false. Unfortunately the laymen doesn't get this, and they believe that published=true.

As a layman, you should be looking to sources of information that have been vetted for truth, like textbooks. Textbooks are made to distill the most reliable information from the literature by a team of experts.

One paper isn't truth, a dozen independent papers, all pointing to the same thing is. That is what we call the "scientific consensus".

Re: I tried to report scientific misconduct. How did it go?

#288

Post by valarauko » Wed, Jan 27, 2021, 4:41 PM UTC

read_if_gay_ wrote: post #235

remram wrote: post #211

Earlier quoted context omitted.

All you're arguing is that it's "not the most sketchy" not that it's less sketchy than the average. And you're offering no proof or example of that either.

If you're going to get hung up on technical details, note that GP didn't ask whether it's less sketchy than average, just whether it's less sketchy. Finding concrete proof or examples is obviously hard in this subject matter (how are you going to prove something as abstract as sketchiness), but here's one observation: predatory conferences mostly only exist outside the West. To be even more concrete, two of the most…

>two of the most infamous predatory publishers (WASET and OMICS) are based in Turkey and India respectively

Well, as an Indian postdoc working in the US, I can speak to some of these sketchy behaviours. In terms of the predatory publishers, my Indian institution had its own filters, and most labs have their own as well. For example, for a while we had an institutional restriction on submitting manuscripts to conference proceedings, with the justification that the hard time limit equals substandard peer review. In addition, for the longest time we were not allowed to submit anything to open source journals, with similar justifications. Publishing in a journal with an IF Who does publish in these predatory journals? Smaller, less well funded universities with desperate students, ever since the government mandated first author publications as a requirement for receiving PhD degrees.

Re: I tried to report scientific misconduct. How did it go?

#289

Post by leovailati » Wed, Jan 27, 2021, 4:44 PM UTC

liminal wrote: post #276

How do we prevent fraud? Is there a way to change the incentives? Statistical analysis of results to detect bogus numbers? Only cite research that has been independently replicated and apply a "provisional" label to the first results? Then only cite the duplicating lab and not the first one, so the incentive is there for doing the grunt work?

I generally like the idea of rewarding replication studies. Replication/validation could be a required process that runs concurrently to peer review. The researchers who run the experiments to replicate results could rewarded by being added as contributors to the original paper, so they also get the citations. And like you suggest, the paper would then be marked as "validated" or something similar. I wonder if any journals out there are already doing something like this.

There is, of course, the danger of collusion among original authors and validators. Hopefully the fear of having your results rebuked would prevent people from trying to publish bullshit in the first place.

Another problem is logistics. Research labs have their own ideas they want to push forward, so spending time and resources proving or (even worse) disproving some else's idea doesn't sound that great. Also, even if it gives you citations, it probably wouldn't help you with your thesis.

Anyways, it's a tough problem to solve.

Re: I tried to report scientific misconduct. How did it go?

#290

Post by roel_v » Wed, Jan 27, 2021, 4:49 PM UTC

driverdan wrote: post #270

roel_v wrote: post #225

Earlier quoted context omitted.

"Peer review" is not "have someone else re-do the experiment". That's just not feasible, especially since reviews are done without pay. It's not realistic to expect people to spend more than a few hours reviewing a paper. That amount of time is barely enough to check for overall conceptual issues and maybe flag some really glaring deficiencies. (And then conclude with 'accepted with minor revisions', those 'revisions…

But there were glaring deficiencies in the stats. Anyone reviewing the papers should have caught them.

Well yeah in the papers of the OP maybe, I don't know. I more meant to address several commentors in this thread that seem to think in general that peer review is 'redo the research' and/or 'validate that it's correct'. It's not.

Nowadays when you see articles results of new research of covid19 in the media, those articles often include 'hasn't been peer reviewed yet' or 'reviewed by other scientists' or any such verbiage, either as a disclaimer or as 'now it must be true'. But that's not how it works; it's not because something has been 'peer reviewed' that it's 'The Truth' or 'Real Science'. Peer review, in reality, just weeds out (most) quacks (although in the OP's case it seems it didn't even do that) and checks that the paper is not completely out of touch with what is happening in and known about the field. It's not QA of the work itself.

(I don't care to debate if it should be, and if more money should be spend on replication etc, just providing some real world context on something that is quite opaque to and often misunderstood by those not in academia)