Live data from Hacker News

New academic journal only publishes 'unsurprising' research rejected by others

cbc.ca

111–120 of 207 posts

Re: New academic journal only publishes 'unsurprising' research rejected by others

#111
post #32

Why don’t research groups just publish to their own websites or directories like arxiv? What’s the role of an academic journal in 2020? Honest question, I’d love to see more blogging from hard science academics but I’m wondering if there’s a reason why that’s challenging or if it’s just academic culture. We should have a Substack/OnlyFans for scientists.

> What’s the role of an academic journal in 2020?

Status. Brand. Trust. That's all it's really about. If Newton and Einstein started a curated website of the best papers of the year, those papers would probably experience increased citations, and you would probably pay a premium to subscribe to that list.

Re: New academic journal only publishes 'unsurprising' research rejected by others

#113
post #33

I'd also love to see a journal that agreed to publish work before any results are known. Researchers would submit hypotheses and methodology for review, and the journal would publish the results after the experiments were conducted, regardless of their outcome. It would still incentivize interesting hypotheses, but wouldn't lead to results-biased publications.

There is a workshop[1] at NeurIPS this year (2020) for experimenting with this model. I hope it is adopted more widely, especially in the ML community to disincentivize the +1-2% performance increase in SOTA papers.

[1] https://nips.cc/Conferences/2020/Schedule?showEvent=16158

Re: New academic journal only publishes 'unsurprising' research rejected by others

#114
The root problem is the reward system in academia. Even today, when the relevant players are discussing how to improve it, the question is always phrased as needing to find better ways to reward excellence.

If you want to reward excellence, you're by definition looking for the exceptional, the surprising. It might sound good, and surprising discoveries are necessary, but they're not the only thing that's good for science.

We need ways to recognise more than just exciting results; researchers should also be rewarded for robust contributions to the body of human knowledge.

(This is also why Plaudit.pub, a project I volunteer for, allows researchers to explicitly recognise robust or exciting work as separate qualities.)

Re: New academic journal only publishes 'unsurprising' research rejected by others

#115

Earlier quoted context omitted.

There are sociology studies, published in closed journals, that detail the positive impacts that public libraries have on communities. The PhDs of that field must either be (1) blind to the irony, (2) totally resigned to this state of affairs, or (3) themselves complicit as managers of closed journals. One more indictment: I'll bet they're all extremely depressed about how scientifically illiterate the general popula…

Well said. I don’t understand why public institutions at the very least aren’t required to share their research. And considering that the majority of higher ed public or private wouldn’t last a day without federal backed loans, I’m not sure only public institutions should be held to this fire. I take actual pleasure in pirating research papers.

Any research funded by the NIH is required to be published open access at PubMed Central.

Re: New academic journal only publishes 'unsurprising' research rejected by others

#116

We also need a journal to publish methods that failed. I did so much work during my PhD that was dead end and is not documented.

https://www.journalnetwork.org/journals/international-journa...

"Start Year: 2014"

"This is a new journal. No publications have been accepted yet."

Looks dead to me.

Re: New academic journal only publishes 'unsurprising' research rejected by others

#117
post #29

Half of my PhD thesis is considered "unpublishable" because, after doing the work, my supervisors felt it's actually "unsurprising" that it didn't work out. We took methods that had been exploited to improve on previous results for over a decade to their logical extreme, and found that this method no longer leads to improvements. After doing the work it seems obvious. A paper on the subject would almost be considered…

My experience: I proposed a thesis to an advisor who deemed it unlikely to work. He ran it by a colleague who came to the same conclusion. I went to a different major and pursued the same thesis and invited the previous faculty who initially turned it down to the defense because it was relevant to their field. It was frustrating to hear them voice their opinions in the defense that they felt, “of course it world work…

https://en.wikipedia.org/wiki/Hindsight_bias

Re: New academic journal only publishes 'unsurprising' research rejected by others

#118
post #2

Good, science needs more of this. A negative result is a result, and if you don't publish it, then you end up skewing the distribution of results. Which can lead to people (and meta studies) drawing incorrect conclusions.

Ben Goldacre talks about this in Bad Science. Well worth a read.

Re: New academic journal only publishes 'unsurprising' research rejected by others

#119
post #96

Earlier quoted context omitted.

This is normal practice. To publish in a respectable journal you are charged £1000+. To publish your paper as open access, you can be charged another ~£1000 for the privledge (IEEE).

Never encountered paying for non-open access publication. I have only encountered "article processing fees" in relation to open access publishing. If you're okay with having your paper paywalled, you do not need to pay for journal publication. So I'm curious in which domain/subdomain you have encountered this.

In my area of semiconductor engineering/ detectors, we generally have to pay to publish. Journals do seem to be slowly moving away from this for just the publication, however as you say for open access you still need to pay. Depending on the funding provider for the research, it can be compulsory for the papers to be open access, so we would end up paying anyway.

Re: New academic journal only publishes 'unsurprising' research rejected by others

#120

Half of my PhD thesis is considered "unpublishable" because, after doing the work, my supervisors felt it's actually "unsurprising" that it didn't work out. We took methods that had been exploited to improve on previous results for over a decade to their logical extreme, and found that this method no longer leads to improvements. After doing the work it seems obvious. A paper on the subject would almost be considered…

That sucks to hear. Null results are important, as you say, if only to dissuade others from doing the same. See also the "file-drawer problem" ( https://en.wikipedia.org/wiki/Publication_bias ). Also, with regards to the incentives in the field and the lack of null results, there's always Ioannidis's classic work ( https://journals.plos.org/plosmedicine/article?id=10.1371/jo... ).

I disagree. Negative results are important. Null results are of very limited interest. The two are worlds apart.

A null result simply means you tried something and it didn't work. But you don't know why. You haven't proven it didn't work. There are literally millions of reasons why something might not work. For instance, you could try to use compound X to cure disease Y, observe no effect, and conclude that X doesn't cure Y. But what if somewhere in the process of making X you made an uncaught mistake and you instead used X'?

A negative result means that you tried something and you came to the proven conclusion it doesn't work. This is, crucially, as hard to obtain as a positive result. In my example, it would imply a much longer process than simply "apply X, see no effect in Y, make a few robustness checks, done".

You could say "Well, publish the null anyway, somebody will catch the mistake". Unlikely. There are already so many papers out there that keeping up is impossible. If we were publishing also null results this number will grow tenfold at the very least. Nobody could possibly check everything. They will see a paper "X doesn't cure Y" and call it knowledge, stifling a possible cure virtually forever.

Am I splitting hairs? Perhaps. But I think HN prizes itself to be a scientifically minded community, and thus it has a mandate to use terms correctly. Confusing "null" with "negative" is a sin.

I hope one day I'll find a way to strongly and passionately argue against the "null results are as important as positive results" position. It is a bad meme. Charitably, I consider it most of the times a honest mistake. But sometimes it gives me the impression it is a cheap trick used by people to erode the reputation of academia.

Post reply on HN