Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Time to break academic publishing's stranglehold on research

newscientist.com

Post reply on HN
Print view View original

31–40 of 312 posts

Re: Time to break academic publishing's stranglehold on research

#31

Post by avs733 »

Angostura wrote: post #21
avs733 wrote: post #19

Add in that the quality of the system is massively massively broken...peer review is about as accurate as the flip of a coin. It does not promote gorund breaking or novel research, it barely (arguably doesn't) even contribute to quality research. I had a colleague recently be told by a journal editor 'we don't publish critiques from junior scholars.' So much for the nature of peer review being entirely driven by the…

What would be a good alternative to peer-review though? Genuinely interested.

So the short answer is I don't know, and I don't think there is a single answer. Different types of scholarship need different standards and processes in my eyes. It isn't so much 'whats the alternative' to what is really a singular step as much as it is a rethink of how scholarship is performed, evaluated, rewarded, and considered. Otherwise the incentives across the playing field don't change enough to make meaningful change meaningful.

What would help? My opinions...

-FOSS publishing -Abandonment of journal metrics, they really serve no purpose besides trophy hunting at this point

-Stronger transparency initiatives

-Public review (c.f. Lim)

-better journal metadata (it affects citations)

-Greater shaming of misbehavior of ALL types

-Elimination of stupid policies like 'issue lengths' The article either should be or should not be published...saying there isn't space in a given issue is insane given that >>90% of article access is online.

-a hell of a lot of older faculty retiring out of the way of science.

Re: Time to break academic publishing's stranglehold on research

#32

Post by apathy »

justinclift wrote: post #25
impendia wrote: post #18

Earlier quoted context omitted.

I'm a math professor, I can answer this. We tend to think of journals as being run by their editorial boards -- i.e., by scientists in our field. The publishers are in the background, not taking a very active role, simply skimming off truckloads of money in the process. In particular, peer-review requests tend to come from scientists whom we know at least by reputation, and often personally. When I spend serious time…

> If I get asked to peer-review a paper that looks uninteresting to me, or something outside my interests and expertise, ... or write a quick report recommending that the paper not be accepted . That seems kind of harsh?

Not really. If work is flawed, it will need to be fixed before recommending it to readers (which is now the only marginal value provided by editors). If it appears to be unfixable, the correct choice may be to recommend that it not be endorsed by a journal.

You can always put work up on arXiv if the goal is priority or distribution. Peer review is, in principle, designed to ensure that work is new and true (and that what is new is true). It’s not meant to fix flawed work, but to make good work better.

Re: Time to break academic publishing's stranglehold on research

#33

Post by avs733 »

DrNuke wrote: post #27
Angostura wrote: post #21

Earlier quoted context omitted.

What would be a good alternative to peer-review though? Genuinely interested.

For many STEM fields, conference talks, presentations and written contributions are notably faster and better suited to non-tenured applied researchers than the journals route.

unless the people who review their tenure cases don't count them because they are woefully out of date and out of touch.

It has been amazing to come to understand just how effective the corporate takeover of universities has been in the thinking of senior faculty. I have had faculty complain about policies that they literally are the only one's empowered to change because it has never occurred to them that they actually have power.

Conferences, generally, are a better path to science because of dialogue. Having a conference with 2500 presentations because everyone needs another CV line item don't accomplish that because no one is in the room.

Re: Time to break academic publishing's stranglehold on research

#34

Post by apathy »

williamstein wrote: post #17
ylmm wrote: post #14

Earlier quoted context omitted.

I've only worked with professors in economics and political science but I'm under the impression that this is true more generally of academia. Some people do it to help advance the state of the field. Career incentives and standing in the field also come into play. As the poster above mentioned, a professor would be able to say more. To respond succinctly to your comments: money isn't the only incentive in life, so i…

I'm a professor (in mathematics), and there are some reasons why I personally might say "yes" to review a paper. (1) If the paper is so interesting that when I see it I think "I really, really want to read this paper!", then I am likely to peer review it because as a reviewer when I read it, I get to ask any question and the author will take my questions and concerns very seriously. (2) I owe the editor who is asking…

Well, yeah, but without students & postdocs it is very difficult to make a material impact on your field of endeavor.

There’s also an ulterior motive for some lines of work, where sending a paper to be critically reviewed by rivals is a mechanism to torture test the work & conclusions.

This can be taken too far (especially in biology, the glam fluffing and $million additional irrelevant experiments are legendary), but in principle, if even your most motivated critics can’t find a fatal flaw in your work, it’s a reasonable bet that the work is sound. At least, that’s the principle. Editorial overrides sometimes break this safeguard, though (lord knows I’ve seen a few).

Re: Time to break academic publishing's stranglehold on research

#35

Post by avs733 »

lumost wrote: post #23
Angostura wrote: post #21

Earlier quoted context omitted.

What would be a good alternative to peer-review though? Genuinely interested.

It strikes me that the challenge of peer review is that it is non-transparent, and gate-keeping. It is impossible for one to read the peer review feedback related to a published paper, and it's similarly difficult to publish controversial pieces that break established norms. Contrast this to the peer review culture of popular open source projects - major pull requests have extensive and transparent dialogue, and disa…

gatekeeping in science is a theoretical good...the difficulty in practice is highlighted by the general 'three paths' for citations...articles that never get cited, articles that get cited a lot at first and then are forgotten, and articles that aren't cited until much later and then cited en masse. That last category is really the 'good science' because it does what we want foundational research to do - disrupt current and limiting ways of thinking.

Dialogue is an unmitigated good, but dialogue also tends to be dominated by the normative voice. Where science is different than OSPs is that OSPs need to work with what exists now, whereas science needs to engage in tension and informed dialogue in a more foudnational way..

Re: Time to break academic publishing's stranglehold on research

#36

Post by RMarcus »

It's worth noting that the computer science community (at least in systems and machine learning) have already made some strides in this direction. Systems conferences like VLDB and CIDR post the proceedings online for free (and most ICDE and SIGMOD articles are available through the author's website). In machine learning, almost every conference paper is also available via arXiv.

When a paper is seemingly not available online, I've always gotten a free copy via an email to the author... And then there's sci-hub. It's not the way it should be (i.e. you shouldn't have to hunt around for publicly-funded research), but at least it's something.

Re: Time to break academic publishing's stranglehold on research

#37

Post by avip »

avs733 wrote: post #31
Angostura wrote: post #21

Earlier quoted context omitted.

What would be a good alternative to peer-review though? Genuinely interested.

So the short answer is I don't know, and I don't think there is a single answer. Different types of scholarship need different standards and processes in my eyes. It isn't so much 'whats the alternative' to what is really a singular step as much as it is a rethink of how scholarship is performed, evaluated, rewarded, and considered. Otherwise the incentives across the playing field don't change enough to make meaning…

The Sphenodon in the room is that the raison d'etre of the entire Academic structure is unclear. So that should be clarified first. Advancing science is one thing. Publishing as many papers as possible, breaking research into MPUs is another. Lecturing to youth completely uninterested to be taught is yet another vertical.

Re: Time to break academic publishing's stranglehold on research

#38

Post by apathy »

ylmm wrote: post #15
Y7ZCQtNo39 wrote: post #13

Earlier quoted context omitted.

I can't remember the last time I've read a headline about the peer-reviewer that confirmed some groundbreaking finding. Maybe it gives you the opportunity to network with higher-profile contributors to your field, if you offer to peer review their work (though to be honest, I don't know how the peer review process works, and if you can even "offer" to perform it for a specific paper).

At least in the social sciences, journal editors delegate papers to reviewers. Your first sentence also suggests that you don't know what peer review is. Peer review is not the same thing as replication, which is replicating the results of a paper after it's been published (e.g., confirming some groundbreaking finding). Peer review happens at the stage before publication of the original paper. Researcher(s) submit th…

Well, yeah, but learning to review is less difficult than learning how to communicate clearly, crisply, and forcefully. So in a well administered review, most benefits accrue to the authors even if the reviews are poorly done, so long as they are ethical. (Editors can step in when reviewers are being unreasonable or ignoring explicit instructions; unfortunately professional editors at glam journals can also step in to push exciting or politically expedient work into press before there is time to adequately vet it, which sucks and gives everyone a bad name).

Re: Time to break academic publishing's stranglehold on research

#39

Post by impendia »

justinclift wrote: post #25
impendia wrote: post #18

Earlier quoted context omitted.

I'm a math professor, I can answer this. We tend to think of journals as being run by their editorial boards -- i.e., by scientists in our field. The publishers are in the background, not taking a very active role, simply skimming off truckloads of money in the process. In particular, peer-review requests tend to come from scientists whom we know at least by reputation, and often personally. When I spend serious time…

> If I get asked to peer-review a paper that looks uninteresting to me, or something outside my interests and expertise, ... or write a quick report recommending that the paper not be accepted . That seems kind of harsh?

Often (but definitely not always), requests to peer-review are accompanied by statements like "We consider our Journal X to be on par with Journals Y and Z, and aim to be extremely selective. Please only recommend publication if blah-blah-blah...."

I'll then read the introduction. If I don't have sufficient expertise to referee the paper, I'll decline the request. If I do, then I'll see what the authors have accomplished. If, in my judgment, this doesn't rise to the level that the editors asked for, then it doesn't take me a long time to decide this and say so.

Conversely, if the introduction does impress me, then I will want to check the proofs in very close detail. In this case I will commit to writing a detailed report in the future.

It's quite common for papers to be declined from individual journals; it's happened to me plenty. There are tons of other journals out there; you can submit somewhere else. And when I decide that a negative report is called for, I write it right away, so as to not keep the authors waiting forever.

Re: Time to break academic publishing's stranglehold on research

#40

Post by impendia »

oaeide wrote: post #30
justinclift wrote: post #25

Earlier quoted context omitted.

> If I get asked to peer-review a paper that looks uninteresting to me, or something outside my interests and expertise, ... or write a quick report recommending that the paper not be accepted . That seems kind of harsh?

I reacted when I read this as well. I am hoping he meant that he would recommend that paper to be published/not published... ...I fear however that my hopes are in vain

by "not accepted" I mean "not published in the particular journal I was asked to review for".
Post reply on HN