Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The culture of rejection in computer science publications

sigbed.org

Post reply on HN
Print view View original

21–30 of 172 posts

Re: The culture of rejection in computer science publications

#21

Post by ghaff »

impendia wrote: post #14
RcouF1uZ4gsC wrote: post #6

> The double-blind review process has eliminated natural biases that may have influenced our reviews in the past, including tendencies to reject based on gender or to accept from the “better” institutions. I don’t know how much double-blind really helps. Especially in a specific field, you can usually tell what person/lab/company wrote the paper without seeing the authors’ names.

I once reviewed a math paper for a double-blind journal. Most math journals aren't double blind (as you say, I can often tell who the authors are) -- but this is one focused on expositional articles that undergrads might enjoy. Anyway, I read it, didn't find it all that interesting, and recommended that the paper be rejected. Afterwards, I googled the paper's title and found a signed copy: the author was one of the m…

As someone who has been on the conference committee for (non-academic) conferences, I'm not a particular fan of blind selections. Yes, I get the desire to not just give slots to "the usual suspects" which at least historically has often happened at a lot of conferences I attended. But the reality is that there are people who you know based on overwhelming experience will give great talks that attendees will appreciate and learn from. Even if they submitted an abstract that didn't immediately catch your eye among the pile you're going to accept 25% of, do you really want to reject them? Assuming the committee has some commitment to new/less known speakers there are IMO better ways to spread the love than blinding.

Re: The culture of rejection in computer science publications

#22

Post by ballenf »

> The emphasis on novelty has deep roots in academic publishing. It used to be that publishing was expensive, and any repetition came at the expense of other things that could have been published. Today, however, publishing is essentially free.

Requiring novelty shows respect for the readers' time. Paper and ink costs were never the primary limiting factors (even if the publishers' claimed otherwise to save face).

"repetition came at the expense of other things" -- no, repetition comes at the expense equal to Number of Readers * Time Wasted on Each non-novel paper.

If the author just stuck to "novelty is hard to really know" then it's a much stronger argument.

Re: The culture of rejection in computer science publications

#23

Post by mikkergp »

qzx_pierri wrote: post #19

It doesn't matter what color you are. It doesn't matter where you're from. If you create something like what Dijkstra created back in 56' (undeniable innovation), then no door will remain shut. That's the beauty of Math, CompSci, or STEM as a whole. Not to be too blunt, but I'd rather we keep this soft, "everyone should be accepted" nonsense out of CompSci. The cream will rise to the top. Anyone complaining just need…

> If you create something like what Dijkstra created back in 56' (undeniable innovation), then no door will remain shut. That's the beauty of Math, CompSci, or STEM as a whole.

Don’t get me wrong, this seems like a nice idea, I’d love to live in this world, but hasn’t it been disproven time and again throughout history? This board itself is flooded with engineers with great ideas being dismissed because of “realities of the business” or “changing priorities”, much less “political realities” etc et Al.

In fact I’d say the reality is probably opposite,(as a matter of principal, obviously great ideas do eventually make it but) truly great innovative ideas are ignored, it is only the incremental ideas that don’t make anyone uncomfortable that are accepted.

Re: The culture of rejection in computer science publications

#24

Post by karpierz »

qzx_pierri wrote: post #19

It doesn't matter what color you are. It doesn't matter where you're from. If you create something like what Dijkstra created back in 56' (undeniable innovation), then no door will remain shut. That's the beauty of Math, CompSci, or STEM as a whole. Not to be too blunt, but I'd rather we keep this soft, "everyone should be accepted" nonsense out of CompSci. The cream will rise to the top. Anyone complaining just need…

The cream generally rises, sure.

But at what rate? And how long will they stay in the game without recognition before giving up? And do those numbers differ based on your color or where you're from?

Re: The culture of rejection in computer science publications

#25

Post by logisticseh »

impendia wrote: post #12

I am an academic mathematician -- who has had job applications rejected, papers rejected, grant proposals rejected. Not always, but it's not exactly a rare occurrence. I've also been on the other side, and it also sucks to reject people. It's an unfortunate reality of academia that there are fewer resources (jobs, grant funding, etc.) available, than there are researchers who are prepared to put them to good use. Fur…

I don't buy the attention filter argument. No one -- and I really do mean no one -- is going to read the entire contents of the proceedings of even just one of these conferences. NeurIPS -- a single CS conference -- is more than twice the size of the Joint Math Meetings. ICRA and ICML are just as large or larger, and AAAI isn't far behind. That's just one sub-field of CS. There are so many papers coming out every year that I simply cannot keep up with two of my own niches. Adding more papers to that firehose wouldn't materially change the situation.

I've reviewed for some (high quality) Mathematics journals. Papers tend to be more complete, for sure, but the reviewing is much less rejectionist. I'm not aware of any Mathematics journal with a 10% acceptance rate, and even 20% is probably on the low end.

> It's an unfortunate reality of academia that there are fewer resources (jobs, grant funding, etc.) available, than there are researchers who are prepared to put them to good use.

I don't think this is true in CS. Universities outside of an elite set really struggle to hire and retain high quality faculty. It's at a crisis level outside of R1. Teaching-oriented institutions have mostly have stopped trying to hire traditional academics; a masters degree with some teaching experience is sufficient.

Some of this is due to industry -- high-quality faculty candidates tend to also have 3x-5x offers in industry, and it's hard to turn down a guaranteed early retirement for the grind and uncertainty of the tenure track. But I think some of it is also that students who would make good teachers and mentors lose confidence due to a series of unnecessary paper rejections and decide to nope out of academia.

Again, I spend a lot of time around academic mathematics. The rejectionist culture in CS is real. And not just conferences, btw. An NSF program manager started my last review panel by telling us that scores are consistently way lower in CS than in any other field and to please chill out.

Re: The culture of rejection in computer science publications

#26

Post by thomaslangston »

logisticseh wrote: post #25
impendia wrote: post #12

I am an academic mathematician -- who has had job applications rejected, papers rejected, grant proposals rejected. Not always, but it's not exactly a rare occurrence. I've also been on the other side, and it also sucks to reject people. It's an unfortunate reality of academia that there are fewer resources (jobs, grant funding, etc.) available, than there are researchers who are prepared to put them to good use. Fur…

I don't buy the attention filter argument. No one -- and I really do mean no one -- is going to read the entire contents of the proceedings of even just one of these conferences. NeurIPS -- a single CS conference -- is more than twice the size of the Joint Math Meetings. ICRA and ICML are just as large or larger, and AAAI isn't far behind. That's just one sub-field of CS. There are so many papers coming out every yea…

What does R1 mean in this context?

Re: The culture of rejection in computer science publications

#27

Post by hiptobecubic »

ballenf wrote: post #22

> The emphasis on novelty has deep roots in academic publishing. It used to be that publishing was expensive, and any repetition came at the expense of other things that could have been published. Today, however, publishing is essentially free. Requiring novelty shows respect for the readers' time. Paper and ink costs were never the primary limiting factors (even if the publishers' claimed otherwise to save face). "r…

But without reproducibility the paper is basically worthless. There's already a crisis of unreproducible work and the strong novelty bias is basically why.

Re: The culture of rejection in computer science publications

#28

Post by nautilius »

MontyCarloHall wrote: post #17

Add upvotes/downvotes to preprint servers. Weight voting power by the user’s subfield, dynamically determined by who’s upvoting them, PageRank style (so a scientist who’s mostly upvoted by others working in numerical linear algebra would have a lot of voting power on a preprint describing a novel matrix factorization scheme, and very little voting power on a preprint describing a novel thread scheduling algorithm). T…

Haha, yeah no way anyone’s going to game that system! No way some clique recruits all of the voting power (collaborators to undergrads) to push their work through, all it takes is LeCun to simply read all papers and his vote will balance everything out!

Re: The culture of rejection in computer science publications

#29

Post by PuppyTailWags »

ballenf wrote: post #22

> The emphasis on novelty has deep roots in academic publishing. It used to be that publishing was expensive, and any repetition came at the expense of other things that could have been published. Today, however, publishing is essentially free. Requiring novelty shows respect for the readers' time. Paper and ink costs were never the primary limiting factors (even if the publishers' claimed otherwise to save face). "r…

I don't think requiring novelty is respect for a reader's time. I think requiring high quality is respect, even if it is adding to existing bodies of evidence. Novel drivel is just drivel regardless of its novelty; it requires high quality analysis and research for anyone to get any benefit from that newness.

Re: The culture of rejection in computer science publications

#30

Post by logisticseh »

thomaslangston wrote: post #26
logisticseh wrote: post #25

Earlier quoted context omitted.

I don't buy the attention filter argument. No one -- and I really do mean no one -- is going to read the entire contents of the proceedings of even just one of these conferences. NeurIPS -- a single CS conference -- is more than twice the size of the Joint Math Meetings. ICRA and ICML are just as large or larger, and AAAI isn't far behind. That's just one sub-field of CS. There are so many papers coming out every yea…

What does R1 mean in this context?

Universities that offer doctoral degrees and have "Very High Research Activity" according to the Carnegie Classification of Institutions of Higher Education.

Specifically, the 130 or so institutions listed here: https://en.wikipedia.org/wiki/List_of_research_universities_...

Heuristically, think "major private universities and flagship public universities".

Post reply on HN