Live data from Hacker News

The culture of rejection in computer science publications

sigbed.org

161–170 of 172 posts

Re: The culture of rejection in computer science publications

#161

Earlier quoted context omitted.

I left my PhD in computational geometry for similar reasons. I was told flat out by my advisor that the chances of me getting a paper accepted in even a middling journal were low. Not normally low (as in tough review), but so low because the entire field is dominated by a handful of very high profile researchers and mathematicians that essentially gate keep knowledge on it. It was also necessary to publish some paper…

One slight saving grace (after having graduated) is to feel fine writing documents like papers and just putting them up on https://arxiv.org/ under my own terms. One additional technique I learned from others is to use submissions to conferences as a way to get feedback on your work without any expectation that it gets accepted. You do that early knowing there are flaws, but do it to get early warnings from reviewers…

>> One additional technique I learned from others is to use submissions to conferences as a way to get feedback on your work without any expectation that it gets accepted. You do that early knowing there are flaws, but do it to get early warnings from reviewers about rejection points.

This is, of course, one if the problems with current research as it wastes everyone's (yours, reviewers, ACs, etc) time. In particular, because you're not guaranteed practicable or actionable feedback and because you're increasing the time reviewers must review work (i.e., more papers to review as submissions are higher and quality lower).

Re: The culture of rejection in computer science publications

#162
post #142
post #16

Earlier quoted context omitted.

Could you give a bit more context about this? I'm not familiar with the subject.

I think the performative, results-based culture of academia in general (particularly in Asia and not just Chinese) has seriously succumbed to Goodhart's law: https://en.wikipedia.org/wiki/Goodhart%27s_law Because you are judged on citations, publication count in prestigious journals, and research dollars spent... you have people citing themselves and their friends in citation rings, gaming the referee process to pick…

A starting point for the above exploration: https://elmorigin.webs.com/

Re: The culture of rejection in computer science publications

#163

Earlier quoted context omitted.

> My goal in doing work and writing papers is to see them disseminated. The acceptance/rejection process is asinine -- studies show it's basically random. I've had one paper in my whole career where the reviewers did a proper review (e.g. worked through the math). The rest were quick skims. Comments often show the reviewers never read the paper. The stuff that makes it through this process is often nonsense, while ve…

Because the leaders are the people who made their careers in the current system and they wouldn't benefit from making things more meritocratic. These are the people who argues endlessly saying meritocracy is bad for reason X or reason Y, they just want to keep their current privileges.

I'm still a bit surprised how basic human defects can reach even the 'smartest' spheres of society.

Re: The culture of rejection in computer science publications

#164
post #82

The academic system is completely broken for Computer Science, and I don't really see a way to fix it. The economic realities of the field just make it too risky to allow an exceptionally gifted individual to remain out in the open publishing research that could potentially destroy your business model. Justifying these low acceptance rates as somehow prestigious is really just creating even more perverse incentives t…

I have never pursued this type of publication, but why on earth does an "acceptance rate" even exist for these (journal or conference) publications? Why not publish them all? Endorse those that are selected, add commentary to those with which there is disagreement, but is a batch inclusion of them all so technically difficult?

The purpose of a journal is to be read, the purpose of a conference is to be heard, and they should be optimized for the desires of the audience, not the desires of the authors.

A journal or conference that publishes everything is useless to its audience, which expects significant curation and filtering to pare down the incoming firehose to something that (a) fits a specific topic and (b) has a filter for quality and (c) pick the most interesting ones out of all the far-too-many decent papers that fit the first two criteria.

There's no problem to publish them all somewhere, that's being done on e.g. arXiv, pretty much every recent useful CS paper is there as well, often before being published in a "proper venue" - but a "publish everything" service is not enough, people do want to read more selective sources instead of trying to skim everything that's posted on arXiv.

Re: The culture of rejection in computer science publications

#165
post #86

One side effect I didn't see mentioned in the article: One professor of mine spoke of the LPU, the least publishable unit. So, if you're lucky enough to have some novel ideas, and build something nice out of it, don't put it all into one coherent and easily digested journal paper! The number of publications counts. Instead, chop it into little pieces that are just "novel" or noteworthy enough (LPUs), and publish them…

Honestly, I think this is often said cynically but is a good practice overall. Would you rather have to read and understand one giant commit reflecting 2 years of work, or 10 well-documented and logically complete individual commits?

The analogy doesn't hold up.

Commits have a more fixed size.

Formal logic cannot be reduced in character count too far, at least without making it unintelligible (fear the day Alex the intern wheels out Unicode emoji variables).

Spreading the commits out has a purpose.

Academic papers could be vastly compressed to reduce cognitive load.

Brevity and intelligibility however is not a KPI. Self aggrandisement is.

Re: The culture of rejection in computer science publications

#167

I am an academic mathematician -- who has had job applications rejected, papers rejected, grant proposals rejected. Not always, but it's not exactly a rare occurrence. I've also been on the other side, and it also sucks to reject people. It's an unfortunate reality of academia that there are fewer resources (jobs, grant funding, etc.) available, than there are researchers who are prepared to put them to good use. Fur…

I don't buy the attention filter argument. No one -- and I really do mean no one -- is going to read the entire contents of the proceedings of even just one of these conferences. NeurIPS -- a single CS conference -- is more than twice the size of the Joint Math Meetings. ICRA and ICML are just as large or larger, and AAAI isn't far behind. That's just one sub-field of CS. There are so many papers coming out every yea…

> I don't think this is true in CS. Universities outside of an elite set really struggle to hire and retain high quality faculty. [...] Some of this is due to industry -- high-quality faculty candidates tend to also have 3x-5x offers in industry, and it's hard to turn down a guaranteed early retirement for the grind and uncertainty of the tenure track.

That seems to just confirm that resources are lacking. There's no shortage of people, but there's a shortage of money to pay them.

Re: The culture of rejection in computer science publications

#168

Earlier quoted context omitted.

One slight saving grace (after having graduated) is to feel fine writing documents like papers and just putting them up on https://arxiv.org/ under my own terms. One additional technique I learned from others is to use submissions to conferences as a way to get feedback on your work without any expectation that it gets accepted. You do that early knowing there are flaws, but do it to get early warnings from reviewers…

>> One additional technique I learned from others is to use submissions to conferences as a way to get feedback on your work without any expectation that it gets accepted. You do that early knowing there are flaws, but do it to get early warnings from reviewers about rejection points. This is, of course, one if the problems with current research as it wastes everyone's (yours, reviewers, ACs, etc) time. In particular…

I won’t deny that.

Re: The culture of rejection in computer science publications

#169
post #157
post #145

Earlier quoted context omitted.

studies show it's basically random The "basically" is important though, because there are some nuances to it. However, the point I've actually come here to make is that since publications are a strong factor for your career progress in academia, a corollary of the above is that making it in academia is basically random, too. Which is also true for other reasons, though: for every open professor position in a certain…

My experience is that for elite schools -- Stanford and MIT -- the remaining factor is how much one is willing to cheat. There is a random component, a merit-based component, but most (and I have large n here) successful affiliated faculty candidates did so by cheating in some way. That can be data baking, credit theft, or a whole slew of other techniques, but at least in my department, most new faculty at least at t…

And one more: https://medium.com/@tnvijayk/potential-organized-fraud-in-ac...

Re: The culture of rejection in computer science publications

#170
post #169
post #157

Earlier quoted context omitted.

My experience is that for elite schools -- Stanford and MIT -- the remaining factor is how much one is willing to cheat. There is a random component, a merit-based component, but most (and I have large n here) successful affiliated faculty candidates did so by cheating in some way. That can be data baking, credit theft, or a whole slew of other techniques, but at least in my department, most new faculty at least at t…

And one more: https://medium.com/@tnvijayk/potential-organized-fraud-in-ac...

From the article:

"99.99% of us are honest but the dishonest 0.01% can cause serious, repeated damage."

My experience is that this is much more like a 50/50 split at elite schools, at least when you look at people who succeed at making it to faculty positions. BMJ estimates 20% of publications are based on fabricated data:

https://blogs.bmj.com/bmj/2021/07/05/time-to-assume-that-hea...

That sounds about right for what I've seen at MIT. Note that 20% of publications being based on fabricated data is in-line with my 50/50 split figure. Researchers who cheat only do it part of the time, and often in ways which don't involve direct data fabrication. Critically, the numbers go up significantly for high-impact publications -- they types that make the news and make scientific careers. By the time MIT's PR machine picks up a publication, and the press picks up from there, the odds of it being fraudulent are much higher than 50/50.

Post reply on HN