Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The culture of rejection in computer science publications

sigbed.org

Post reply on HN
Print view View original

121–130 of 172 posts

Re: The culture of rejection in computer science publications

#121

Post by blagie »

whatshisface wrote: post #117
BeetleB wrote: post #89

Earlier quoted context omitted.

> Comments often show the reviewers never read the paper. This. I was not in computer science, but in a different technical field, and this is sadly common. We would often have to appeal to the editor with "The topic the reviewer said we didn't address? It's in Section X. Get us another reviewer."

The biggest mystery in the whole thing is why someone who is volunteering to review papers anonymously would bother to do it badly when they could simply not do it at all.

No mystery. Behavior converges to incentives:

* You do get academic points for chairing a conference, and as a chair, you do need to find reviewers.

* A colleague is running a conference, and asks you to do a favor. You want to help your colleague. Reviewing papers wins you points with them, and declining to review burns bridges. When you're running a conference, you'd like them to reciprocate. Plus, they might be on a grant / hiring / etc. board / committee / etc. later on. Burning bridges in academia is very bad.

On the other hand, there is no incentive to invest more than 30-600 seconds per review. Neither you nor your friend really have any reason to care about the quality of the conference.

As this process repeats, people put in less and less time each time around, since it doesn't matter. The process converges to random noise.

Re: The culture of rejection in computer science publications

#122

Post by sideshowb »

BeetleB wrote: post #89
blagie wrote: post #62

Earlier quoted context omitted.

I'm well-known in a research community. I'm positioned such that I don't need more academic points. I've mostly stopped publishing in branded prestige academic venues, in part due to rejection rates. My goal in doing work and writing papers is to see them disseminated. The acceptance/rejection process is asinine -- studies show it's basically random. I've had one paper in my whole career where the reviewers did a pro…

> Comments often show the reviewers never read the paper. This. I was not in computer science, but in a different technical field, and this is sadly common. We would often have to appeal to the editor with "The topic the reviewer said we didn't address? It's in Section X. Get us another reviewer."

I've learned to address those ones diplomatically with "the topic you mention is now included in section X". Technically true, and lower friction.

Re: The culture of rejection in computer science publications

#123

Post by whatshisface »

blagie wrote: post #121
whatshisface wrote: post #117

Earlier quoted context omitted.

The biggest mystery in the whole thing is why someone who is volunteering to review papers anonymously would bother to do it badly when they could simply not do it at all.

No mystery. Behavior converges to incentives: * You do get academic points for chairing a conference, and as a chair, you do need to find reviewers. * A colleague is running a conference, and asks you to do a favor. You want to help your colleague. Reviewing papers wins you points with them, and declining to review burns bridges. When you're running a conference, you'd like them to reciprocate. Plus, they might be on…

>Reviewing papers wins you points with them, and declining to review burns bridges. When you're running a conference, you'd like them to reciprocate.

Surely they'd get upset if you rejected all of the good papers, thereby ensuring that they would have a bad conference.

Re: The culture of rejection in computer science publications

#124

Post by SilasX »

sideshowb wrote: post #122
BeetleB wrote: post #89

Earlier quoted context omitted.

> Comments often show the reviewers never read the paper. This. I was not in computer science, but in a different technical field, and this is sadly common. We would often have to appeal to the editor with "The topic the reviewer said we didn't address? It's in Section X. Get us another reviewer."

I've learned to address those ones diplomatically with "the topic you mention is now included in section X". Technically true, and lower friction.

Haha yes. Everyday example of this frustration (really happened):

"So when is their wedding?"

'Next week on Saturday.'

"Whoa whoa whoa, do you mean this coming Saturday, or the Saturday that happens next week?"

'Next week on Saturday.'

"Okay, gotcha, thanks, it was kinda unclear before."

Re: The culture of rejection in computer science publications

#125

Post by yCombLinks »

ackbar03 wrote: post #87
logisticseh wrote: post #67

Earlier quoted context omitted.

I've spent a lot of time with working for or closely interfacing with a half dozen academic institutions. I left academia by choice -- with multiple TT offers in hand -- so this isn't sour grapes. I am highly confident in my assessment that the personalities found on the typical R1 tenure track are exactly the sort of personalities I avoid hiring or working with at all costs. There are exceptions, but they prove the…

> I am highly confident in my assessment that the personalities found on the typical R1 tenure track are exactly the sort of personalities I avoid hiring or working with at all costs. And what is that exactly out of curiosity?

My experience working with a former academic that was awful to work with: Self-absorbed, self-promoting, accomplished next to nothing but talked a big game, shit on everything everyone else did, even though their code ran the business

Re: The culture of rejection in computer science publications

#126

Post by sgt101 »

chasil wrote: post #98
ghaff wrote: post #92

Earlier quoted context omitted.

Because there is an implied stamp of approval for publications. These days nothing is keeping people from publishing elsewhere if they want to.

If the organization is receiving public funds of any kind, then they should be required to (electronically) publish all submissions unless the authors withdraw. There has been quite enough censorship and paywalling of research that my taxes fund.

The bullshit will wash over everything though - there are lots of people out there that are not acting in good faith (ie. Russians) and they will use your suggestion to claim status for all sorts of wickedness.

Re: The culture of rejection in computer science publications

#127

Post by agumonkey »

blagie wrote: post #62
ath0 wrote: post #32

I'm long past my academia phase, but recently led the PC for an industry conference (accept rate: ~15%). 1. Curation is important both for the physical limits (venues only fit a certain number of people), attention limits (attendees will usually retain only a handful of "nuggets" no matter how packed the agenda is) and interaction limits (you can't meet everyone at a large conference). 2. If the goal of a conference…

I'm well-known in a research community. I'm positioned such that I don't need more academic points. I've mostly stopped publishing in branded prestige academic venues, in part due to rejection rates. My goal in doing work and writing papers is to see them disseminated. The acceptance/rejection process is asinine -- studies show it's basically random. I've had one paper in my whole career where the reviewers did a pro…

> My goal in doing work and writing papers is to see them disseminated. The acceptance/rejection process is asinine -- studies show it's basically random. I've had one paper in my whole career where the reviewers did a proper review (e.g. worked through the math). The rest were quick skims. Comments often show the reviewers never read the paper. The stuff that makes it through this process is often nonsense, while very high-quality work is often cut.

How come this is not fixed ?

Re: The culture of rejection in computer science publications

#128

Post by sgt101 »

A lot of this is caused by the bad faith of some state actors.

Many computer science conferences are under systematic assault from these places, which systematically swamp the PC's with submissions.

What needs to happen is :

- Regionalisation; make conferences regional only. So that submission can only come from that area or small group of nations. This will reduce travel demands and increase plurality.

- Sharp constraints on personal submission: one and only one paper as an author by anyone.

- Block outs : you get in one year, you skip a year.

Re: The culture of rejection in computer science publications

#129

Post by dr_hooo »

chasil wrote: post #98
ghaff wrote: post #92

Earlier quoted context omitted.

Because there is an implied stamp of approval for publications. These days nothing is keeping people from publishing elsewhere if they want to.

If the organization is receiving public funds of any kind, then they should be required to (electronically) publish all submissions unless the authors withdraw. There has been quite enough censorship and paywalling of research that my taxes fund.

You are likely not aware of just how much crap gets submitted for any given conference/journal.

Re: The culture of rejection in computer science publications

#130

Post by tokinonagare »

blagie wrote: post #62
ath0 wrote: post #32

I'm long past my academia phase, but recently led the PC for an industry conference (accept rate: ~15%). 1. Curation is important both for the physical limits (venues only fit a certain number of people), attention limits (attendees will usually retain only a handful of "nuggets" no matter how packed the agenda is) and interaction limits (you can't meet everyone at a large conference). 2. If the goal of a conference…

I'm well-known in a research community. I'm positioned such that I don't need more academic points. I've mostly stopped publishing in branded prestige academic venues, in part due to rejection rates. My goal in doing work and writing papers is to see them disseminated. The acceptance/rejection process is asinine -- studies show it's basically random. I've had one paper in my whole career where the reviewers did a pro…

> Comments often show the reviewers never read the paper.

And when they do, it's not sure they understood it or even put the slightest towards understanding. I've a rejected paper where one of the comment was that the header of a table featuring 4 columns named N, V, ADJ, ADV was "hard to understand". The table was between two paragraphs each mentioning nouns, verbs, adjectives and adverbs, in a paper mostly about dictionary...

Post reply on HN