Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Queueing to publish in AI and CS

damaru2.github.io

Post reply on HN
Print view View original

11–20 of 59 posts

Re: Queueing to publish in AI and CS

#11

Post by davidmartirubio »

William_BB wrote: post #8

There's two main problems. First, the field is too saturated -- too many people publishing papers. Second, the reviewers are the same people that publish papers -- it's a zero sum game (regardless of acceptance criteria). Submission numbers this year have been absolutely crazy. I honestly don't think it can be solved.

And yet, submission numbers follow the (N = actually new papers)/(p = rate of acceptance) trend, approximately. The difference between a 20% and 35% acceptance rate is ~5N vs ~3N papers in the submission pool.

Re: Queueing to publish in AI and CS

#12

Post by philipwhiuk »

If there's more good papers due to more activity, doesn't that allow for the possibility of more conferences and journals to publish to, which increases the acceptance rate?

I don't think that the solution has to be that existing conferences and journals accept more.

Re: Queueing to publish in AI and CS

#14

Post by William_BB »

lou1306 wrote: post #7

I agree that a) rejecting a paper that has been recommended for acceptance by _all_ reviewers (something that routinely happens in, say, NeurIPS) is nonsense. However, in-person conferences have physical limits. In the case of, again, NeurIPS, you may get accepted and _not_ present the paper to an audience. This is also a bit of a travesty. The community would be better off working with established journals so that t…

NeurIPS scoring system is inherently subjective. People will have wildly different interpretations of, say, 3 vs 4, or 4 vs 5. You can get lucky and draw only reviewers that, on average, "overrate" papers in their batch. The opposite can happen too, obviously. 4444 vs 3444 is just noise.

Re: Queueing to publish in AI and CS

#15

Post by matusp »

philipwhiuk wrote: post #12

If there's more good papers due to more activity, doesn't that allow for the possibility of more conferences and journals to publish to, which increases the acceptance rate? I don't think that the solution has to be that existing conferences and journals accept more.

The problem is that external environment (namely hiring) creates pressure to publish in a handful of prestigious venues. Nobody wants to be the odd one out that has their papers in some less well known conferences.

Re: Queueing to publish in AI and CS

#16

Post by accurrent »

My general experience as a PhD student makes me feel that we probably should look at the system as a whole. I get the feeling most universities are abusing publications as a way of assessing their own students. It means people will tend to publish slop for the sake of publication instead of genuinely tackling hard problems. I personally have seen my university get away with forming defence committees with 0 knowledge of the field they're assessing. If the university can't assess its students work then they should not be teaching.

I think both conferences and journals are broken in this regard. It doesn't help that professors primary jobs these days is to be a social media influencer and attract funding. How the funding is used doesn't seem to matter or impact their careers. What we need is more accountability from senior researchers. They should be at the very least assessing their own students work before stamping their name on the work.

On the flip side it isn't untrue that there are major breakthroughs happening daily at this point in many fields. We just don't have the bandwidth to handle all the information overload.

Re: Queueing to publish in AI and CS

#18

Post by macleginn »

As pointed out in the comments, conferences have physical limits on the number of presenters (even in poster sessions), so the number of accepted papers at top conferences will likely stop growing at some point. (Perhaps it should have been capped already.) This will probably lead to new conferences appearing to satisfy the "demand", but more conferences of varying quality is probably better than reviewing at top conferences being random at best.

Re: Queueing to publish in AI and CS

#19

Post by icoder »

accurrent wrote: post #16

My general experience as a PhD student makes me feel that we probably should look at the system as a whole. I get the feeling most universities are abusing publications as a way of assessing their own students. It means people will tend to publish slop for the sake of publication instead of genuinely tackling hard problems. I personally have seen my university get away with forming defence committees with 0 knowledge…

I used to be in (molecular biology) research. At some point my supervisors were already working towards a paper in their mind, while I was still doubting (the statistical significance of) my findings.

In the end I left my Phd track before actually finishing it. My conclusion is that I like research(ing stuff) as a verb, but I don't like research as an institute.

Re: Queueing to publish in AI and CS

#20

Post by maweki »

capestart wrote: post #17

The real issue is careers valuing where papers are published over what they contribute, and incentives won’t change until hiring and funding reward depth over volume.

> valuing where papers are published over what they contribute

And who is the arbiter of that? This is an imperfect but easy shorthand. Like valuing grades and degrees instead of what people actually took away from school.

In an ideal world we would see all this intangible worth in people's contributions. But we don't have time for that.

So the PhD committee decides on exactly that measure whether there are enough published articles for a cumulative dissertation and if that's enough. What's exactly the alternative? Calling in fresh reviews to weigh the contributions?

Post reply on HN