Live data from Hacker News

The culture of rejection in computer science publications

sigbed.org

61–70 of 172 posts

Re: The culture of rejection in computer science publications

#61
post #57

Academia as a whole is such an interesting place. I value it highly because I value learning. I love school, I loved the conversations and deep dives into interesting waters that nourished me in both college and graduate school, and I loved the opportunity to be fully immersed, without needing to get a job, in an atmosphere of learning. But the rest of it? The petty hierarchies, the papers born of countless hours of…

If the last paragraph is your perspective on being a pawn in an academic structure, I would love to hear what you think of being a pawn in a corporation.

Re: The culture of rejection in computer science publications

#62
post #32

I'm long past my academia phase, but recently led the PC for an industry conference (accept rate: ~15%). 1. Curation is important both for the physical limits (venues only fit a certain number of people), attention limits (attendees will usually retain only a handful of "nuggets" no matter how packed the agenda is) and interaction limits (you can't meet everyone at a large conference). 2. If the goal of a conference…

I'm well-known in a research community. I'm positioned such that I don't need more academic points. I've mostly stopped publishing in branded prestige academic venues, in part due to rejection rates.

My goal in doing work and writing papers is to see them disseminated. The acceptance/rejection process is asinine -- studies show it's basically random. I've had one paper in my whole career where the reviewers did a proper review (e.g. worked through the math). The rest were quick skims. Comments often show the reviewers never read the paper. The stuff that makes it through this process is often nonsense, while very high-quality work is often cut.

The very best paper I wrote in my career has never seen the light of day. It was shortened to a 4-page work-in-progress because a reviewer didn't read it (the feedback was literally nonsense: that the sample size was small enough to be anecdotal; I had the largest sample size in the history of the research field).

The only impact of this egoistical search for prestige-by-low-accept-rates is that people who have better things to do with their time leave, and that research dissemination is slowed.

Those excuses make little sense in the real world:

1) If your conference has a 10% accept rate, it's easy enough to book a bigger venue next year. I've been to conferences with dozens of people, and ones with tens of thousands. It all works well. Bigger ones work better, if anything.

2) PCs aren't thoughtful enough to do that well, and even so, the goal of a conference shouldn't be to select things which resonate with the entrenched PC. That's why many ideas need to wait for a generation of old, conservative professors to die to make it out there.

3) The whole obsession with prestige is stupid and misguided.

Journals and conferences ought to have quality bars. Are there typos and grammar errors? Were there clear IRB ethic violations? Did you use error bars on your plots? Was data fabricated? Is the research methodologically sound? Is it coherent and readable? And so on. If it passes those bars, it should be published. If no one reads it / attends a talk, that's okay too -- importance can and should be determined after-the-fact.

Re: The culture of rejection in computer science publications

#63

The academic system is completely broken for Computer Science, and I don't really see a way to fix it. The economic realities of the field just make it too risky to allow an exceptionally gifted individual to remain out in the open publishing research that could potentially destroy your business model. Justifying these low acceptance rates as somehow prestigious is really just creating even more perverse incentives t…

Many similar problems exist in most fields of academia. Biology has an even worse academic environment IMO, but unless you are highly computational it's not as easy to sell out to industry. Yeah there's pharma, but the straight out of PhD salaries aren't that exciting and the work environment is generally not as good as tech.

Not disagreeing with you at all on the CS front though obviously. I am interested to see how things go in the next few decades in the respective fields, as the ease of exit does affect who stays as you mention. But it also affects who joins in the first place, the pressure felt to get results, and hopefully down the road systemic incentive to fix the problems.

CS also has the perk of being easier to rejoin - you don't need to make a huge initial investment in most researchers to give them a chance. So I'm optimistic for reform in CS academia down the road. But it's a long road, and if academic politics continue to prevail then I'm deeply concerned about the state of all of our research institutions.

Seriously, I was recently at reunions for a "top" university where many people go into graduate programs, and it became a running joke trying to find an alum from any PhD program that wasn't jaded as fuck. Even some of the people that I was most confident would be killing it weren't (or at least felt they weren't). The majority were actively exploring industry opportunities and considered themselves unlikely to do an academic postdoc.

Re: The culture of rejection in computer science publications

#64

Earlier quoted context omitted.

Why not? You’re currently posting to a website that consistently curates high quality content via voting.

It can work well in conjunction with good and empowered moderators.

True, but I feel like the strength of the moderation at HN is that it is mostly formal and less editorial. People are moderated mostly for how they say things, not what they're saying, especially by explicitly choosing as the primary criteria for the site what hackers find interesting.

Not that you couldn't have a narrower focus, but it should be an explicit focus where the question of whether something merits moderation is a yes or no question that could be answered by anyone, rather than a know it when I see it stance that just reflects the current personal standards of an individual or clique.

Does this belong on Hacker News? Was it upvoted and not flagged too often by people with sufficient karma on Hacker News? Then dang doesn't have to make a decision. This is not a perfect characterization, because moderation here does rarely drop into attempts to exclude types of discussions that the patrons find annoying (which is rationalized because the subjects are controversial and draw in people who want to fight, and repetitive because these people fight all the time so have habits and standard arguments.)

I'm really just expanding on the word "empowered" here, because most mods have absolute power unless they have to answer to other mods. The difference here is not power, it's philosophy.

Re: The culture of rejection in computer science publications

#65

>Serving on a PC is a yeoman’s service, and the community owes them a debt of gratitude. However, I believe that a toxic culture has emerged. This blog is a call for PCs to change their priorities. does that acronym not stand for what i think it does?

Probably not, but I don't know what it actually does stand for. It's definitely not Personal Computer

The acronym is not web-search friendly either. Been looking through this thread and search engines. Nothing helped.

For example, not a single one of these [1] makes sense to me in that context.

[1]: https://www.allacronyms.com/PC/university

Re: The culture of rejection in computer science publications

#66
post #32

I'm long past my academia phase, but recently led the PC for an industry conference (accept rate: ~15%). 1. Curation is important both for the physical limits (venues only fit a certain number of people), attention limits (attendees will usually retain only a handful of "nuggets" no matter how packed the agenda is) and interaction limits (you can't meet everyone at a large conference). 2. If the goal of a conference…

> 3. While the size of any one conference has to be fixed, tech has made it infinitely easier to create new conferences and journals with other focus areas. They may not start with the prestige of a larger journal, but if the papers published start to have an impact, it can catalyze an entire subfield of work.

Does it though? The largest conferences I go to as a CS academic have hundreds of people. There are academic areas where 10x people participate. The size limitation is a self-imposed excuse to keep acceptance low. I have been PC chair of two conferences and my attempts to expand the conference numbers were shot down by the steering committee precisely for this reason, not because we couldn't find a larger room.

Re: The culture of rejection in computer science publications

#67
post #60

Earlier quoted context omitted.

> The "cream" rising to the top is often less genius and more politically savvy with the right connections on the PC. I'm generally nauseated when I interact with American CS academics. Every time I attend a conference, PC, or NSF panel, I am so glad I chose industry. It's like IRL twitter. (Europe seems to be better for some reason.) > If you're smart enough to rise, you'll get an offer from the private sector you s…

Should faculty be paid more? Absolutely. Should Ph.D. students be paid more? Absolutely!! But the blanket statement you make in (1) is wrong and strikes me as awfully close to the extreme left-wing and right-wing mindsets of "the system is fucked up beyond repair, all that remains to be done is to tear it down". The reality is more nuanced than this, and the picture you paint of industry is hardly that rosy, even at…

I've spent a lot of time with working for or closely interfacing with a half dozen academic institutions. I left academia by choice -- with multiple TT offers in hand -- so this isn't sour grapes.

I am highly confident in my assessment that the personalities found on the typical R1 tenure track are exactly the sort of personalities I avoid hiring or working with at all costs. There are exceptions, but they prove the rule (and I can often poach them anyways).

I don't think I said anything about industry other than that it pays 3x-5x better than the TT, and I'm pretty darn confident that's true. I am clear-eyed about the issues in industry, but the personalities are much better.

I really do believe that the massive pay disparity between CS industry and CS academia is, in part, a "toxic personality that can't play well with others" tax. And I really do believe that you'd get more mentally/emotionally healthy people on the TT if it paid better.

Anyways, we can agree to disagree, because we agree on the solution in any case.

Re: The culture of rejection in computer science publications

#68
post #22

> The emphasis on novelty has deep roots in academic publishing. It used to be that publishing was expensive, and any repetition came at the expense of other things that could have been published. Today, however, publishing is essentially free. Requiring novelty shows respect for the readers' time. Paper and ink costs were never the primary limiting factors (even if the publishers' claimed otherwise to save face). "r…

A clear exposition of an obscure and hard to understand result is lack of 'respect for readers' time'? How so?

Re: The culture of rejection in computer science publications

#69
post #2

I wonder if this culture arises out of necessity. Many conferences claim to not have a set quota of papers they accept, but in practice there is a limitation to how many presentations can be accommodated given the physical and temporal limitations of the venue. The quota may grow slightly over time, either by shrinking the time allotted to every presentation (ICFP is really squeezing it these years, for example) or a…

But journals do have an economic restriction. More pages cost more money and the subscription fees only go so far. I suppose page charges might balance some of the editorial costs, but those are also limiting. They can't be too high or they'll chase away submissions.

Re: The culture of rejection in computer science publications

#70
Nothing has made me happier than to simply leave the academic/publishing side of CS and focus entirely on things that I find interesting and can do on a single desktop computer. If I share, I share and give people my code and ideas freely, but don't publish. Why would I want to take all my hard work and beg somebody with power to put it on a piece of paper?

If I have something useful to say I'll put it in arxiv. Peer reviewers just wasted my time.

Post reply on HN