Behind the scenes: the struggle for each paper (2021)

> Page limits force you to focus.

This can be solved in better ways, which is, in fact, reviewers. I'm okay with a soft requirement but a standardization is what I'm getting at as being problematic. Some papers are noisy because they should be 3 pages but are 10. Some papers are noisy because they are 10 pages and should be 30. There is no universal rule, and that's what I'm getting at.

> It's also easier to find reviewers for short papers than for long ones.

That's a separate problem that needs to be addressed, but is not easy.

> Some the issues you mention are specific to CS conferences.

Yes, but the author here is CS and we are on a CS focused website. But in general what I said isn't specific to conferences. If conferences are the problem then let's abandon them in favor of good science instead of keeping them around (or turn them into being meetup focused). Certainly the lack of back and forth between authors and reviewers is not a meaningful review process (most author rebuttals are limited to one page and often reviewers are not aligned in critiques). Are we all on the same team (better science) or strictly competing against one another?

> For example would not touch like reproducibility and data and code availability.

These are different issues, which are certainly important. But I do think in some way this would help. OpenReview does allow you to post comments many months after. Effectively think about this as a GitHub issues page. It certainly could be organized better but it is better than what exists now. OR also has links for code and community implementations (as does arxiv now). Here's an example that has all these things[0]. Granted data is missing, but I don't see why this can't also be integrated, but would need to also push cultural norms.

> Then you will need moderation

I think OR has this a bit solved, similarly arxiv. They are not anonymous accounts and are tied to your ORCID record. Arxiv requires you to have a verifier that is already someone with an arxiv account. Yes, this can be abused, but it is also an easier moderation problem that say Reddit or HN even. I think if you're posting bullying comments under a named profile, then it is good that that is visible so others can see. Mind you, bullying does already exist but it is just behind closed doors. It is worse now because only the Area Chair can take action and often they are over worked and works do get dismissed (which results in A LOT of wasted time, and money) because of this bullying. The larger the field, the more noise too and the more this happens. It is just far less common to see people bullying in public than behind closed doors.

I must stress though, that there is no perfect system here. There is no system that can make the amount of bullying 0. So we have to be careful in our critiques because there will always be valid critiques that are in fact of concern (like this one) but are fundamentally unsolvable. The question then becomes if we improve upon the existing frameworks and if whatever costs have been made are worth the added benefits. So I just want to make sure that this idea isn't killed because an impossible bar, despite the critique being valid.

Edit: I'd actually add that this system encourages reproduction. Because if we still measure on citations and number of publications this means that reproduction works can still count towards those metrics and thus someone's career advancements. The whole conference/journal system currently discourages such effort in favor of the absurdly nebulous novelty concept (which also makes papers noisy). My proposal would also allow for the publication of failures, which is also an important thing for academics.

[0] https://openreview.net/forum?id=Hkxzx0NtDB

I see your point and I think that brings us a bit closer to alignment. But I think if someone is __incapable__ of writing such a paper there would likely be many larger flags and they probably should not have been able to pass their undergraduate curriculum.

I do want to make it clear: I'm not opposed to arbitrary filters when there is a high number of applicants and you simply need to reduce the number. I am opposed to pretending that such a filter is not arbitrary. I think we need to be clear about how strong of a signal any filter is, and be quite explicit that they are not all equal indicators. That is my main point: being explicit about the strength of a signal.

On regards to training, I do agree that schooling isn't __just__ training, but I'd fully disagree that this isn't one of the most important aspects of it, even in grad school. Your first two years (in US systems) are nearly identical to a masters and highly focused on classes. What are classes but training? Even being a TA or lecturer is, in part, training (full instructor of record would not be). Post conditional, I still think you are in training at least up until candidacy. That is much more arguable given the variability of advisors, with some being very hands on (training) and some being very hands off (on your own).

I'd prefer something as "The purpose of PhD programs is to train people to advance the field." Because by all accounts, it seems like you've done this (even with the self-deprecating humor. That is exceptionally common in PhDs too lol). I still maintain training because this isn't the end, but the beginning. Post PhD is where you can choose to go to be an academic researcher or industry researcher (or abandon research). Those are the actual jobs (which should have continued training) but your degree is more akin to a certification from your institution. You do come out with a body of work that is distinct from the institution, but the institution's goal is not to keep you around and continue performing work. They are explicitly formed to graduate you. To educate you. And what is education except a form of training?

Fwiw, I think we are decently aligned, but sometimes text is hard to communicate, especially post by post. I do think your critiques are valid, even where I disagree.

Yes, the only hard rule in my department is having a minimum of 50 pages, the idea that 100 pages is not enough came from the scientists applying their own experiences from years ago. Technically there was nothing they could do about her thesis having fewer pages, but as inexperienced students, it's obviously a little scary when people you look up to sound concerned (since academia is full of all sorts of unproductive and unstated expectations).

A friend at another department only had a minimum requirement of 5 pages, and his thesis ended up being just a collection of his publications.

Nearly three times the number of papers published by Claudine Gay. Why isn't he President of Harvard?