Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Behind the scenes: the struggle for each paper (2021)

jeffhuang.com

Post reply on HN
Print view View original

21–30 of 47 posts

Re: Behind the scenes: the struggle for each paper (2021)

#21

Post by BrandoElFollito »

dotnet00 wrote: post #18
BrandoElFollito wrote: post #14

Another thing is that there is not enough pushback from the community at large. My PhD thesis was less than 40 pages long. The introduction was 1/2 a page (basically "if you need an introduction you should not read this, here are 3,4 books to get you started"). Then I copied/pasted from my articles and then came the acknowledgments (which I actually fund valuable because I wanted to thank my advisor for his non-scien…

A friend who recently finished her PhD had a similar experience, where all the senior scientists at our lab were concerned because her thesis was "only" 100 pages long and she didn't go through a professional editor to have it perfected. My preliminary defense thesis had to be 50+ pages, but during the presentation, it was pretty obvious that the committee had at best looked at the table of contents. It all feels lik…

Theses at my time were about 200 pages long. A friend of mine wrote two tomes.

I clearly stated that I would not waste my time and the reviewers are free to provide comments and we will see during the defense.

I found out that a lot of these "rules" are traditions that one can challenge and suddenly they are not traditions anymore.

Re: Behind the scenes: the struggle for each paper (2021)

#22

Post by jltsiren »

godelski wrote: post #13
schneems wrote: post #8

I had an assignment in the OMSCS course where we had to turn the results of a project into a paper and a presentation. It was eye opening on why so many CS papers are difficult to decipher. I’m used to writing on the web where the scroll is unlimited and everything is hyperlink able and potentially interactive. Journal papers are limited by length and so was our assignment. I had to cut virtually all helpful explanat…

> Journal papers are limited by length and so was our assignment I have always thought this was a bit silly and that it creates really weird effects that also decrease readability. An interesting point is that reviewers are not required to read the appendix of works. So everything is required to be in the front matter. This is a bit silly when we do things like research graphics or do generative works and such. You w…

Page limits force you to focus. As a researcher, you are often expected to communicate your ideas in 1 page, 3 pages, 10 pages, or 30 pages, for various purposes. If a journal asks for a 10-page paper, you write a 10-page paper. If a conference asks for a 1-page abstract, you write a 1-page abstract. Most people reading a paper are not interested in going through all the details, and those details should usually not be in the main paper.

It's also easier to find reviewers for short papers than for long ones.

Some the issues you mention are specific to CS conferences. Because there is only time for 1-2 rounds of reviews, the reviews focus more on accepting/rejecting the paper and less on clearing any misunderstandings before judging it. Conferences are are also more likely to have one-size-fits-all page limits, while journals often have several catagories of papers with different expectations of length.

Re: Behind the scenes: the struggle for each paper (2021)

#23

Post by nkurz »

godelski wrote: post #19
nkurz wrote: post #16

Earlier quoted context omitted.

Your point is correct, but I'm not surprised by the difference. I think "legibility" is the term of art here. Writing a paper like this makes it almost[1] certain to the institution he is applying to that he is capable of writing a paper of this quality, while all the other metrics (GPA, GRE, etc) are much more probabilistic. Since someone incapable of writing such a paper is probably unsuited for a PhD, it seems ent…

I agree and there's a lot in my comment to point to that. But my point is to distinguish between the metrics and the goals. I'm certain the author included in their CV that they had a pending paper when applying, so there is a signal, albeit a weaker but publishing is a weak signal to begin with. I agree that you need to use metrics. But we need to be clear that metrics are not enough and very incomplete themselves.…

>> Since someone incapable of writing such a paper is probably unsuited for a PhD

> I very much disagree with this.

Your disagreement is justified. I phrased that poorly. I meant it as a shorthand for "incapable of being trained to write such a paper". Showing that you already have the skill is proof, everything else just points to the possibility with varying degrees of accuracy.

I in turn disagree that "the purpose of schooling is to train people", at least if "schooling" refers to PhD programs. I think it's more that there aren't enough applicants who are able to perform without extensive training, so in practical terms PhD programs need to be willing to provide training. But at the same time, it's perfectly understandable that they would prefer to take applicants who have demonstrated ability to perform over those with statistical potential.

I'd prefer something like "The purpose of PhD programs is to advance the field". I'm personally in the odd category that I've co-authored several computer science research papers despite having dropped out to become a programmer prior to my BA. I've demonstrated my ability to perform much of the role of a PhD while simultaneously demonstrating that I perhaps shouldn't be relied upon to finish!

Re: Behind the scenes: the struggle for each paper (2021)

#25

Post by academia_hack »

ajsnigrutin wrote: post #10
fallon54 wrote: post #5

AKA why you probably don't want to be in academia

Yep... Everyone in academia complains about publishing papers, about the high prices of publishing, about "publish or perish", and then when they come high enough in "academia", require the same pain from the newcomers. It's like a closed circle of people both requiring papers for maintaining and advancing your carreer and at the same time complaining about those papers (and the publishing process), and not even thin…

I've giving "Accept - Minor Revisions" to every paper I've peer reviewed since getting my PhD other than two that were outright plagiarism. Figure it's important to the morale of grad students to get some positive validation and the vast majority of published research is garbage anyways so I don't feel particularly inclined to defend the trash heap as an unpaid reviewer. In practice, I find that I've tipped the scales in favor of a lot of borderline papers over the years and am quite happy about that.

Re: Behind the scenes: the struggle for each paper (2021)

#26

Post by sideshowb »

godelski wrote: post #20
outrun86 wrote: post #17

Earlier quoted context omitted.

Distill.pub was one effort to modernize publishing in CS. Chris Olah wrote some thoughts [1] about why he didn’t feel it was tenable. Seems like the primary challenge was the additional effort and skill involved in crafting rich-content/interactive material. [1] https://distill.pub/2021/distill-hiatus/

Honestly, I don't get why we don't just submit to OpenReview and call it a day. Paper is visible and distributed. There are comment sections where peer review can not just happen, but happen in the open (added bonus!). You can iterate and even see the difference between submissions. What is the conference/journal providing that isn't covered here? A stamp of approval? From a well known noisy system that creates other…

Promotion and filtering I guess? What does a record label provide when you can just upload music to Spotify?

Re: Behind the scenes: the struggle for each paper (2021)

#27

Post by vladms »

godelski wrote: post #20
outrun86 wrote: post #17

Earlier quoted context omitted.

Distill.pub was one effort to modernize publishing in CS. Chris Olah wrote some thoughts [1] about why he didn’t feel it was tenable. Seems like the primary challenge was the additional effort and skill involved in crafting rich-content/interactive material. [1] https://distill.pub/2021/distill-hiatus/

Honestly, I don't get why we don't just submit to OpenReview and call it a day. Paper is visible and distributed. There are comment sections where peer review can not just happen, but happen in the open (added bonus!). You can iterate and even see the difference between submissions. What is the conference/journal providing that isn't covered here? A stamp of approval? From a well known noisy system that creates other…

Not sure the openness of the review would solve so many problems of the system. For example would not touch like reproducibility and data and code availability.

Then you will need moderation (or do you imagine that things will be civilized between people on the internet?) and would need to manage various possibilities of bullying/targeting/etc. Of course these things can happen now, but difference would be between a potentially fully automated and simple system and something very clunky (be friends with an editor, convince him to report who are the reviewers, manage to recognize another of his papers, etc.)

Re: Behind the scenes: the struggle for each paper (2021)

#28

Post by sideshowb »

jltsiren wrote: post #22
godelski wrote: post #13

Earlier quoted context omitted.

> Journal papers are limited by length and so was our assignment I have always thought this was a bit silly and that it creates really weird effects that also decrease readability. An interesting point is that reviewers are not required to read the appendix of works. So everything is required to be in the front matter. This is a bit silly when we do things like research graphics or do generative works and such. You w…

Page limits force you to focus. As a researcher, you are often expected to communicate your ideas in 1 page, 3 pages, 10 pages, or 30 pages, for various purposes. If a journal asks for a 10-page paper, you write a 10-page paper. If a conference asks for a 1-page abstract, you write a 1-page abstract. Most people reading a paper are not interested in going through all the details, and those details should usually not…

I think desirability of page limits is very subject specific. Some people will just waffle if you don't give them a page limit. Other times it means there's not room for the technical details.

Re: Behind the scenes: the struggle for each paper (2021)

#29

Post by vladms »

fallon54 wrote: post #5

AKA why you probably don't want to be in academia

I think most of the things in life come with a lot of struggle, strange things, things that should be different and so on.

Making a startup? Go and check how hard and crazy that is. Make a family? Similar convoluted process with ups and downs.

What I think is wrong is that people have a very "idealized" image of a scientist scribbling on a board and equation and getting some prize (or defeating the aliens). These images are good for kids but after high-school I think people should give it a thought and say "ok, things are not exactly how I imagined in life, lets try to understand more what I like and want". You know the same process that makes people realize there is no Santa Claus.

Re: Behind the scenes: the struggle for each paper (2021)

#30

Post by cvwright »

ajsnigrutin wrote: post #15
JohnKemeny wrote: post #12

Earlier quoted context omitted.

This is only true to some extent, having myself been on a fair number of hiring committees. While the institution and national agencies measure impact in terms of number of "level 1/level 2" papers, colleagues don't care at all about this value. What's important is number of single-author papers, number of papers without their advisor, number of different small group collaborations, and most of all, having papers acc…

Sure, but the same group of people that complains about (the publishing of) papers, is then in a position to change that, but doesn't. All those people that went through this process and complained, then (well.. a few years later) sit in univeristy comitees that decide what the hiring (and scoring) rules are, what are the requirements fo TAs, for professors and tenures, etc., and decide, that the "pain of publishing"…

The problem is that academia is otherwise a pretty cushy job. It attracts a lot of people who want the prestige and like to talk, but don’t actually want to do any work.

Peer review and the paper chase are the least bad solution that we’ve come up with to address this.

Post reply on HN