Live data from Hacker News

Best Papers vs. Top Cited Papers in Computer Science

arnetminer.org

21–29 of 29 posts

Re: Best Papers vs. Top Cited Papers in Computer Science

#21
In 2009, Bartneck et al. did a scientometric analysis of papers presented at CHI - the most prestigious academic HCI conference [1]:

"The papers acknowledged by the best paper award committee were not cited more often than a random sample of papers from the same years."

[1] http://www.bartneck.de/publications/2009/scientometricAnalys...

Re: Best Papers vs. Top Cited Papers in Computer Science

#22

You could also see this as 'citation counting' not being a very precise measure of paper quality/importance.

This is certainly the case. On the other hand, "best papers" are not necessarily really the most important papers. Often politics, e.g., proportional representation of all important sub-fields among the best paper nominees, decide about whether a paper gets named "best" or not.

Anecdotally, I know of awesome papers that were not selected as "best paper" and 'cute idea' papers without substance that got a "best paper" award.

Re: Best Papers vs. Top Cited Papers in Computer Science

#23
post #8

Earlier quoted context omitted.

That somehow reinforces that publishing source code with CS research papers is more than necessary :)

Indeed, it makes me wonder if published code leads to higher citation counts. Although if it does, I suspect it has more to do with the fact that there's a product & marketing around it.

I wonder if it's more subtle than that - if your paper has associated source code, then it's likely that people reading it might try it out, and the ideas that you've presented will stick around more than a short paper with no follow up material.

In other words, papers are only a short glimpse into your research, presenting code allows an interested reader to look deeper, and means that they're more likely to remember what you've done, and cite it later.

Re: Best Papers vs. Top Cited Papers in Computer Science

#24
post #21

In 2009, Bartneck et al. did a scientometric analysis of papers presented at CHI - the most prestigious academic HCI conference [1]: "The papers acknowledged by the best paper award committee were not cited more often than a random sample of papers from the same years." [1] http://www.bartneck.de/publications/2009/scientometricAnalys...

CHI has a huge number of papers though, and consequently a large number of best papers...so it is difficult for anything to stand out a priori.

Re: Best Papers vs. Top Cited Papers in Computer Science

#25
post #22

You could also see this as 'citation counting' not being a very precise measure of paper quality/importance.

This is certainly the case. On the other hand, "best papers" are not necessarily really the most important papers. Often politics, e.g., proportional representation of all important sub-fields among the best paper nominees, decide about whether a paper gets named "best" or not. Anecdotally, I know of awesome papers that were not selected as "best paper" and 'cute idea' papers without substance that got a "best paper"…

Or papers given the most 10 year influential award that aren't influential. It's all relative!

Re: Best Papers vs. Top Cited Papers in Computer Science

#26
post #19

As an NLP researcher, I see lots of NLP papers on top of AAAI. I think that the high citation count is due to (i) people in NLP actually bothering to tell others why their approach is interesting (ii) other people being interested in the same / a similar kind of thing [avoiding the discipline-of-one problem that niche AI applications would have] and (iii) NLP having a reasonably developed "canon" about what counts as…

[deleted]

Re: Best Papers vs. Top Cited Papers in Computer Science

#27
Nitpicking, but why are they claiming to provide MAP (mean average precision) scores when their description and equation indicates that they are computing average precision, not MAP. According to the definition of MAP [1] that they link to, MAP is computed across multiple queries while average precision is computed for one [2]. Furthermore, they truncate their calculation to only consider the top 3 cited papers (i.e., they don't go all the way to 100% recall), so it's not even really the average precision.

[1] http://en.wikipedia.org/wiki/Information_retrieval#Mean_aver...

[2] http://en.wikipedia.org/wiki/Information_retrieval#Average_p...

Re: Best Papers vs. Top Cited Papers in Computer Science

#28
post #18
post #5

Reviewing SIGMOD, it appears that a lot of the citations earned are less about innovative research, and more about the everyone using the software tools they published. And a survey paper in the field of big data analysis (survey papers are citation bate, but won't be pulling in many grants or awards).

Once a paper gets big enough you basically have to cite it any time you touch on vaguely related just to prove that you are aware of it, almost as a shibboleth. My wife is an academic and on more than one occasion has she gotten comments back from peer review along the lines of "good article, but you why haven't you cited famous papers X,Y and Z".

I really appreciate this practice. It makes it much easier to get up to speed on something new.

Re: Best Papers vs. Top Cited Papers in Computer Science

#29
post #15
post #12

Earlier quoted context omitted.

My high school science teacher told us that he was given the prize for the photoelectric effect because that was a well understood mechanism by the time of the awarding. It was generally understood that relativity was going to be a bigger deal (concept wise), but since it was still so hotly debated / not yet thoroughly accepted, the commission wanted to ensure that he was given a prize before death (the award is not…

"In 1905 Albert Einstein published a paper that explained experimental data from the photoelectric effect as being the result of light energy being carried in discrete quantized packets. This discovery led to the quantum revolution." So, he basically described photons as a particle for the first time. That's also ridiculously huge. http://en.wikipedia.org/wiki/Photoelectric_effect Considering they had been generally…

You are forgetting Sir Isaac :-)

I don't think he used the word photon, but he did describe light as particles, and had (flawed, but that is science) arguments for that (http://www.studyphysics.ca/newnotes/20/unit04_light/chp1719_...)

Post reply on HN