Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Queueing to publish in AI and CS

damaru2.github.io

Post reply on HN
Print view View original

51–59 of 59 posts

Re: Queueing to publish in AI and CS

#51

Post by dcminter »

jszymborski wrote: post #46
sroussey wrote: post #45

Earlier quoted context omitted.

poorly…

It would be v. funny if I got that wrong, but I do feel the need to point out that "badly" is indeed grammatically correct here because this is HN and pedantry is always on topic. People over-correct and feel like they can't use "badly" because there is "feeling badly" discourse [0], but that pertains to "feeling" being a linking verb. "Write" is just your bog standard verb for which "badly", an adverb, is a totally…

This is just a by the by, but in British English "feeling poorly" mostly means that you are ill. Amusingly it's become slightly euphemistic, so if someone is "a bit poorly" they probably have sniffles or a minor fever. If they are "very poorly" then you probably heard it from a hospital and they're just about dead.

Thus "I feel badly" ... "ok, what did you do?" vs. "I feel poorly" ... "ok, I'll get a bucket."

Re: Queueing to publish in AI and CS

#52

Post by lomase »

thrwo47294738 wrote: post #39

Honest question: why not charge a fee per submission or per review? Or if the problem is bad papers, a fee that is returned unless it’s a universal strong reject. Or if you don’t want to miss the best papers, a fee only for resubmitted papers? Or a fee that is returned if your paper is strong accept? Or a fee that is returned if your paper is accepted. There’s some model that has to be fair (not a financial burden to…

The people using AI are already paying to OpenAI or whoever to create those fake papers.

Re: Queueing to publish in AI and CS

#53

Post by jltsiren »

bonoboTP wrote: post #47
bilbo0s wrote: post #40

Earlier quoted context omitted.

Fees mean very little. Go for the jugular. Impact the career of people putting out substandard papers. Come up with a score for "citation strength" or something. Any given bad actor with too many substandard papers to his/her credit begins to negatively impact the "citation strength" of any paper on which they are a co-author. Maybe even negatively impacting the "citation strength" of papers that even cite papers aut…

It doesn't address the core issue. It's credential inflation. Not sure that enough people understand that the vast vast majority of research papers are written in order to fulfil criteria to graduate with a PhD. It's all PhD students getting through their program. That's the bulk of the literature. There was a time when nobody went to school. Then everyone did 4 years elementary to learn reading, writing and basic ar…

The number of new PhDs in the US ballooned in the 1960s. It went from 4.9 new PhDs / 100k people in 1960 to 14.5 in 1970 and 17.1 in 2024.

While the growth in the number of new PhDs has been modest, the number of published papers has grown much faster. I would attribute that to changes in administrative culture. Both the government and the universities have become driven by metrics, which means everyone must produce something the administrators can measure.

Re: Queueing to publish in AI and CS

#54

Post by bonoboTP »

jltsiren wrote: post #53
bonoboTP wrote: post #47

Earlier quoted context omitted.

It doesn't address the core issue. It's credential inflation. Not sure that enough people understand that the vast vast majority of research papers are written in order to fulfil criteria to graduate with a PhD. It's all PhD students getting through their program. That's the bulk of the literature. There was a time when nobody went to school. Then everyone did 4 years elementary to learn reading, writing and basic ar…

The number of new PhDs in the US ballooned in the 1960s. It went from 4.9 new PhDs / 100k people in 1960 to 14.5 in 1970 and 17.1 in 2024. While the growth in the number of new PhDs has been modest, the number of published papers has grown much faster. I would attribute that to changes in administrative culture. Both the government and the universities have become driven by metrics, which means everyone must produce…

I would imagine there is more recent ballooning in AI topics, which was the focus of the article. Based on the growth of the flagship conferences it's quite evident that it's not just exponentially more frequent paper submissions but indeed more exponentially more PhD students submitting. (I mean ML/AI/NLP/vision conferences over the last ~15 years).

Re: Queueing to publish in AI and CS

#55

Post by dimatura »

Al-Khwarizmi wrote: post #33
dimatura wrote: post #31

Earlier quoted context omitted.

Depends on who is doing the "careers valuing" and how closely they're looking. At a coarse level, especially for jobs in industry, venue is a pretty simple (but obviously imperfect) indicator for quality. If you've managed to publish one or more papers at the most selective venues (esp. as main author), then I would assume there's a decent chance you are good at research, even if I don't know anything about the subfi…

> But for academic or other high-level research jobs, whoever is doing the valuing is going to look at a lot more than just the venue. Depends on where. In some countries (e.g. mine, Spain), the notion that evaluation should be "objetive" leads to it degenerating into a pure bean-counting exercise: a first-quartile JCR indexed journal paper is worth 10 points, a top-tier (according to a specific ranking) conference p…

Yeah, that's a good point. I was thinking in the US context, but I also have some experience for the academic evaluation process in Chile, and there were similar issues to what you were describing. The "bean counting" part of it was an issue for academics in CS, because the rules were the same across departments, even where it didn't really make sense. So for example CS profs got no credit (towards promotions) for publishing in conferences, even if they were highly selective ones like ICCV or N(eur)IPS.

Re: Queueing to publish in AI and CS

#56

Post by Al-Khwarizmi »

dimatura wrote: post #55
Al-Khwarizmi wrote: post #33

Earlier quoted context omitted.

> But for academic or other high-level research jobs, whoever is doing the valuing is going to look at a lot more than just the venue. Depends on where. In some countries (e.g. mine, Spain), the notion that evaluation should be "objetive" leads to it degenerating into a pure bean-counting exercise: a first-quartile JCR indexed journal paper is worth 10 points, a top-tier (according to a specific ranking) conference p…

Yeah, that's a good point. I was thinking in the US context, but I also have some experience for the academic evaluation process in Chile, and there were similar issues to what you were describing. The "bean counting" part of it was an issue for academics in CS, because the rules were the same across departments, even where it didn't really make sense. So for example CS profs got no credit (towards promotions) for pu…

Yes, the issue you mention is also typical in many countries with subpar systems. In Spain it used to be exactly the same. Lately, top conferences are getting recognition in some contexts, but there are still some calls where they don't count and it's better to have a crappy journal paper. I mostly publish in conferences but always have to make sure to have enough indexed journal papers per 6-year period to feed the system.

Re: Queueing to publish in AI and CS

#57

Post by nsagent »

suddenlybananas wrote: post #9
Al-Khwarizmi wrote: post #4

Earlier quoted context omitted.

I'm in CS and I submit both to conferences and journals (the former because it's what people actually read, the latter because of evaluation requirements in my country). And I can tell you that (IMO of course) the conference model is immensely better, and idealization of journals in the CS community is a clear case of "grass is always greener". Revise and resubmit is evil. It gives the reviewers a lot of power over p…

I agree there's tons of problems with journals as well, I think an entirely different system could probably be better. Even preprints with some sort of public facing moderated comments could be more effective. However, I think this notion of a paper becoming "obsolete" if it isn't published fast enough speaks to the deeper problems in ML publishing; it's fundamentally about publicizing and explaining a cool technique…

I also used to submit to *ACL conferences exclusively, and was even a SAC, but got out of the academic game altogether. I'm still conducting research, but as an independent researcher for my own startup. The system seems to be more and more bananas over time and no one is willing to just force a change. It's really a tragedy of the commons.

That said, why don't conferences work like journals: if you're rejected you cannot resubmit. Find a new conference. That gets rid of the queuing problem. Yes, you'll have some amazing papers that will not be accepted by a top conference. So what, it happens to everyone. Plenty of influential papers were not published in a top conference in ML/AI/NLP.

Re: Queueing to publish in AI and CS

#58

Post by pyuser583 »

jwrallie wrote: post #25

I just witnessed LLM (ab)use coming from one graduate student (not the first to do it and definitely not the last), where they submitted a conference paper draft for their coauthors and advisors to review short before a deadline, with completely regurgitated material plus hallucinations backed up by multiple non-existent citations. The problem is every coauthor wants to increase submissions, LLMs are great at making…

"there are LLM written papers being peer reviewed by LLMs, but fear it not, even if they are accepted the will not be cited because LLMs are hallucinating citations that better support their arguments!"

I just shared that with several friends, and we are all having a good laugh. Thank you very much.

Re: Queueing to publish in AI and CS

#59

Post by diamondage »

bonoboTP wrote: post #50
diamondage wrote: post #49

Earlier quoted context omitted.

Actually, the problem is pricing. If we could identify and correctly value new concepts, then we can dispense with citations and just use the correct sum of concept valuations. Perhaps a correctly designed futures market would not only solve getting the right PhD students the right jobs, but bring a lot of speculative capital into fundamental research?

That's a very economics-minded approach. Also, I'm not quite sure what the futures would be about. That a paper will... get N citations? get a job for the first author? Achieve N stars on GitHub? N likes on social media? Be patented and put in a product? Turn X USD in profit? Bet on retraction? Bet on acceptance? On awards? Or replicability? The first question is what scientific research is actually for. Is it merely…

The question 'what is science actually for' can be sidestepped. Everyone can agree that it has value, albeit we disagree on the actual value...this is why you need a market. As to how things get priced in such a market, this is a subject for further research...To start, it just needs to tie to something measurable. Heck we've created memecoins with far less backing. Also, we've carved up the conceptual space on a very course grained level with patents, we just need a more immediate, and granular system for doing so...
Post reply on HN