Papers tend to roll down hill. Failure to accept by the big conferences pushes them to lower tier venues.
Queueing to publish in AI and CS
41–50 of 59 posts
Re: Queueing to publish in AI and CS
#42I just witnessed LLM (ab)use coming from one graduate student (not the first to do it and definitely not the last), where they submitted a conference paper draft for their coauthors and advisors to review short before a deadline, with completely regurgitated material plus hallucinations backed up by multiple non-existent citations. The problem is every coauthor wants to increase submissions, LLMs are great at making…
Anecdotally: I tried hook and crook to get the best flagship model at the time (Opus) to help with technical writing for a submission.
First, these models are not good at technical writing at all. They have no sense of the weight of a single sentence, they just love to blather.
Second, they can't keep the core technical story consistent throughout their completions. In other words, they can't "keep the main thing the main thing".
I had an early draft with AI writing, but by the time we submitted our work -- there was not a single piece of AI writing in the paper. And not without trying, I really did some iterations on trying to carefully craft context, give them a sense of the world model in which they needed to evaluate their additions, yada yada.
For clear and concise technical communication, it's a waste of time right now.
Re: Queueing to publish in AI and CS
#43I just witnessed LLM (ab)use coming from one graduate student (not the first to do it and definitely not the last), where they submitted a conference paper draft for their coauthors and advisors to review short before a deadline, with completely regurgitated material plus hallucinations backed up by multiple non-existent citations. The problem is every coauthor wants to increase submissions, LLMs are great at making…
Anecdotally: I tried hook and crook to get the best flagship model at the time (Opus) to help with technical writing for a submission. First, these models are not good at technical writing at all. They have no sense of the weight of a single sentence, they just love to blather. Second, they can't keep the core technical story consistent throughout their completions. In other words, they can't "keep the main thing the…
I'm so happy I have pre-LLM publications and blog posts to prove that my blathering isn't because I'm lazy and used Claude, it's just how I write (i.e., badly).
Re: Queueing to publish in AI and CS
#44Honest question: why not charge a fee per submission or per review? Or if the problem is bad papers, a fee that is returned unless it’s a universal strong reject. Or if you don’t want to miss the best papers, a fee only for resubmitted papers? Or a fee that is returned if your paper is strong accept? Or a fee that is returned if your paper is accepted. There’s some model that has to be fair (not a financial burden to…
Fees mean very little. Go for the jugular. Impact the career of people putting out substandard papers. Come up with a score for "citation strength" or something. Any given bad actor with too many substandard papers to his/her credit begins to negatively impact the "citation strength" of any paper on which they are a co-author. Maybe even negatively impacting the "citation strength" of papers that even cite papers aut…
I think that's basically Impact Factor / Page Rank....
Re: Queueing to publish in AI and CS
#45Earlier quoted context omitted.
Anecdotally: I tried hook and crook to get the best flagship model at the time (Opus) to help with technical writing for a submission. First, these models are not good at technical writing at all. They have no sense of the weight of a single sentence, they just love to blather. Second, they can't keep the core technical story consistent throughout their completions. In other words, they can't "keep the main thing the…
I'm so happy I have pre-LLM publications and blog posts to prove that my blathering isn't because I'm lazy and used Claude, it's just how I write (i.e., badly).
poorly…
Re: Queueing to publish in AI and CS
#46Earlier quoted context omitted.
I'm so happy I have pre-LLM publications and blog posts to prove that my blathering isn't because I'm lazy and used Claude, it's just how I write (i.e., badly).
poorly…
It would be v. funny if I got that wrong, but I do feel the need to point out that "badly" is indeed grammatically correct here because this is HN and pedantry is always on topic.
People over-correct and feel like they can't use "badly" because there is "feeling badly" discourse [0], but that pertains to "feeling" being a linking verb. "Write" is just your bog standard verb for which "badly", an adverb, is a totally valid modifier.
[0] https://www.merriam-webster.com/grammar/do-you-feel-bad-or-f...
Re: Queueing to publish in AI and CS
#47Honest question: why not charge a fee per submission or per review? Or if the problem is bad papers, a fee that is returned unless it’s a universal strong reject. Or if you don’t want to miss the best papers, a fee only for resubmitted papers? Or a fee that is returned if your paper is strong accept? Or a fee that is returned if your paper is accepted. There’s some model that has to be fair (not a financial burden to…
Fees mean very little. Go for the jugular. Impact the career of people putting out substandard papers. Come up with a score for "citation strength" or something. Any given bad actor with too many substandard papers to his/her credit begins to negatively impact the "citation strength" of any paper on which they are a co-author. Maybe even negatively impacting the "citation strength" of papers that even cite papers aut…
It doesn't address the core issue. It's credential inflation.
Not sure that enough people understand that the vast vast majority of research papers are written in order to fulfil criteria to graduate with a PhD. It's all PhD students getting through their program. That's the bulk of the literature.
There was a time when nobody went to school. Then everyone did 4 years elementary to learn reading, writing and basic arithmetic. Then everyone did 8 years, which included more general knowledge. Then it became the default to do 12 years to get to the high school diploma. Then it became default to do a bachelor's to get even simple office jobs. Then it's a masters. Then to actually stand out now in a way that a BSc or MSc made you stand out, you need a PhD. PhD programmes are ballooning. Just as the undergrad model had to change quite a bit when it went from 30 highly-motivated nerds starting CS in a year vs. 1000. These are massive systems, the tens or hundreds of thousands of PhD students must somehow be pushed through this system like clockwork. Just for one conference you get tens of thousands of authors submitting similar amount of papers and tens of thousands of reviewers.
You can't simply halt such a huge machine with a few little clever ideas.
Re: Queueing to publish in AI and CS
#48Honest question: why not charge a fee per submission or per review? Or if the problem is bad papers, a fee that is returned unless it’s a universal strong reject. Or if you don’t want to miss the best papers, a fee only for resubmitted papers? Or a fee that is returned if your paper is strong accept? Or a fee that is returned if your paper is accepted. There’s some model that has to be fair (not a financial burden to…
It would be a disproportionate blow to researchers in countries with less resources and/or more bureaucratized systems (which e.g. demand to see a "result" if you have paid a fee), who just wouldn't submit.
Re: Queueing to publish in AI and CS
#49Earlier quoted context omitted.
Fees mean very little. Go for the jugular. Impact the career of people putting out substandard papers. Come up with a score for "citation strength" or something. Any given bad actor with too many substandard papers to his/her credit begins to negatively impact the "citation strength" of any paper on which they are a co-author. Maybe even negatively impacting the "citation strength" of papers that even cite papers aut…
It doesn't address the core issue. It's credential inflation. Not sure that enough people understand that the vast vast majority of research papers are written in order to fulfil criteria to graduate with a PhD. It's all PhD students getting through their program. That's the bulk of the literature. There was a time when nobody went to school. Then everyone did 4 years elementary to learn reading, writing and basic ar…
Actually, the problem is pricing. If we could identify and correctly value new concepts, then we can dispense with citations and just use the correct sum of concept valuations. Perhaps a correctly designed futures market would not only solve getting the right PhD students the right jobs, but bring a lot of speculative capital into fundamental research?
Re: Queueing to publish in AI and CS
#50Earlier quoted context omitted.
It doesn't address the core issue. It's credential inflation. Not sure that enough people understand that the vast vast majority of research papers are written in order to fulfil criteria to graduate with a PhD. It's all PhD students getting through their program. That's the bulk of the literature. There was a time when nobody went to school. Then everyone did 4 years elementary to learn reading, writing and basic ar…
Actually, the problem is pricing. If we could identify and correctly value new concepts, then we can dispense with citations and just use the correct sum of concept valuations. Perhaps a correctly designed futures market would not only solve getting the right PhD students the right jobs, but bring a lot of speculative capital into fundamental research?
That's a very economics-minded approach. Also, I'm not quite sure what the futures would be about. That a paper will... get N citations? get a job for the first author? Achieve N stars on GitHub? N likes on social media? Be patented and put in a product? Turn X USD in profit? Bet on retraction? Bet on acceptance? On awards? Or replicability?
The first question is what scientific research is actually for. Is it merely for profitable technological applications? The Greek or the humanistic or the enlightenment ideal wasn't just that. Fundamental research can be its own endeavor, simply to understand more clearly something. We don't only do astronomy for example in order to build some better contraption and understanding evolution wasn't only about producing better medicine. But it's much harder to quantify elegance or aesthetics of an idea and its impact.
And if you say that this should only be a small segment, and most of it should be tech-optimization, I can accept that, but currently science runs also on this kind of aesthetic idealist prestige. In the overall epistemic economy of society, science fills a certain role. It's distinct from "mere" engineering. The Ph in PhD stands for philosophy.