Earlier quoted context omitted.
GP suggested a life ban. Maybe suspend for 6 months instead? That's a long time without publishing in the current publish-or-perish academia.
> Maybe suspend for 6 months instead? Suspend for 6 months from a conference that is held yearly?
2% of ICML papers desk rejected because the authors used LLM in their reviews
161–170 of 172 posts
Re: 2% of ICML papers desk rejected because the authors used LLM in their reviews
#162Earlier quoted context omitted.
In that case, I hope these frauds have been banned for life.
I'm not sure what experience anyone in this thread has with grad level research as a student/author, but I can assure you that heads roll over this kind of thing. A professor's career is built on reputation, and that reputation is as strong as their students' (who do much of the "work" such as it is). It comes down to the professor, but this can be a career-ending moment for those students and I'm quite confident the…
Those authors are still extremely prestigious professors in the field, and have suffered essentially no penalty. https://en.wikipedia.org/wiki/Growth_in_a_Time_of_Debt
Re: 2% of ICML papers desk rejected because the authors used LLM in their reviews
#163The declaration of no-LLM was done for social prestige or maybe self-deception of self-sufficiency like "I don't need LLM". And when it was time to do the actual work, the dependency kicked in like drugs. A lesson for all of us with LLMs in our workflow.
I don't personally use LLMs for this kind of stuff, but I'll certainly not sign a "No LLM" pledge unless you give me some kind of benefit.
Re: 2% of ICML papers desk rejected because the authors used LLM in their reviews
#164Took me a while understand. So, the same person has both submitted their research article to the conference, and also acted as a reviewer for articles submitted by other people. And if they in their review work have agreed to a "no LLM use" policy, but got exposed using LLMs anyway, then their submitted research article is desk rejected. Theoretically, someone could have submitted a stellar research article, but beca…
In any case, I had reached the same interpretation before reading your post, thinking that this is the only interpretation that could make any sense, but I'm still not convinced that this is what happened. Hopefully, no "innocent authors' articles were rejected because they happened to go to a bad reviewer".
Re: 2% of ICML papers desk rejected because the authors used LLM in their reviews
#165The declaration of no-LLM was done for social prestige or maybe self-deception of self-sufficiency like "I don't need LLM". And when it was time to do the actual work, the dependency kicked in like drugs. A lesson for all of us with LLMs in our workflow.
What was the upside to making the "No LLM" declaration? I don't personally use LLMs for this kind of stuff, but I'll certainly not sign a "No LLM" pledge unless you give me some kind of benefit.
Re: 2% of ICML papers desk rejected because the authors used LLM in their reviews
#166Earlier quoted context omitted.
I'm not sure what experience anyone in this thread has with grad level research as a student/author, but I can assure you that heads roll over this kind of thing. A professor's career is built on reputation, and that reputation is as strong as their students' (who do much of the "work" such as it is). It comes down to the professor, but this can be a career-ending moment for those students and I'm quite confident the…
Depends on the field. One of the most influential papers in economics was found to be incorrectly constructed with signs pointing to just straight up fraud. Basically it didn't include data that it said it did, which when included reverses the conclusion. Then when the authors were called out, they doubled down offering up the explanation that the conclusion again reverses if you add a third set of cherry picked data…
Re: 2% of ICML papers desk rejected because the authors used LLM in their reviews
#167Earlier quoted context omitted.
Depends on the field. One of the most influential papers in economics was found to be incorrectly constructed with signs pointing to just straight up fraud. Basically it didn't include data that it said it did, which when included reverses the conclusion. Then when the authors were called out, they doubled down offering up the explanation that the conclusion again reverses if you add a third set of cherry picked data…
All due respect this is by no means one of the most influential papers in economics.
There's a reason why it's one of the only economic papers that has its own wiki page.
Re: 2% of ICML papers desk rejected because the authors used LLM in their reviews
#168Earlier quoted context omitted.
What was the upside to making the "No LLM" declaration? I don't personally use LLMs for this kind of stuff, but I'll certainly not sign a "No LLM" pledge unless you give me some kind of benefit.
The other commenter explained that the policy was applied as reciprocal, "The declaration of no-LLM was done so you are not judged yourself by an LLM." Basically, they didn't want LLM near their own paper's review.
Although, a bunch of LLM researchers basically saying with their actions "Don't judge me with an LLM" is particularly ironic. Doubly so when caught using the LLM for the task they, themselves, want to opt out of.
Re: 2% of ICML papers desk rejected because the authors used LLM in their reviews
#169Earlier quoted context omitted.
Correct. We also have evidence both from cheating in sports and in academia that stiff punishments do not work. Many people hold the false belief that if it is easy to cheat then the punishments must be extremely severe to scare would be cheaters. It just does not work. Preventing cheating is way easier said than done.
> We also have evidence both from cheating in sports and in academia that stiff punishments do not work. Maybe so, but there is evidence that lack of punishment also don't work. Neither extreme "works". Just because terminal punishments do not prevent the worst cheating does not in any way imply that slap on the wrists reduce incidents of cheating.
Re: 2% of ICML papers desk rejected because the authors used LLM in their reviews
#170Earlier quoted context omitted.
> We also have evidence both from cheating in sports and in academia that stiff punishments do not work. Maybe so, but there is evidence that lack of punishment also don't work. Neither extreme "works". Just because terminal punishments do not prevent the worst cheating does not in any way imply that slap on the wrists reduce incidents of cheating.
That's the American spirit. "We got to do something!" "Does it work?" "We must do something!"
Are you claiming that one of the extremes "works"? That the "light punishment" route reduces cheating? Or maybe has no effect on cheating?
There are two extremes; I am not arguing that the one extreme (terminal punishment) reduces cheating, I am saying that the other extreme (light punishment) does not reduce cheating!
You say that stiff punishments have no effect on the cheating rate, right? Compare to what exactly? Compared to no punishments? Compared to light punishments? Compared to medium punishments? Compared to heavy but non-terminal punishments?
Now that I've reread your comment, I'm extremely skeptical that terminal punishments have no effect on the cheating rate compared to light punishments or compared to medium punishments.
It's an extraordinary claim, so I want to see this "lots of evidence"; the evidence should basically show no correlation between cheating and punishments.
I want to see that chart you base your belief on.