Earlier quoted context omitted.
I'm not sure what experience anyone in this thread has with grad level research as a student/author, but I can assure you that heads roll over this kind of thing. A professor's career is built on reputation, and that reputation is as strong as their students' (who do much of the "work" such as it is). It comes down to the professor, but this can be a career-ending moment for those students and I'm quite confident the…
[flagged]
2% of ICML papers desk rejected because the authors used LLM in their reviews
101–110 of 172 posts
Re: 2% of ICML papers desk rejected because the authors used LLM in their reviews
#102Earlier quoted context omitted.
Yes for those in group B I'd suspect many were doing exactly what these cheaters in group A were doing - submitting the unaltered output of an LLM as their review.
The rejection is based on the dishonesty of explicitly committing to standard A and then knowingly violating it, not on LLM use as such. I think that's pretty fair, considering that everyone could have just chosen B if they wanted to.
Re: 2% of ICML papers desk rejected because the authors used LLM in their reviews
#103Interesting, so someone submitting a paper for review could also submit one with hidden instructions for LLMs to summarise or review it in a very positive light. Given this detection method works so well in the use case of feeding reviewing LLMs instructions, it should also work for the original submitted paper itself, as long as it was passed along with its watermark intact. Even those just using LLMs to summarise c…
> Interesting, so someone submitting a paper for review could also submit one with hidden instructions for LLMs to summarise or review it in a very positive light. Has been done: https://www.theguardian.com/technology/2025/jul/14/scientist...
LLMs have a real problem with not treating context differently from instructions. Because they intermingle the two they will always be vulnerable to this in some form.
Re: 2% of ICML papers desk rejected because the authors used LLM in their reviews
#104Earlier quoted context omitted.
I'm not sure what experience anyone in this thread has with grad level research as a student/author, but I can assure you that heads roll over this kind of thing. A professor's career is built on reputation, and that reputation is as strong as their students' (who do much of the "work" such as it is). It comes down to the professor, but this can be a career-ending moment for those students and I'm quite confident the…
[flagged]
LLMs can be very useful tools. However we also know there are a lot of bad uses and we are still trying to figure out where there are problems and where there are none.
Re: 2% of ICML papers desk rejected because the authors used LLM in their reviews
#105Earlier quoted context omitted.
I have a very simple solution to this but it is a bit expensive. I run two laptops, one that I talk to an LLM on and another where I do all my work and which is my main machine. The LLM is strictly there in a consulting role, I've done some coding experiments as well (see previous comments) but nothing that stood out to me as a major improvement. The trick is: I can't cut-and-paste between the two machines. So there…
This is a nice solution, but I think it speaks to just how enticing the problem is. This is the sort of tactic someone with a gambling addiction would employ. I don't say that to be rude to you: I've had to do similar things with regard to addicting infinite-scroll internet sites, and I definitely give in more than I'd like.
Re: 2% of ICML papers desk rejected because the authors used LLM in their reviews
#106Took me a while understand. So, the same person has both submitted their research article to the conference, and also acted as a reviewer for articles submitted by other people. And if they in their review work have agreed to a "no LLM use" policy, but got exposed using LLMs anyway, then their submitted research article is desk rejected. Theoretically, someone could have submitted a stellar research article, but beca…
But if anything, I think the whole anti-LLM review philosophy is wrong. If anything we need multiple deep background and research analyses of papers. So many papers are trash or are publishing what has already been done or are missing things. The volume of AI papers makes it impossible for a human alone to really critique work because hundreds of new papers come out a day.
Re: 2% of ICML papers desk rejected because the authors used LLM in their reviews
#107Earlier quoted context omitted.
>several hidden sentences in Finnish Is this a reference to something?
Not at all. It's just that reportedly LLMs used to have a blind spot for prompt injection in languages with relatively few speakers and grammar dissimilar to that of English.
Re: 2% of ICML papers desk rejected because the authors used LLM in their reviews
#108Earlier quoted context omitted.
What? Why is that a false choice? The only way you got caught here is if you literally gave an LLM the PDF and used its response verbatim. And they didn't give a permanent ban or anything, these authors can just resubmit to another conference, of which there are many.
Imagine you are poor and a rich person offers you a choice to steal some bread or some beer. It’s not a real choice because you are poor and therefore steal. The rich person offering the choice is wrong.
Are you objecting to ICML’s reciprocal reviewing policy?
The alternative as I see it would be to charge for submissions and pay reviewers. There are pros but also clear cons when it comes to fairness.
Re: 2% of ICML papers desk rejected because the authors used LLM in their reviews
#109Took me a while understand. So, the same person has both submitted their research article to the conference, and also acted as a reviewer for articles submitted by other people. And if they in their review work have agreed to a "no LLM use" policy, but got exposed using LLMs anyway, then their submitted research article is desk rejected. Theoretically, someone could have submitted a stellar research article, but beca…
Slightly more nuanced in that the reciprocal reviewer may have been essentially forced to sign despite having other commitments or may not have even been the lead contributor. Nowadays if a student submits a side project to a top-tier conference then it is required that if any authors have significant publication count in top-tier venues, then one must be a mandatory reviewer. Then one must sign that agreement. Stude…
What about you not putting your name on the paper? Or does it hurt the student if they publish in their own name only?
Re: 2% of ICML papers desk rejected because the authors used LLM in their reviews
#110Earlier quoted context omitted.
I'm not sure what experience anyone in this thread has with grad level research as a student/author, but I can assure you that heads roll over this kind of thing. A professor's career is built on reputation, and that reputation is as strong as their students' (who do much of the "work" such as it is). It comes down to the professor, but this can be a career-ending moment for those students and I'm quite confident the…
[flagged]
Read the article. They self-selected into the no-LLM group and then copy/pasted from an LLM. Not only dishonest but just not smart.