Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

2% of ICML papers desk rejected because the authors used LLM in their reviews

blog.icml.cc

Post reply on HN
Print view View original

61–70 of 172 posts

Re: 2% of ICML papers desk rejected because the authors used LLM in their reviews

#61

Post by matusp »

bonoboTP wrote: post #12
hodgehog11 wrote: post #4

I'm amazed that such a simple method of detection worked so flawlessly for so many people. This would not work for those who merely used LLMs to help pinpoint strengths and weaknesses in the paper; there are separate techniques to judge that. Instead, it only detects those who quite literally copied and pasted the LLM output as a review. It's incredible how so many people thought it was fair that their paper should b…

I'm not surprised at all. The ML research community isn't a community any more, it's turned into a dog-eat-dog low-trust fierce competition. So much more people, papers, churn, that everyone is just fending for themselves. Any moment that you charitably spend on community service can be felt as a moment you take away from the next project, jeopardizing the next paper, getting scooped, delaying your graduation, your c…

And the real punchline is that the deluge of papers barely matters, as the academic field is barely moving, and the most interesting innovations are happening on the product side.

Re: 2% of ICML papers desk rejected because the authors used LLM in their reviews

#62

Post by causalityltd »

The declaration of no-LLM was done for social prestige or maybe self-deception of self-sufficiency like "I don't need LLM". And when it was time to do the actual work, the dependency kicked in like drugs. A lesson for all of us with LLMs in our workflow.

Re: 2% of ICML papers desk rejected because the authors used LLM in their reviews

#63

Post by embedding-shape »

laughingcurve wrote: post #54
mikkupikku wrote: post #11

Earlier quoted context omitted.

If this is tolerated, it sends exactly the wrong kind of message. The students, if they are, should be banned for life. Let them serve as an example for myriads of future students, this will be a better outcome in the long run. This didn't trip for people who were merely bouncing ideas off a LLM, they caught people who copy and pasted straight from their LLM.

Thank goodness we have you passing judgment on the internet; otherwise who else would be around for us to do it? I'm glad you're willing to destroy someone for a mistake rather than letting them learn and change. We all know that arbitrary and harsh punishments solve everything.

> destroy someone for a mistake

"Oops, you told me not to do this, and I volunteered to agree to these stricter standards yet I flagrantly disregarded them, please forgive me" doesn't seem like something you just accidentally do, it's a conscious choice.

Re: 2% of ICML papers desk rejected because the authors used LLM in their reviews

#64

Post by mikkupikku »

linkregister wrote: post #16
mikkupikku wrote: post #11

Earlier quoted context omitted.

If this is tolerated, it sends exactly the wrong kind of message. The students, if they are, should be banned for life. Let them serve as an example for myriads of future students, this will be a better outcome in the long run. This didn't trip for people who were merely bouncing ideas off a LLM, they caught people who copy and pasted straight from their LLM.

It's not a fully consensus view, but a majority of sociologists agree that high severity deterrence has limited effectiveness against crime. Instead, certainty of enforcement is the most salient factor.

Deterrence is only part of it. It's morally instructive, it tells people that they live in a society that takes rules seriously.

Re: 2% of ICML papers desk rejected because the authors used LLM in their reviews

#65

Post by iso1631 »

causalityltd wrote: post #62

The declaration of no-LLM was done for social prestige or maybe self-deception of self-sufficiency like "I don't need LLM". And when it was time to do the actual work, the dependency kicked in like drugs. A lesson for all of us with LLMs in our workflow.

Sure I use LLMs in my workflows. I use a calculator too.

I can divide 98,324,672,722 by 161,024 by hand. At least I used to be able to do, but nobody is going to pay me to do that when a calculator exists.

Likewise I can write a bunch of assembly (well OK I can't), but why would I do that when my compiler can convert my intention into it.

Re: 2% of ICML papers desk rejected because the authors used LLM in their reviews

#66

Post by ancillary »

anonymousDan wrote: post #59
hodgehog11 wrote: post #9

Earlier quoted context omitted.

I was thinking this too, but I don't believe this is the case, and I feel like it would not be a good idea either. Most of these people are likely students; this should be a learning moment, but I don't think it is yet grounds for their entire academic career to be crippled by being unable to publish in a top-tier ML venue.

ML reviewing is a total joke. Why do you have noob students reviewing a conference paper.

I've been an AC (the person who manages the reviewing process and translates reviews into accept/reject decisions) at ICML and similar conferences a few times. In my experience, grad students tend to be pretty good reviewers. They have more time, they are less jaded, and they are keener to do a good job. Senior people are more likely to have the deep and broad field knowledge to accurately place a paper's value, but they are also more likely to write a short shallow review and move on. I think the worst reviews I've seen have been from senior people.

Re: 2% of ICML papers desk rejected because the authors used LLM in their reviews

#67

Post by bjourne »

linkregister wrote: post #16
mikkupikku wrote: post #11

Earlier quoted context omitted.

If this is tolerated, it sends exactly the wrong kind of message. The students, if they are, should be banned for life. Let them serve as an example for myriads of future students, this will be a better outcome in the long run. This didn't trip for people who were merely bouncing ideas off a LLM, they caught people who copy and pasted straight from their LLM.

It's not a fully consensus view, but a majority of sociologists agree that high severity deterrence has limited effectiveness against crime. Instead, certainty of enforcement is the most salient factor.

Correct. We also have evidence both from cheating in sports and in academia that stiff punishments do not work. Many people hold the false belief that if it is easy to cheat then the punishments must be extremely severe to scare would be cheaters. It just does not work. Preventing cheating is way easier said than done.

Re: 2% of ICML papers desk rejected because the authors used LLM in their reviews

#68

Post by quinndupont »

qbit42 wrote: post #48
quinndupont wrote: post #38

How is nobody considering the broader political economy of scholarly publications and reviews? These are UNPAID reviews! Sure, maybe ICML isn’t Elsevier, but they are cousins to the socially parasitic and exploitative companies, at the very least. Hiding behind a false “choice” to not use AI or basically not use AI isn’t an appropriate proposal. This is crooked and shameful. We should boycott ICML except we can’t bec…

What? Why is that a false choice? The only way you got caught here is if you literally gave an LLM the PDF and used its response verbatim. And they didn't give a permanent ban or anything, these authors can just resubmit to another conference, of which there are many.

Imagine you are poor and a rich person offers you a choice to steal some bread or some beer. It’s not a real choice because you are poor and therefore steal. The rich person offering the choice is wrong.

Re: 2% of ICML papers desk rejected because the authors used LLM in their reviews

#69

Post by bjourne »

grey-area wrote: post #20

Interesting, so someone submitting a paper for review could also submit one with hidden instructions for LLMs to summarise or review it in a very positive light. Given this detection method works so well in the use case of feeding reviewing LLMs instructions, it should also work for the original submitted paper itself, as long as it was passed along with its watermark intact. Even those just using LLMs to summarise c…

> Interesting, so someone submitting a paper for review could also submit one with hidden instructions for LLMs to summarise or review it in a very positive light.

Has been done: https://www.theguardian.com/technology/2025/jul/14/scientist...

Re: 2% of ICML papers desk rejected because the authors used LLM in their reviews

#70

Post by maleldil »

noduerme wrote: post #57
linkregister wrote: post #16

Earlier quoted context omitted.

It's not a fully consensus view, but a majority of sociologists agree that high severity deterrence has limited effectiveness against crime. Instead, certainty of enforcement is the most salient factor.

Enforcement without consequences just wears down the people who are supposed to enforce it.

GP suggested a life ban. Maybe suspend for 6 months instead? That's a long time without publishing in the current publish-or-perish academia.
Post reply on HN