Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

2% of ICML papers desk rejected because the authors used LLM in their reviews

blog.icml.cc

Post reply on HN
Print view View original

51–60 of 172 posts

Re: 2% of ICML papers desk rejected because the authors used LLM in their reviews

#51

Post by duskdozer »

Tade0 wrote: post #35
grey-area wrote: post #20

Interesting, so someone submitting a paper for review could also submit one with hidden instructions for LLMs to summarise or review it in a very positive light. Given this detection method works so well in the use case of feeding reviewing LLMs instructions, it should also work for the original submitted paper itself, as long as it was passed along with its watermark intact. Even those just using LLMs to summarise c…

> Interesting, so someone submitting a paper for review could also submit one with hidden instructions for LLMs to summarise or review it in a very positive light. I may or may not know a guy who added several hidden sentences in Finnish to his CV that might have helped him in landing an interview.

>several hidden sentences in Finnish

Is this a reference to something?

Re: 2% of ICML papers desk rejected because the authors used LLM in their reviews

#52

Post by IshKebab »

Lerc wrote: post #49

I have heard people say that they find that people who broadcast their distaste for LLMs secretly use it. I was fairly sceptical of the claim, but this seems to suggest that it happens more than I would have thought. One wonders what leads them to the AI rejecting option in the first place.

I bet plenty of people that leave voicemails don't like listening to them.

Re: 2% of ICML papers desk rejected because the authors used LLM in their reviews

#53

Post by notrealyme123 »

mikkupikku wrote: post #7
bonoboTP wrote: post #5

To be clear, as the article says, these authors were offered a choice and agreed to be on the "no LLMs allowed" policy. And detection was not done with some snake oil "AI detector" but by invisible prompt injection in the paper pdf, instructing LLMs to put TWO long phrases into the review. They then detected LLM use through checking if both phrases appear in the review. This did not detect grammar checks and touchups…

In that case, I hope these frauds have been banned for life.

In many cases authors and reviewers are not the same. In your first two publications to such venues you are not allowed to review yourself and need someone else.

I think consequences are well deserved, but hopefully not on the authors cost (if innocent).

Re: 2% of ICML papers desk rejected because the authors used LLM in their reviews

#54

Post by laughingcurve »

mikkupikku wrote: post #11
hodgehog11 wrote: post #9

Earlier quoted context omitted.

I was thinking this too, but I don't believe this is the case, and I feel like it would not be a good idea either. Most of these people are likely students; this should be a learning moment, but I don't think it is yet grounds for their entire academic career to be crippled by being unable to publish in a top-tier ML venue.

If this is tolerated, it sends exactly the wrong kind of message. The students, if they are, should be banned for life. Let them serve as an example for myriads of future students, this will be a better outcome in the long run. This didn't trip for people who were merely bouncing ideas off a LLM, they caught people who copy and pasted straight from their LLM.

Thank goodness we have you passing judgment on the internet; otherwise who else would be around for us to do it? I'm glad you're willing to destroy someone for a mistake rather than letting them learn and change. We all know that arbitrary and harsh punishments solve everything.

Re: 2% of ICML papers desk rejected because the authors used LLM in their reviews

#55

Post by noduerme »

hodgehog11 wrote: post #9
mikkupikku wrote: post #7

Earlier quoted context omitted.

In that case, I hope these frauds have been banned for life.

I was thinking this too, but I don't believe this is the case, and I feel like it would not be a good idea either. Most of these people are likely students; this should be a learning moment, but I don't think it is yet grounds for their entire academic career to be crippled by being unable to publish in a top-tier ML venue.

2% would be on the very low end of the number of people who lie, get caught, and become repeat offenders anyway.

Re: 2% of ICML papers desk rejected because the authors used LLM in their reviews

#57

Post by noduerme »

linkregister wrote: post #16
mikkupikku wrote: post #11

Earlier quoted context omitted.

If this is tolerated, it sends exactly the wrong kind of message. The students, if they are, should be banned for life. Let them serve as an example for myriads of future students, this will be a better outcome in the long run. This didn't trip for people who were merely bouncing ideas off a LLM, they caught people who copy and pasted straight from their LLM.

It's not a fully consensus view, but a majority of sociologists agree that high severity deterrence has limited effectiveness against crime. Instead, certainty of enforcement is the most salient factor.

Enforcement without consequences just wears down the people who are supposed to enforce it.

Re: 2% of ICML papers desk rejected because the authors used LLM in their reviews

#59

Post by anonymousDan »

hodgehog11 wrote: post #9
mikkupikku wrote: post #7

Earlier quoted context omitted.

In that case, I hope these frauds have been banned for life.

I was thinking this too, but I don't believe this is the case, and I feel like it would not be a good idea either. Most of these people are likely students; this should be a learning moment, but I don't think it is yet grounds for their entire academic career to be crippled by being unable to publish in a top-tier ML venue.

ML reviewing is a total joke. Why do you have noob students reviewing a conference paper.

Re: 2% of ICML papers desk rejected because the authors used LLM in their reviews

#60

Post by boelboel »

Lerc wrote: post #49

I have heard people say that they find that people who broadcast their distaste for LLMs secretly use it. I was fairly sceptical of the claim, but this seems to suggest that it happens more than I would have thought. One wonders what leads them to the AI rejecting option in the first place.

Many addicts know doing drugs is bad. I'm sure a good portion of them are against drugs being freely available everywhere but they're still addicts.
Post reply on HN