Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

2% of ICML papers desk rejected because the authors used LLM in their reviews

blog.icml.cc

Post reply on HN
Print view View original

111–120 of 172 posts

Re: 2% of ICML papers desk rejected because the authors used LLM in their reviews

#111

Post by gcr »

mikkupikku wrote: post #11
hodgehog11 wrote: post #9

Earlier quoted context omitted.

I was thinking this too, but I don't believe this is the case, and I feel like it would not be a good idea either. Most of these people are likely students; this should be a learning moment, but I don't think it is yet grounds for their entire academic career to be crippled by being unable to publish in a top-tier ML venue.

If this is tolerated, it sends exactly the wrong kind of message. The students, if they are, should be banned for life. Let them serve as an example for myriads of future students, this will be a better outcome in the long run. This didn't trip for people who were merely bouncing ideas off a LLM, they caught people who copy and pasted straight from their LLM.

This year, having their own submissions desk-rejected is strong enough of a signal that the policy has some teeth behind it. Let’s ban em for life next year.

I strongly feel that deterrence should be the goal here, not retribution IMO.

Re: 2% of ICML papers desk rejected because the authors used LLM in their reviews

#112

Post by zulban »

I've learned a bit today about how often people on hn read the article when commenting. Or potentially bots who are way off. The title alone isn't enough to totally grasp what happened here, or the methods used.

Extremely conservative detection. The real number must be much higher.

Re: 2% of ICML papers desk rejected because the authors used LLM in their reviews

#113

Post by matusp »

bonoboTP wrote: post #77
matusp wrote: post #61

Earlier quoted context omitted.

And the real punchline is that the deluge of papers barely matters, as the academic field is barely moving, and the most interesting innovations are happening on the product side.

I disagree with this. Usually the products are based on published research. This is not easily seen by the enthusiast power user base. Of course it's only a small fraction of all papers that end up actually being used. Most are mainly about advancing careers and strengthening CVs.

I have been in both academia and industry for years, and I don't think the model you describe is true anymore. It was definitely true 10 years ago, but the situation has flipped. Now, I see really ambitious and impactful research coming out of industry labs. Academia is often lagging behind the state of the art because they lack the resources (data, compute, and skills) to compete.

Academia is also incentivized such that everyone works on the same popular topics to secure grants and citations. This is currently LLMs, where academia needs to compete with multi-billion corporations on a technology that is notoriously expensive. In effect, many researchers work on topics that are pretty non-consequential from the get go (such as N+1th evaluation dataset), but it's the only way for them to stay relevant.

Re: 2% of ICML papers desk rejected because the authors used LLM in their reviews

#114

Post by ethmarks »

jacquesm wrote: post #19
everdrive wrote: post #6

Earlier quoted context omitted.

Generally speaking people have worse impulse control than they believe they do. Once you give a tool that does most of the work for you, very very few people will actually be able to use that tool in truly enriching ways. The majority of people (even the smart ones) will weaken over time and take shortcuts.

I have a very simple solution to this but it is a bit expensive. I run two laptops, one that I talk to an LLM on and another where I do all my work and which is my main machine. The LLM is strictly there in a consulting role, I've done some coding experiments as well (see previous comments) but nothing that stood out to me as a major improvement. The trick is: I can't cut-and-paste between the two machines. So there…

In a similar vein, I want a text editor where pasting from an external source isn't allowed. If you try, it should instantly remove the pasted text. Copy-pasting from inside the document would still be allowed (it could detect this by keeping track of every string in the document that has been selected by the cursor and allowing pastes that match one of those strings).

It wouldn't work in every use case (what if you need to include a verbatim quote and don't want to make typos by manually typing it?), but it'd be useful when everything in the document should be your words and you want to remove the temptation to use LLMs.

Re: 2% of ICML papers desk rejected because the authors used LLM in their reviews

#115

Post by rat9988 »

mikkupikku wrote: post #7
bonoboTP wrote: post #5

To be clear, as the article says, these authors were offered a choice and agreed to be on the "no LLMs allowed" policy. And detection was not done with some snake oil "AI detector" but by invisible prompt injection in the paper pdf, instructing LLMs to put TWO long phrases into the review. They then detected LLM use through checking if both phrases appear in the review. This did not detect grammar checks and touchups…

In that case, I hope these frauds have been banned for life.

Banned from doing free work?

Re: 2% of ICML papers desk rejected because the authors used LLM in their reviews

#116

Post by joshvm »

grey-area wrote: post #20

Interesting, so someone submitting a paper for review could also submit one with hidden instructions for LLMs to summarise or review it in a very positive light. Given this detection method works so well in the use case of feeding reviewing LLMs instructions, it should also work for the original submitted paper itself, as long as it was passed along with its watermark intact. Even those just using LLMs to summarise c…

This definitely happened to a paper that I submitted a couple of years ago. ChatGPT 4 was the frontier. The reviewer gave a positive, if bland, summary with some reasonable suggestions for improvement and some nitpicks. There were no grammar or line-number comments like those from other reviewers. They were all issues that would have been resolved by reading the appendices, but the reviewer hadn't uploaded into ChatGPT. Later on I was able to replicate the output almost exactly myself.

What I found funny was that if you asked ChatGPT to provide a score recommendation, it was also significantly higher than what that reviewer put. They were lazy and gave a middle grade (borderline accept/reject). We were accepted with high scores from the other reviews, but it was a bit annoying that they seemingly didn't even interpret the output from the model.

The learning experience was this: be an honourable academic, but it's in your interest to run your paper through Claude or ChatGPT to see what they're likely to criticise. At the very least it's a free, maybe bad, review. But you will find human reviewers that make those mistakes, or misinterpret your results, so treat the output with the same degree of skepticism.

Re: 2% of ICML papers desk rejected because the authors used LLM in their reviews

#117

Post by mikkupikku »

quinndupont wrote: post #39
mikkupikku wrote: post #7

Earlier quoted context omitted.

In that case, I hope these frauds have been banned for life.

It’s an unethical, false choice. The reviewers are not perfectly rational agents that do free work, they have real needs and desires. Shame on ICML for exploiting their desperation.

[flagged]

Re: 2% of ICML papers desk rejected because the authors used LLM in their reviews

#118

Post by lukan »

mikkupikku wrote: post #11
hodgehog11 wrote: post #9

Earlier quoted context omitted.

I was thinking this too, but I don't believe this is the case, and I feel like it would not be a good idea either. Most of these people are likely students; this should be a learning moment, but I don't think it is yet grounds for their entire academic career to be crippled by being unable to publish in a top-tier ML venue.

If this is tolerated, it sends exactly the wrong kind of message. The students, if they are, should be banned for life. Let them serve as an example for myriads of future students, this will be a better outcome in the long run. This didn't trip for people who were merely bouncing ideas off a LLM, they caught people who copy and pasted straight from their LLM.

Between banning someone for life and not doing anything, there usually are some other options.

Re: 2% of ICML papers desk rejected because the authors used LLM in their reviews

#119

Post by mikkupikku »

echelon wrote: post #89
jvanderbot wrote: post #85

Earlier quoted context omitted.

I'm not sure what experience anyone in this thread has with grad level research as a student/author, but I can assure you that heads roll over this kind of thing. A professor's career is built on reputation, and that reputation is as strong as their students' (who do much of the "work" such as it is). It comes down to the professor, but this can be a career-ending moment for those students and I'm quite confident the…

[flagged]

I consider LLMs to be a very useful tool and use them every day. But if I sign a slip of paper saying I won't use them for some project, and then use them anyway, not merely using them but copying without even the pretense of putting it into my own words, then that's fraud. LLMs being a tool is completely orthogonal to this fraud.

Re: 2% of ICML papers desk rejected because the authors used LLM in their reviews

#120

Post by mikkupikku »

andybak wrote: post #94
mikkupikku wrote: post #64

Earlier quoted context omitted.

Deterrence is only part of it. It's morally instructive, it tells people that they live in a society that takes rules seriously.

What is the aim of "moral instruction" if not deterrence? Surely it needs be instruction in pursuit of an outcome?

It makes honest people feel rewarded, valued and acknowledge. It teaches people who wish to follow the rules and conform to social norms what those norms are and where we actually draw the line in practice.

https://en.wikipedia.org/wiki/Punishment#Education_and_denun...

Post reply on HN