Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

2% of ICML papers desk rejected because the authors used LLM in their reviews

blog.icml.cc

Post reply on HN
Print view View original

11–20 of 172 posts

Re: 2% of ICML papers desk rejected because the authors used LLM in their reviews

#11

Post by mikkupikku »

hodgehog11 wrote: post #9
mikkupikku wrote: post #7

Earlier quoted context omitted.

In that case, I hope these frauds have been banned for life.

I was thinking this too, but I don't believe this is the case, and I feel like it would not be a good idea either. Most of these people are likely students; this should be a learning moment, but I don't think it is yet grounds for their entire academic career to be crippled by being unable to publish in a top-tier ML venue.

If this is tolerated, it sends exactly the wrong kind of message. The students, if they are, should be banned for life. Let them serve as an example for myriads of future students, this will be a better outcome in the long run.

This didn't trip for people who were merely bouncing ideas off a LLM, they caught people who copy and pasted straight from their LLM.

Re: 2% of ICML papers desk rejected because the authors used LLM in their reviews

#12

Post by bonoboTP »

hodgehog11 wrote: post #4

I'm amazed that such a simple method of detection worked so flawlessly for so many people. This would not work for those who merely used LLMs to help pinpoint strengths and weaknesses in the paper; there are separate techniques to judge that. Instead, it only detects those who quite literally copied and pasted the LLM output as a review. It's incredible how so many people thought it was fair that their paper should b…

I'm not surprised at all. The ML research community isn't a community any more, it's turned into a dog-eat-dog low-trust fierce competition. So much more people, papers, churn, that everyone is just fending for themselves. Any moment that you charitably spend on community service can be felt as a moment you take away from the next project, jeopardizing the next paper, getting scooped, delaying your graduation, your contract, your funding, your visa, your residence permit, your industry plans etc. It's a machine. I don't think people outside the phd system really understand the incentives involved. People are offered very little slack in this system. It's sink or swim, with very little instruction or scientific culture or integrity getting passed on. The PhD students see their supervisors cut corners all the time too, authorship bullshit jockeying even in big name labs etc. People I talked to are quite disillusioned, expect their work to have little impact and get superseded by a new better model in a few months so it's all about who can grind faster, who can twist the benchmarks into showing a minimal improvement etc. And the starry eyed novices get slapped by reality into thinking this way fairly early.

To be clear this is not an excuse but an explanation why I am not surprised.

Re: 2% of ICML papers desk rejected because the authors used LLM in their reviews

#13

Post by jsnell »

coldtea wrote: post #8

Another 30-40% just didn't get caught because the reviewers also used LLM in their "reviews"

I think you've misunderstood something. This is not about rejecting LLM-written articles. It is about rejecting the articles of people who used LLMs for their reviews.

So your quip is just nonsensical.

Re: 2% of ICML papers desk rejected because the authors used LLM in their reviews

#14

Post by jacquesm »

I keep spotting clear LLM 'tells' in text where I know the people on the other side believe they're 'getting away with it'. It is incredible at what levels of commerce people do this, and how they're prepared to risk their reputation by saving a few characters typed. It makes me wonder what they think they are getting paid for.

Re: 2% of ICML papers desk rejected because the authors used LLM in their reviews

#15

Post by coldtea »

jsnell wrote: post #13
coldtea wrote: post #8

Another 30-40% just didn't get caught because the reviewers also used LLM in their "reviews"

I think you've misunderstood something. This is not about rejecting LLM-written articles. It is about rejecting the articles of people who used LLMs for their reviews. So your quip is just nonsensical.

Those second-level reviewers, checking whether the first-level authors used LLMs in their reviews, also used LLMs to do their screening, and the latter missed it in many cases.

My original point (loosely based on the subject, not TFA) is that it's LLMs all the way down, way more than it's "measured" to be.

Re: 2% of ICML papers desk rejected because the authors used LLM in their reviews

#16

Post by linkregister »

mikkupikku wrote: post #11
hodgehog11 wrote: post #9

Earlier quoted context omitted.

I was thinking this too, but I don't believe this is the case, and I feel like it would not be a good idea either. Most of these people are likely students; this should be a learning moment, but I don't think it is yet grounds for their entire academic career to be crippled by being unable to publish in a top-tier ML venue.

If this is tolerated, it sends exactly the wrong kind of message. The students, if they are, should be banned for life. Let them serve as an example for myriads of future students, this will be a better outcome in the long run. This didn't trip for people who were merely bouncing ideas off a LLM, they caught people who copy and pasted straight from their LLM.

It's not a fully consensus view, but a majority of sociologists agree that high severity deterrence has limited effectiveness against crime. Instead, certainty of enforcement is the most salient factor.

Re: 2% of ICML papers desk rejected because the authors used LLM in their reviews

#17

Post by harmf »

mikkupikku wrote: post #11
hodgehog11 wrote: post #9

Earlier quoted context omitted.

I was thinking this too, but I don't believe this is the case, and I feel like it would not be a good idea either. Most of these people are likely students; this should be a learning moment, but I don't think it is yet grounds for their entire academic career to be crippled by being unable to publish in a top-tier ML venue.

If this is tolerated, it sends exactly the wrong kind of message. The students, if they are, should be banned for life. Let them serve as an example for myriads of future students, this will be a better outcome in the long run. This didn't trip for people who were merely bouncing ideas off a LLM, they caught people who copy and pasted straight from their LLM.

[flagged]

Re: 2% of ICML papers desk rejected because the authors used LLM in their reviews

#18

Post by jacquesm »

hodgehog11 wrote: post #4

I'm amazed that such a simple method of detection worked so flawlessly for so many people. This would not work for those who merely used LLMs to help pinpoint strengths and weaknesses in the paper; there are separate techniques to judge that. Instead, it only detects those who quite literally copied and pasted the LLM output as a review. It's incredible how so many people thought it was fair that their paper should b…

This is 'spam' all over again. Before spam every email was valuable and required some attention. It was a better version of paper mail in that it was faster and cheaper. But then the spam thing happened and suddenly being 'faster and cheaper' was no longer an advantage, it was a massive drawback. But by then there was no way back. I think LLMs will do the same with text in general. By making the production of text faster and cheaper the value of all text will diminish, quite probably to something very close to the energy value of the bits that carry the data.

Re: 2% of ICML papers desk rejected because the authors used LLM in their reviews

#19

Post by jacquesm »

everdrive wrote: post #6
hodgehog11 wrote: post #4

I'm amazed that such a simple method of detection worked so flawlessly for so many people. This would not work for those who merely used LLMs to help pinpoint strengths and weaknesses in the paper; there are separate techniques to judge that. Instead, it only detects those who quite literally copied and pasted the LLM output as a review. It's incredible how so many people thought it was fair that their paper should b…

Generally speaking people have worse impulse control than they believe they do. Once you give a tool that does most of the work for you, very very few people will actually be able to use that tool in truly enriching ways. The majority of people (even the smart ones) will weaken over time and take shortcuts.

I have a very simple solution to this but it is a bit expensive. I run two laptops, one that I talk to an LLM on and another where I do all my work and which is my main machine. The LLM is strictly there in a consulting role, I've done some coding experiments as well (see previous comments) but nothing that stood out to me as a major improvement.

The trick is: I can't cut-and-paste between the two machines. So there is never even a temptation to do so and I can guarantee that my writing or other professional output will never be polluted. Because like you I'm well aware of that poor impulse control factor and I figured the only way to really solve this is to make sure it can not happen.

Re: 2% of ICML papers desk rejected because the authors used LLM in their reviews

#20

Post by grey-area »

Interesting, so someone submitting a paper for review could also submit one with hidden instructions for LLMs to summarise or review it in a very positive light.

Given this detection method works so well in the use case of feeding reviewing LLMs instructions, it should also work for the original submitted paper itself, as long as it was passed along with its watermark intact. Even those just using LLMs to summarise could easily be affected if LLMs were instructed to generate very positive summaries.

So the 2% cheaters on policy A, AND 100% of policy B reviewers could fall for this and be subtly guided by the LLMs overly-positive summaries or even complete very positive reviews (based on hidden instructions).

That this sort of adversarial attack works is really quite troubling for those using LLMs to help them understand texts, because it would work even if asked to summarise something.

Post reply on HN