Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

2% of ICML papers desk rejected because the authors used LLM in their reviews

blog.icml.cc

Post reply on HN
Print view View original

131–140 of 172 posts

Re: 2% of ICML papers desk rejected because the authors used LLM in their reviews

#131

Post by causalityltd »

auggierose wrote: post #127
causalityltd wrote: post #123

Earlier quoted context omitted.

Is this written in the linked article? Or the info is from other places online? Because I didn't see this. Article seems to say that this choice was given just for review (how you will review not how you will get reviewed) and the consequence of getting caught, their paper being rejected, was a punishment, not the original trade-off or motivation for choosing option A. Happy to be corrected.

It is implied in the term "reciprocal reviewing". You are of course reviewing and being reviewed under the same policy. Happy to correct.

oh, now I got it, thanks.

I was being too generous :)

It fits though, quite funnily: They did not want LLM near their own papers because they could not have imagined injecting prompts to get a good review and that's the same lack-of-awareness (i guess you could say 'skill issue') which made them not look for prompt injections in the first place.

If i wanted to extend the joke further, injecting prompts into your own pdf to get good reviews by reviewers using LLMs is actually work. Skill and work. And if they had that, they wouldn't be in this soup.

I'm sorry if I am the only one laughing, but I am.

Couldn't game the system the accurate way so ... got caught gaming it the lazy way!

I do feel sorry for them, I do, they must have worked hard on their papers, but this is funny. Thanks.

Re: 2% of ICML papers desk rejected because the authors used LLM in their reviews

#132

Post by sieste »

armchairhacker wrote: post #27
CoastalCoder wrote: post #22

Earlier quoted context omitted.

This line of reasoning interests me because it seems to arise in other contexts as well. Do very harsh punishments significantly reduce future occurrences of the offense in question? I've heard opponents of the death penalty argue that it's generally not the case. E.g., because often the criminals aren't reasoning in terms that factor in the death penalty. On the other hand (and perhaps I'm misinformed), I've heard t…

Is the death penalty scarier than life in prison?

I'm not sure it was meant that way, but nice metaphor. For some students "academic death" might really be better than a life of being trapped in a system that they can only navigate by cheating.

Re: 2% of ICML papers desk rejected because the authors used LLM in their reviews

#133

Post by bluefirebrand »

linkregister wrote: post #16
mikkupikku wrote: post #11

Earlier quoted context omitted.

If this is tolerated, it sends exactly the wrong kind of message. The students, if they are, should be banned for life. Let them serve as an example for myriads of future students, this will be a better outcome in the long run. This didn't trip for people who were merely bouncing ideas off a LLM, they caught people who copy and pasted straight from their LLM.

It's not a fully consensus view, but a majority of sociologists agree that high severity deterrence has limited effectiveness against crime. Instead, certainty of enforcement is the most salient factor.

The point of a punishment is not solely to deter future crimes, it's also to actually punish the present crime though

For instance jail time is not *just a deterrence, it's physically preventing someone from committing more crimes against the public

Re: 2% of ICML papers desk rejected because the authors used LLM in their reviews

#135

Post by causalityltd »

iso1631 wrote: post #65
causalityltd wrote: post #62

The declaration of no-LLM was done for social prestige or maybe self-deception of self-sufficiency like "I don't need LLM". And when it was time to do the actual work, the dependency kicked in like drugs. A lesson for all of us with LLMs in our workflow.

Sure I use LLMs in my workflows. I use a calculator too. I can divide 98,324,672,722 by 161,024 by hand. At least I used to be able to do, but nobody is going to pay me to do that when a calculator exists. Likewise I can write a bunch of assembly (well OK I can't), but why would I do that when my compiler can convert my intention into it.

yeah but will you promise to do it by hand and then use a calculator?

Or will you have every intention to keep the promise but it would seem such a chore by now (cuz calculator is such a part of your workflow) that you would minimize the sanctity of your promise in your mind?

If yes, that's dependency, not usual use.

(I just learned that choosing no-LLM also meant no-LLM on their own papers, so I am less generous with motivations now. Wasn't dependency, just plain old self-interest. Thanks for your point.)

Re: 2% of ICML papers desk rejected because the authors used LLM in their reviews

#137

Post by hn_go_brrrrr »

lukan wrote: post #118
mikkupikku wrote: post #11

Earlier quoted context omitted.

If this is tolerated, it sends exactly the wrong kind of message. The students, if they are, should be banned for life. Let them serve as an example for myriads of future students, this will be a better outcome in the long run. This didn't trip for people who were merely bouncing ideas off a LLM, they caught people who copy and pasted straight from their LLM.

Between banning someone for life and not doing anything, there usually are some other options.

Like burned at the stake, tarred and feathered, drawn and quartered, etc.?

Re: 2% of ICML papers desk rejected because the authors used LLM in their reviews

#138

Post by fc417fc802 »

mikkupikku wrote: post #120
andybak wrote: post #94

Earlier quoted context omitted.

What is the aim of "moral instruction" if not deterrence? Surely it needs be instruction in pursuit of an outcome?

It makes honest people feel rewarded, valued and acknowledge. It teaches people who wish to follow the rules and conform to social norms what those norms are and where we actually draw the line in practice. https://en.wikipedia.org/wiki/Punishment#Education_and_denun...

Looked at slightly differently, given a split between high trust and low trust preventing conversions from high to low is similarly important to inducing conversions from low to high.

Re: 2% of ICML papers desk rejected because the authors used LLM in their reviews

#139

Post by 112233 »

tdeck wrote: post #121
pton_xd wrote: post #110

Earlier quoted context omitted.

> what's the problem? Read the article. They self-selected into the no-LLM group and then copy/pasted from an LLM. Not only dishonest but just not smart.

Reading the article is exhausting. If I can leave a comment just as well without reading the article, then what's the problem? If I got something wrong, other people will point it out. That's a more efficient use of my time. /s

Not to water down the snark, but isnt cause of situation described in the article the exact mentality you are mocking?

Re: 2% of ICML papers desk rejected because the authors used LLM in their reviews

#140

Post by bumby »

amoss wrote: post #97
echelon wrote: post #89

Earlier quoted context omitted.

[flagged]

This comment doesn't seem to fit the discussion at all? The discussion is not about humans using LLLs to write papers. It is about humans who agreed not to use LLVM in reviewing papers, then did exactly that.

In addition to being a reviewer, they also submitted their own research to this journal. So it leads to the question: if they were willing to cheat on the side of review with less incentive, why wouldn’t they cheat on the side that provides more incentives?

(Meaning, your career doesn’t get boosted much for reviewing papers, but much more so for publishing papers)

Post reply on HN