Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

2% of ICML papers desk rejected because the authors used LLM in their reviews

blog.icml.cc

Post reply on HN
Print view View original

81–90 of 172 posts

Re: 2% of ICML papers desk rejected because the authors used LLM in their reviews

#81

Post by bonoboTP »

quinndupont wrote: post #38

How is nobody considering the broader political economy of scholarly publications and reviews? These are UNPAID reviews! Sure, maybe ICML isn’t Elsevier, but they are cousins to the socially parasitic and exploitative companies, at the very least. Hiding behind a false “choice” to not use AI or basically not use AI isn’t an appropriate proposal. This is crooked and shameful. We should boycott ICML except we can’t bec…

Your job as an academic is to disseminate your research and engage with the research community through service such as reviews, talks etc. It's part of the job, and people get a salary as university employees or company employees for this.

ML conferences aren't for profit ventures. If you submit papers and expect others to review it, you should reciprocate as well.

Re: 2% of ICML papers desk rejected because the authors used LLM in their reviews

#82

Post by everdrive »

jacquesm wrote: post #19
everdrive wrote: post #6

Earlier quoted context omitted.

Generally speaking people have worse impulse control than they believe they do. Once you give a tool that does most of the work for you, very very few people will actually be able to use that tool in truly enriching ways. The majority of people (even the smart ones) will weaken over time and take shortcuts.

I have a very simple solution to this but it is a bit expensive. I run two laptops, one that I talk to an LLM on and another where I do all my work and which is my main machine. The LLM is strictly there in a consulting role, I've done some coding experiments as well (see previous comments) but nothing that stood out to me as a major improvement. The trick is: I can't cut-and-paste between the two machines. So there…

This is a nice solution, but I think it speaks to just how enticing the problem is. This is the sort of tactic someone with a gambling addiction would employ. I don't say that to be rude to you: I've had to do similar things with regard to addicting infinite-scroll internet sites, and I definitely give in more than I'd like.

Re: 2% of ICML papers desk rejected because the authors used LLM in their reviews

#83

Post by everdrive »

retsibsi wrote: post #33
everdrive wrote: post #6

Earlier quoted context omitted.

Generally speaking people have worse impulse control than they believe they do. Once you give a tool that does most of the work for you, very very few people will actually be able to use that tool in truly enriching ways. The majority of people (even the smart ones) will weaken over time and take shortcuts.

I think you're framing this behaviour too generously. Laziness is one thing, lack of integrity is another, and this seems to be a straightforward case of cheating and lying.

I think it's just numbers. When one person errs it's a fault of character. When most people err, we call it a systematic fault. Why are most people overweight for the first time in history? Do most people lack the good character to restrict their diet? You could argue yes, however appeals to character won't actually solve the problem.

Re: 2% of ICML papers desk rejected because the authors used LLM in their reviews

#84

Post by auggierose »

causalityltd wrote: post #62

The declaration of no-LLM was done for social prestige or maybe self-deception of self-sufficiency like "I don't need LLM". And when it was time to do the actual work, the dependency kicked in like drugs. A lesson for all of us with LLMs in our workflow.

The declaration of no-LLM was done so you are not judged yourself by an LLM.

Re: 2% of ICML papers desk rejected because the authors used LLM in their reviews

#85

Post by jvanderbot »

mikkupikku wrote: post #7
bonoboTP wrote: post #5

To be clear, as the article says, these authors were offered a choice and agreed to be on the "no LLMs allowed" policy. And detection was not done with some snake oil "AI detector" but by invisible prompt injection in the paper pdf, instructing LLMs to put TWO long phrases into the review. They then detected LLM use through checking if both phrases appear in the review. This did not detect grammar checks and touchups…

In that case, I hope these frauds have been banned for life.

I'm not sure what experience anyone in this thread has with grad level research as a student/author, but I can assure you that heads roll over this kind of thing.

A professor's career is built on reputation, and that reputation is as strong as their students' (who do much of the "work" such as it is). It comes down to the professor, but this can be a career-ending moment for those students and I'm quite confident there were some very uncomfortable discussions as a result of this.

Re: 2% of ICML papers desk rejected because the authors used LLM in their reviews

#87

Post by sampo »

Took me a while understand. So, the same person has both submitted their research article to the conference, and also acted as a reviewer for articles submitted by other people.

And if they in their review work have agreed to a "no LLM use" policy, but got exposed using LLMs anyway, then their submitted research article is desk rejected. Theoretically, someone could have submitted a stellar research article, but because they didn't follow agreed policy when reviewing other people's work, then also their research contribution is not welcome.

(At first I understood that innocent author's articles would have been rejected just because they happened to go to a bad reviewer. But this is not the case.)

Re: 2% of ICML papers desk rejected because the authors used LLM in their reviews

#89

Post by echelon »

jvanderbot wrote: post #85
mikkupikku wrote: post #7

Earlier quoted context omitted.

In that case, I hope these frauds have been banned for life.

I'm not sure what experience anyone in this thread has with grad level research as a student/author, but I can assure you that heads roll over this kind of thing. A professor's career is built on reputation, and that reputation is as strong as their students' (who do much of the "work" such as it is). It comes down to the professor, but this can be a career-ending moment for those students and I'm quite confident the…

[flagged]

Re: 2% of ICML papers desk rejected because the authors used LLM in their reviews

#90

Post by sampo »

maleldil wrote: post #70
noduerme wrote: post #57

Earlier quoted context omitted.

Enforcement without consequences just wears down the people who are supposed to enforce it.

GP suggested a life ban. Maybe suspend for 6 months instead? That's a long time without publishing in the current publish-or-perish academia.

> Maybe suspend for 6 months instead?

Suspend for 6 months from a conference that is held yearly?

Post reply on HN