Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

2% of ICML papers desk rejected because the authors used LLM in their reviews

blog.icml.cc

Post reply on HN
Print view View original

71–80 of 172 posts

Re: 2% of ICML papers desk rejected because the authors used LLM in their reviews

#71

Post by RHSeeger »

mikkupikku wrote: post #11
hodgehog11 wrote: post #9

Earlier quoted context omitted.

I was thinking this too, but I don't believe this is the case, and I feel like it would not be a good idea either. Most of these people are likely students; this should be a learning moment, but I don't think it is yet grounds for their entire academic career to be crippled by being unable to publish in a top-tier ML venue.

If this is tolerated, it sends exactly the wrong kind of message. The students, if they are, should be banned for life. Let them serve as an example for myriads of future students, this will be a better outcome in the long run. This didn't trip for people who were merely bouncing ideas off a LLM, they caught people who copy and pasted straight from their LLM.

It has been shown time and again that, for most people, teaching them to be better and giving second chances is more effective than using forever-punishment as a warning for others.

Re: 2% of ICML papers desk rejected because the authors used LLM in their reviews

#72

Post by maleldil »

anonymousDan wrote: post #59
hodgehog11 wrote: post #9

Earlier quoted context omitted.

I was thinking this too, but I don't believe this is the case, and I feel like it would not be a good idea either. Most of these people are likely students; this should be a learning moment, but I don't think it is yet grounds for their entire academic career to be crippled by being unable to publish in a top-tier ML venue.

ML reviewing is a total joke. Why do you have noob students reviewing a conference paper.

It's usually not "noob" students. Big conferences require reviewers to have at least one (usually more) published paper in major venues. For students, this usually means they went through the process of being the first author on a few papers.

Re: 2% of ICML papers desk rejected because the authors used LLM in their reviews

#73

Post by RHSeeger »

noduerme wrote: post #57
linkregister wrote: post #16

Earlier quoted context omitted.

It's not a fully consensus view, but a majority of sociologists agree that high severity deterrence has limited effectiveness against crime. Instead, certainty of enforcement is the most salient factor.

Enforcement without consequences just wears down the people who are supposed to enforce it.

There's a pretty large area between "no consequences" and "banned forever"

Re: 2% of ICML papers desk rejected because the authors used LLM in their reviews

#75

Post by bonoboTP »

anonymousDan wrote: post #59
hodgehog11 wrote: post #9

Earlier quoted context omitted.

I was thinking this too, but I don't believe this is the case, and I feel like it would not be a good idea either. Most of these people are likely students; this should be a learning moment, but I don't think it is yet grounds for their entire academic career to be crippled by being unable to publish in a top-tier ML venue.

ML reviewing is a total joke. Why do you have noob students reviewing a conference paper.

Because someone has to do it. Conference submissions have ballooned as the field itself has ballooned.

Whats your suggestion?

Re: 2% of ICML papers desk rejected because the authors used LLM in their reviews

#76

Post by jjgreen »

jacquesm wrote: post #36
jjgreen wrote: post #31

Earlier quoted context omitted.

You could ssh in to the "dirty" machine ... just sayin'

Yes, I could. But I've purposefully made linking the two quite hard.

People do similar things installing an app locking their phones so they don't spend all day on them, then "oh just this once", ...

Re: 2% of ICML papers desk rejected because the authors used LLM in their reviews

#77

Post by bonoboTP »

matusp wrote: post #61
bonoboTP wrote: post #12

Earlier quoted context omitted.

I'm not surprised at all. The ML research community isn't a community any more, it's turned into a dog-eat-dog low-trust fierce competition. So much more people, papers, churn, that everyone is just fending for themselves. Any moment that you charitably spend on community service can be felt as a moment you take away from the next project, jeopardizing the next paper, getting scooped, delaying your graduation, your c…

And the real punchline is that the deluge of papers barely matters, as the academic field is barely moving, and the most interesting innovations are happening on the product side.

I disagree with this. Usually the products are based on published research. This is not easily seen by the enthusiast power user base.

Of course it's only a small fraction of all papers that end up actually being used. Most are mainly about advancing careers and strengthening CVs.

Re: 2% of ICML papers desk rejected because the authors used LLM in their reviews

#78

Post by marcosdumay »

bonoboTP wrote: post #75
anonymousDan wrote: post #59

Earlier quoted context omitted.

ML reviewing is a total joke. Why do you have noob students reviewing a conference paper.

Because someone has to do it. Conference submissions have ballooned as the field itself has ballooned. Whats your suggestion?

It's better if nobody does it than to send it to the randomizer.

Re: 2% of ICML papers desk rejected because the authors used LLM in their reviews

#79

Post by bonoboTP »

jacquesm wrote: post #14

I keep spotting clear LLM 'tells' in text where I know the people on the other side believe they're 'getting away with it'. It is incredible at what levels of commerce people do this, and how they're prepared to risk their reputation by saving a few characters typed. It makes me wonder what they think they are getting paid for.

Based on my experience on HN, many people can't see the tells. They may pick up on a few meme things like emdashes or "delve" or "rich tapestry", but can't detect the general tone or cadence reliably.

So maybe those people are right and are getting away with it for most readers of it.

Re: 2% of ICML papers desk rejected because the authors used LLM in their reviews

#80

Post by nis0s »

bonoboTP wrote: post #77
matusp wrote: post #61

Earlier quoted context omitted.

And the real punchline is that the deluge of papers barely matters, as the academic field is barely moving, and the most interesting innovations are happening on the product side.

I disagree with this. Usually the products are based on published research. This is not easily seen by the enthusiast power user base. Of course it's only a small fraction of all papers that end up actually being used. Most are mainly about advancing careers and strengthening CVs.

One key reason you’re wrong is that many interesting things aren’t even getting published, they’re on the DL for years and eventually make it to public spheres and products.

Academia is just a daycare at this point, and many labs shouldn’t exists or get funding. The people who move the field aren’t necessarily the ones with the most citations, they’re usually hard at work in places that don’t publish at all.

Post reply on HN