Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Major AI conference flooded with peer reviews written by AI

nature.com

Post reply on HN
Print view View original

71–80 of 140 posts

Re: Major AI conference flooded with peer reviews written by AI

#71

Post by maxspero »

slashdave wrote: post #63
nkrisc wrote: post #24

> Pangram’s analysis revealed that around 21% of the ICLR peer reviews were fully AI-generated, and more than half contained signs of AI use. The findings were posted online by Pangram Labs. “People were suspicious, but they didn’t have any concrete proof,” says Spero. “Over the course of 12 hours, we wrote some code to parse out all of the text content from these paper submissions,” he adds. But what's the proof? Ho…

"proof" was an unfortunate phrase to use. However, a proper statistical analysis can be objective. And these kinds of tools are perfectly suited to such an analysis.

Yeah, Pangram does not provide any concrete proof, but it confirms many people's suspicions about their reviews. But it does flag reviews for a human to take a closer look and see if the review is flawed, low-effort, or contains major hallucinations.

Re: Major AI conference flooded with peer reviews written by AI

#72

Post by maxspero »

Fuzzwah wrote: post #68
maxspero wrote: post #40

Earlier quoted context omitted.

Co-founder of Pangram here. Our false positive rate is typically around 1 in 10,000. https://www.pangram.com/blog/all-about-false-positives-in-ai... . We also wanted to quantify our EditLens model's FPR on the same domain, so we ran all of ICLR's 2022 reviews. Of 10,202 reviews, Pangram marked 10,190 as fully human, 10 as lightly AI-edited, 1 as moderately AI-edited, 1 as heavily AI-edited, and none as fully AI-gener…

Give your final sentence a re-read there....

Thanks, fixed.

Re: Major AI conference flooded with peer reviews written by AI

#73

Post by Oarch »

maxspero wrote: post #58
nialse wrote: post #53

Earlier quoted context omitted.

Nothing points out that the benchmark is invalid like a zero false positive rate. Seemingly it is pre-2020 text vs a few models rework of texts. I can see this model fall apart in many real world scenarios. Yes, LLMs use strange language if left to their own devices and this can surely be detected. 0% false positive rate under all circumstances? Implausible.

Our benchmarks of public datasets put our FPR roughly around 1 in 10,000. https://www.pangram.com/blog/all-about-false-positives-in-ai... Find me a clean public dataset with no AI involvement and I will be happy to report Pangram's false positive rate on it.

I enjoyed this thoughtful write up. It's a vitally important area for good, transparent work to be done.

Re: Major AI conference flooded with peer reviews written by AI

#75

Post by lifthrasiir »

maxspero wrote: post #49
jampa wrote: post #6

While I think there's significant AI "offloading" in writing, the article's methodology relies on "AI-detectors," which reads like PR for Pangram. I don't need to explain why AI detectors are mostly bullshit and harmful for people who have never used LLMs. [1] 1: https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...

I am not sure if you are familiar with Pangram (co-founder here) but we are a group of research scientists who have made significant progress in this problem space. If your mental model of AI detectors is still GPTZero or the ones that say the declaration of independence is AI, then you probably haven't seen how much better they've gotten. This paper by economists from the University of Chicago economists found zero…

It is not wise to brag about your product when the GP is pointing out that the article "reads like PR for Pangram", no matter AI detectors are reliable or not.

Re: Major AI conference flooded with peer reviews written by AI

#76

Post by pixl97 »

maxspero wrote: post #69
rs186 wrote: post #59

Earlier quoted context omitted.

The response would be more helpful if it directly addresses the arguments in posts from that search result.

There are dozens of first generation AI detectors and they all suck. I'm not going to defend them. Most of them use perplexity based methods, which is a decent separators of AI and human text (80-90%) but has flaws that can't be overcome and high FPRs on ESL text. https://www.pangram.com/blog/why-perplexity-and-burstiness-f... Pangram is fundamentally different technology, it's a large deep learning based model that…

GAN.. Just feed the output of your algorithms back into the LLM while learning. At the end of the day the problem is impossible, but we're not there yet.

Re: Major AI conference flooded with peer reviews written by AI

#77

Post by hn_throwaway_99 »

AI slop has infiltrated so many areas. Check out this article that was on the front page of HN last week, "73% of AI startups are just prompt engineering", with hundreds of points and lots of comments arguing for or against: https://news.ycombinator.com/item?id=46024644

The problem is the entire article is made up. Sure, the author can trace client-side traffic, but the vast majority of start-ups would be making calls to LLMs in their backend (a sequence diagram in the article even points this out!!), where it would be untraceable. There is certainly no way the author can make a broad statement that he knows what's happening across hundreds of startups.

Yet lots of comments just taking these conclusions at face value. Worse, when other commenters and myself pointed out the blatant impossibility of the author's conclusion, got some responses just rehashing how the author said they "traced network traffic", even though that doesn't make any sense as they wouldn't have access to backends of these companies.

Re: Major AI conference flooded with peer reviews written by AI

#78

Post by Herring »

conartist6 wrote: post #30
JohnCClarke wrote: post #19

The question is not are the reviews AI generated. The question is are the reviews accurate?

LOL. So basically the correct sequence of events is: 1. The scientist does the work, putting their own biases and shortcomings into it 2. The reviewer runs AI, generating something that looks plausibly like review of the work but represents the view of a sociopath without integrity, morals, logic, or any consequences for making shit up instead of finding out. 3. The scientist works to determine how much of the review…

Don't kid yourself, all those steps have AI heavily involved in them.

And that's not necessarily a bad thing. If I set up RAG correctly, then tell the AI to generate K samples, then spend time to pick out the best one, that's still significant human input, and likely very good output too. It's just invisible what the human did.

And as models get better, the necessary K will become smaller....

Re: Major AI conference flooded with peer reviews written by AI

#79

Post by jay_kyburz »

ZeroConcerns wrote: post #12
stevemk14ebr wrote: post #9

Earlier quoted context omitted.

Banning calling out AI slop hardly seems like an improvement

What I'm advocating is a "downvote (or ignore) and move on" attitude, as opposed to "I'm going to post about this" stance. Because, similar to "your color scheme is not a11y-friendly" or "you're posting affiliatate-links" or "this is effectively a paywall", there is zero chance of a productive conversation sprouting from that.

what is a11y. Can we just write words out please.

Re: Major AI conference flooded with peer reviews written by AI

#80

Post by vladms »

maxspero wrote: post #71
slashdave wrote: post #63

Earlier quoted context omitted.

"proof" was an unfortunate phrase to use. However, a proper statistical analysis can be objective. And these kinds of tools are perfectly suited to such an analysis.

Yeah, Pangram does not provide any concrete proof, but it confirms many people's suspicions about their reviews. But it does flag reviews for a human to take a closer look and see if the review is flawed, low-effort, or contains major hallucinations.

Was there an analysis of flawed, low-effort reviews in similar conferences before generative AI models?

From what I remember, (long before generative AI) you would still occasionally get very crappy reviews (as author). When I participated (couple of times) to review committees, when there was a high variance between reviews the crappy reviews were rather easy to spot and eliminate.

Now it's not bad to detect crappy (or AI) reviews, but I wonder if it would change much the end result compared to other potential interventions.

Post reply on HN