Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Major AI conference flooded with peer reviews written by AI

nature.com

Post reply on HN
Print view View original

101–110 of 140 posts

Re: Major AI conference flooded with peer reviews written by AI

#101

Post by isoprophlex »

itkovian_ wrote: post #41

Whether it’s actually 20% or not doesn’t matter, everyone is aware the signal of the top confs is in freefall. There are also rings of reviewer fraud going on where groups of people in these niche areas all get assigned their own papers and recommend acceptance and in many cases the AC is part of this as well. Am not saying this is common but it is occurring. It feels as if every layer of society is in maximum extrac…

If the Zucc has a weird day he starts dropping 10-100M salary packages in order to poach AI researchers. No wonder the game is getting rigged up the butthole.

Re: Major AI conference flooded with peer reviews written by AI

#102

Post by Loughla »

immibis wrote: post #91
h00kwurm wrote: post #84

Earlier quoted context omitted.

I don’t think it’s an extrapolation from the ML community into other industries. This evolution of society is objectively happening - artisanship, care for the work beyond capital gain, and commitment to depth in a focused category - are diminishing and harder to find qualities. I’d probably label it related to capital and material social economics. It’s perhaps more unfair and unjustified to not recognize this as a…

Just yesterday I saw this YouTube rant from someone called Jaiden Animations, about how everything is just shit now. https://www.youtube.com/watch?v=NBZv0_MImIY She opens with an example of a bank. She walked in and asked for a debit card. The teller told her to take a seat. 30 minutes later, the teller told her the bank doesn't issue debit cards. Firstly, what kind of bank doesn't issue debit cards, and secondly, wh…

It's because people are commodities now. Human resources exists to manage the shuffle between warm bodies.

It's back to OP's point. There's no such thing as professions now. Just jobs. We put them on and off like hats. With that churn comes lack of institutional knowledge and a rule set handed down from the C Suite for front line employees completely detached from the front line work.

Enshitification run rampant.

Re: Major AI conference flooded with peer reviews written by AI

#103

Post by Majromax »

ThrowawayTestr wrote: post #100
maxspero wrote: post #49

Earlier quoted context omitted.

I am not sure if you are familiar with Pangram (co-founder here) but we are a group of research scientists who have made significant progress in this problem space. If your mental model of AI detectors is still GPTZero or the ones that say the declaration of independence is AI, then you probably haven't seen how much better they've gotten. This paper by economists from the University of Chicago economists found zero…

Are you concerned with your product being used to improve AI to be less detectable?

> Are you concerned with your product being used to improve AI to be less detectable?

The big AI providers don't have any obvious incentive to do this. If it happens 'naturally' in the pursuit of quality then sure, but explicitly training for stealth is a brand concern in the same way that offering a fully uncensored model would be.

Smaller providers might do this (again in the same way they now offer uncensored models), but they occupy a miniscule fraction of the market and will be a generation or two behind the leaders.

Re: Major AI conference flooded with peer reviews written by AI

#105

Post by maleldil »

jay_kyburz wrote: post #79
ZeroConcerns wrote: post #12

Earlier quoted context omitted.

What I'm advocating is a "downvote (or ignore) and move on" attitude, as opposed to "I'm going to post about this" stance. Because, similar to "your color scheme is not a11y-friendly" or "you're posting affiliatate-links" or "this is effectively a paywall", there is zero chance of a productive conversation sprouting from that.

what is a11y. Can we just write words out please.

Accessibility. It's both a very common abbreviation and very easy to search for.

Re: Major AI conference flooded with peer reviews written by AI

#106

Post by radarsat1 »

Maybe what they should do in the future is just automatically provide AI reviews to all papers and state that the work of the reviewers is to correct any problems or fill details that were missed. That would encourage manual review of the AI's work and would also allow authors to predict what kind of feedback they'll get in a structured way. (eg say the standard prompt used was made public so authors could optimize their submission for the initial automatic review, forcing the human reviewer to fill in the gaps)

ok of course the human reviewers could still use AI here but then so could the authors, ad infinitum..

Re: Major AI conference flooded with peer reviews written by AI

#107

Post by sfink »

jay_kyburz wrote: post #79
ZeroConcerns wrote: post #12

Earlier quoted context omitted.

What I'm advocating is a "downvote (or ignore) and move on" attitude, as opposed to "I'm going to post about this" stance. Because, similar to "your color scheme is not a11y-friendly" or "you're posting affiliatate-links" or "this is effectively a paywall", there is zero chance of a productive conversation sprouting from that.

what is a11y. Can we just write words out please.

n0o

Re: Major AI conference flooded with peer reviews written by AI

#108

Post by ThrowawayTestr »

Majromax wrote: post #103
ThrowawayTestr wrote: post #100

Earlier quoted context omitted.

Are you concerned with your product being used to improve AI to be less detectable?

> Are you concerned with your product being used to improve AI to be less detectable? The big AI providers don't have any obvious incentive to do this. If it happens 'naturally' in the pursuit of quality then sure, but explicitly training for stealth is a brand concern in the same way that offering a fully uncensored model would be. Smaller providers might do this (again in the same way they now offer uncensored mode…

They don't have an incentive to make their AIs better? If your product can genuinely detect AI writing, of course they would use it to make their models sound more human. The biggest criticism of AI right now is how robotic and samey it sounds.

Re: Major AI conference flooded with peer reviews written by AI

#109

Post by conartist6 »

Herring wrote: post #78
conartist6 wrote: post #30

Earlier quoted context omitted.

LOL. So basically the correct sequence of events is: 1. The scientist does the work, putting their own biases and shortcomings into it 2. The reviewer runs AI, generating something that looks plausibly like review of the work but represents the view of a sociopath without integrity, morals, logic, or any consequences for making shit up instead of finding out. 3. The scientist works to determine how much of the review…

Don't kid yourself, all those steps have AI heavily involved in them. And that's not necessarily a bad thing. If I set up RAG correctly, then tell the AI to generate K samples, then spend time to pick out the best one, that's still significant human input, and likely very good output too. It's just invisible what the human did. And as models get better, the necessary K will become smaller....

That's a strategy for producing maximally convincing BS content, but the scientific method was absent. I'm sure it was an oversight... ; )

Re: Major AI conference flooded with peer reviews written by AI

#110

Post by iainctduncan »

JohnCClarke wrote: post #19

The question is not are the reviews AI generated. The question is are the reviews accurate?

No.. that is not the question.

This is a conference purporting to do PEER review. No matter how good the AI, it's not a peer review.

Post reply on HN