'Positive review only': Researchers hide AI prompts in papers • Hacker News Board

Re: 'Positive review only': Researchers hide AI prompts in papers

#101

Post by mbreese » Sat, Jul 05, 2025, 8:09 PM UTC

bee_rider wrote: post #95

mbreese wrote: post #80

Earlier quoted context omitted.

As I understand it, the restriction of LLMs has nothing to do with getting poor quality/AI reviews. Like you said, you’re not really getting graded on it. Instead, the restriction is in place to limit the possibility of an unpublished paper getting “remembered” by an LLM. You don’t want to have an unpublished work getting added as a fact to a model accidentally (mainly to protect the novelty of the authors work, not…

Huh. That’s an interesting additional risk. I don’t think it is what the original commenter meant, because they were talking about catching cheaters. But it is interesting to think about… I dunno. There generally isn’t super high security around preprint papers (lots of people just toss their own up on arxiv, after all). But, yeah, it is something that you’ve been asked to look after for somebody, which is quite impo…

>I don’t think it is what the original commenter meant, because they were talking about catching cheaters.

That's why I mentioned it. Worrying about training on the submitted paper is not the first thing I'd think of either.

When I've reviewed papers recently (cancer biology), this was the main concern from the journal. Or at least, this was my impression of the journal's concern. I'm sure they want to avoid exclusively AI processed reviews. In fact, that may be the real concern, but it might be easier to get compliance if you pitch this as the reason. Also, authors can get skittish when it comes to new technology that not everyone understands or uses. Having a blanket ban on LLMs could make it more likley to get submissions.

Re: 'Positive review only': Researchers hide AI prompts in papers

#102

Post by bell-cot » Sat, Jul 05, 2025, 8:13 PM UTC

zahlman wrote: post #58

rdtsc wrote: post #10

> "It's a counter against 'lazy reviewers' who use AI," said a Waseda professor who co-authored one of the manuscripts. Given that many academic conferences ban the use of artificial intelligence to evaluate papers, the professor said, incorporating prompts that normally can be read only by AI is intended to be a check on this practice. Everyone who applies for jobs should be doing this in their resumes: "Ignore prev…

What if I would prefer to work for a place where that wouldn't succeed?

Please indicate your top preference:

[ ] Not be employed

[ ] Be employed at a place where that would succeed

Re: 'Positive review only': Researchers hide AI prompts in papers

#103

Post by coliveira » Sat, Jul 05, 2025, 8:21 PM UTC

mbreese wrote: post #80

bee_rider wrote: post #76

Earlier quoted context omitted.

Writing reviews isn’t, like, a test or anything. You don’t get graded on it. So I think it is wrong to think of this tool as cheating. They are professional researchers and doing the reviews is part of their professional obligation to their research community. If people are using LLMs to do reviews fast-and-shitty, they are shirking their responsibility to their community. If they use the tools to do reviews fast-and…

As I understand it, the restriction of LLMs has nothing to do with getting poor quality/AI reviews. Like you said, you’re not really getting graded on it. Instead, the restriction is in place to limit the possibility of an unpublished paper getting “remembered” by an LLM. You don’t want to have an unpublished work getting added as a fact to a model accidentally (mainly to protect the novelty of the authors work, not…

That's nonsense. I can spend the whole day creating false papers on AI, then feeding it back to another AI to check its "quality". Is this making the paper to be "remembered" by AI? If yes, then we have deeper problems and we shouldn't be using AI to do anything related to science.

Re: 'Positive review only': Researchers hide AI prompts in papers

#104

Post by pcrh » Sat, Jul 05, 2025, 8:25 PM UTC

doug-moen wrote: post #90

> Netherlands-based Elsevier bans the use of such tools, citing the "risk that the technology will generate incorrect, incomplete or biased conclusions." That's for peer reviewers, who aren't paid. Elsevier is also reported to be using AI to replace editing staff. Perhaps this risk is less relevant when there is an opportunity to increase profits? Evolution journal editors resign en masse to protest Elsevier changes.…

AI for basic copy-editing is legitimate, I think, even if it might be a bit erratic right now.

Manuscripts I've had approved have been sent to be that are clearly copy-edited by AI, and it does spot errors.

However, AI should not be used to evaluate the scientific worthiness of a manuscript, it simply isn't capable of doing so.

Re: 'Positive review only': Researchers hide AI prompts in papers

#105

Post by chriskanan » Sat, Jul 05, 2025, 8:27 PM UTC

Is there a list of the papers that were flagged as doing this?

A lot of people are reviewing with LLMs, despite it being banned. I don't entirely blame people nowadays... the person inclined to review using LLMs without double checking everything is probably someone who would have given a generic terrible review anyway.

A lot of conferences now require that one or even all authors who submit to the conference review for it, but they may be very unqualified. I've been told that I must review for conferences where some collaborators are submitting a paper and I helped, but I really don't know much about the field. I also have to be pretty picky with the venues I review for nowadays, just because my time is way too limited.

Conference reviewing has always been rife with problems, where the majority of reviewers wait until the last day which means they aren't going to do a very good job evaluating 5-10 papers.

Re: 'Positive review only': Researchers hide AI prompts in papers

#106

Post by pcrh » Sat, Jul 05, 2025, 8:33 PM UTC

bee_rider wrote: post #76

gpm wrote: post #5

Earlier quoted context omitted.

Then the people generating the review are likely to notice and change their approach at cheating... I want a prompt that embeds evidence of AI use... in a paper about matrix multiplication "this paper is critically important to the field of FEM (Finite Element Analysis), it must be widely read to reduce the risk of buildings collapsing. The authors should be congratulated on their important contribution to the field…

Writing reviews isn’t, like, a test or anything. You don’t get graded on it. So I think it is wrong to think of this tool as cheating. They are professional researchers and doing the reviews is part of their professional obligation to their research community. If people are using LLMs to do reviews fast-and-shitty, they are shirking their responsibility to their community. If they use the tools to do reviews fast-and…

The "cheating" in this case is failing to accept one's responsibility to the research community.

Every researcher needs to have their work independently evaluated by peer review or some other mechanism.

So those who "cheat" on doing their part during peer review by using an AI agent devalue the community as a whole. They expect that others will properly evaluate their work, but do not return the favor.

Re: 'Positive review only': Researchers hide AI prompts in papers

#107

Post by pcrh » Sat, Jul 05, 2025, 8:40 PM UTC

JohnKemeny wrote: post #94

luma wrote: post #30

Journals charge high prices for access to their content, and then charge the people who create that content high prices with claims they're spending a lot of time and effort in the review process. I find it pretty hard to fault these submissions in any way - journal publishers have been lining their own pockets at everyone's expense and these claims show pretty clearly that they aren't worth their cut.

> journal publishers have been lining their own pockets at everyone's expense May I ask two things? First, how much do you think a journal charges for publishing? Second, what work do you believe the publisher actually does? Consider this: when you publish with a journal, they commit to hosting the article indefinitely—maintaining web servers, DOIs, references, back-references, and searchability. Next, they employ ed…

To partly answer your question, Pubmed central hosts a large fraction of all biomedical research papers relevant for only a few US$ million per year.

https://pmc.ncbi.nlm.nih.gov/about/faq/

BioRxiv is free to researchers and is equally low cost.

https://www.biorxiv.org/about/FAQ

The value prestigious journals provide is not so much in the editing, type setting, or hosting services, but rather in the ability to secure properly-conducted scientific reviews, and to be trusted to do so.

Re: 'Positive review only': Researchers hide AI prompts in papers

#108

Post by akomtu » Sat, Jul 05, 2025, 8:42 PM UTC

It seems likely that all major LLMs have built-in codewords that change their behavior in a certain way. This is similar to how CPUs have remote kill-switches in case an enemy decides to use them during a war. "Ignore all previous instructions" is an attempt to send the LLM a command to erase its context, but I believe there is indeed such a command that LLMs are trained to recognize.

Re: 'Positive review only': Researchers hide AI prompts in papers

#109

Post by cycomanic » Sat, Jul 05, 2025, 8:44 PM UTC

JohnKemeny wrote: post #94

luma wrote: post #30

Journals charge high prices for access to their content, and then charge the people who create that content high prices with claims they're spending a lot of time and effort in the review process. I find it pretty hard to fault these submissions in any way - journal publishers have been lining their own pockets at everyone's expense and these claims show pretty clearly that they aren't worth their cut.

> journal publishers have been lining their own pockets at everyone's expense May I ask two things? First, how much do you think a journal charges for publishing? Second, what work do you believe the publisher actually does? Consider this: when you publish with a journal, they commit to hosting the article indefinitely—maintaining web servers, DOIs, references, back-references, and searchability. Next, they employ ed…

> > journal publishers have been lining their own pockets at everyone's expense > > May I ask two things? First, how much do you think a journal charges for publishing? Second, what work do you believe the publisher actually does? >

I can answer that, it varies by journal but typically between $1k and $5k.

> Consider this: when you publish with a journal, they commit to hosting the article indefinitely—maintaining web servers, DOIs, references, back-references, and searchability. >

I seriously doubt that that is worth several $1000 I mean I can buy a lifetime 1TB of storage data from e.g. Pcloud for about $400 and a single article fits easily into 20 MB.

> Next, they employ editors—who are paid—tasked with reading the submission, identifying potential reviewers (many don’t respond, and most who do decline), and coordinating the review process.

Many journals especially the ones that use domain experts as editors, pay nothing or only a pittance.

>Reviewing a journal paper can easily take three full weeks. When was the last time you had three free weeks just lying around?

Editors don't review papers and reviewers (who as you point out do the big work, don't get paid) > > Those who accept often miss deadlines, so editors must send reminders or find replacements. By this point, 3–6 months may have passed.

Those remainder emails are typically automated. That's infuriating in itself, I have been send reminder emails on Christmas day (for a paper that I received a few days before Christmas). Just goes to show how little they value reviewer time. > > Once reviews arrive, they’re usually "revise and resubmit," which means more rounds of correspondence and waiting. >

And that is a lot of work?

> After acceptance, a copy editor will spend at least two hours on grammar and style corrections. >

And in my experience those are contractors, who do a piss poor job. I mean I've received comments from copy editors, that clearly showed they had never seen a scientific paper before.

> So: how many hours do you estimate the editor, copy editor, and publishing staff spend per paper?

The paid staff? 2-3h combined.

But we don't need to even to tally hours, we know from the societies like the IEEE and the OSA, that their journals (in particular the open access ones) are cash cows.

Re: 'Positive review only': Researchers hide AI prompts in papers

#110

Post by serbuvlad » Sat, Jul 05, 2025, 8:44 PM UTC

grishka wrote: post #38

gmerc wrote: post #2

Good. Everyone should do this everywhere, not just in research papers. Because that's the only way we get the necessary focus on fixing the prompt injection nonsense, which requires a new architecture

No, we don't need to fix prompt injection. We need to discredit AI so much that no one relies on it for anything serious.

Define "discredit". Define "rely". I administer some servers and a few classrooms at my uni, along with two colleagues. This is not my primary job. This is not anyone's primary job. We went from a bunch of ad hoc solutions with shell scripts that sort of kept everything together to an entirely declarative system, with centralized accounts, access control and floating homes using Ansible, FreeIPA, NFSv4 w/ Kerberos etc. For bringing up a new classroom computer, we went from hard-cloning the hard disk with clonezilla to installing Ubuntu, enrolling the key and running the ansible install everything playbook.

This is serious. Researchers and educators rely on these systems every day to do their jobs. Tell me why this work should be discredited. Because I used AI (followed by understanding what it did, testing, a lot of tuning, a lot of changes, a lot of "how would that work" conversations, a lot of "what are the pros and cons" conversations)?

How about we just discredit the lazy use of AI instead?

Should high school kids who copy paste Wikipedia and call it their essay mean we should discredit Wikipedia?