Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Updated practice for review articles and position papers in ArXiv CS category

blog.arxiv.org

Post reply on HN
Print view View original

101–110 of 250 posts

Re: Updated practice for review articles and position papers in ArXiv CS category

#101

Post by beloch »

A better policy might be for arXiv to do the following:

1. Require LLM produced papers to be attributed to the relevant LLM and not the person who wrote the prompt.

2. Treat submissions that misrepresent authorship as plagiarism. Remove the article, but leave an entry for it so that there is a clear indication that the author engaged in an act of plagiarism.

Review papers are valuable. Writing one is a great way to gain, or deepen, mastery over a field. It forces you to branch out and fully assimilate papers that you may have only skimmed, and then place them in their proper context. Reading quality review papers is also valuable. They're a great way for people new to a field to get up to speed and they can bring things that were missed to the fore, even for veterans of the field.

While the current generation of AI does a poor job of judging significance and highlighting what is actually important, they could improve in the future. However, there's no need for arXiv to accept hundreds of review papers written by the same model on the same field, and readers certainly don't want to sift through them all.

Clearly marking AI submissions and removing credit from the prompters would adequately future-proof things for when, and if, AI can produce high quality review papers. Clearly marking authors who engage in plagiarism as plagiarists will, hopefully, remove most of the motivation to spam arXiv with AI slop that is misrepresented as the work of humans.

My only concern would be for the cost to arXiv of dealing with the inevitable lawsuits. The policy arXiv has chosen is worse for science, but is less likely to get them sued by butt-hurt plagiarists or the very occasional false positive.

Re: Updated practice for review articles and position papers in ArXiv CS category

#102

Post by zerocrates »

catlifeonmars wrote: post #94
thomascountz wrote: post #2

The HN submission title is incorrect. > Before being considered for submission to arXiv’s CS category, review articles and position papers must now be accepted at a journal or a conference and complete successful peer review. Edit: original title was "arXiv No Longer Accepts Computer Science Position or Review Papers Due to LLMs"

Agree. Additionally, original title, "arXiv No Longer Accepts Computer Science Position or Review Papers Due to LLMs" is ambiguous. “Due to LLMs” is being interpreted as articles written by LLMs, which is not accurate.

No, the post is definitely complaining about articles written by LLMs:

"In the past few years, arXiv has been flooded with papers. Generative AI / large language models have added to this flood by making papers – especially papers not introducing new research results – fast and easy to write."

"Fast forward to present day – submissions to arXiv in general have risen dramatically, and we now receive hundreds of review articles every month. The advent of large language models have made this type of content relatively easy to churn out on demand, and the majority of the review articles we receive are little more than annotated bibliographies, with no substantial discussion of open research issues."

Surely a lot of them are also about LLMs: LLMs are the hot computing topic and where all the money and attention is, and they're also used heavily in the field. So that could at least partially account for why this policy is for CS papers only, but the announcement's rationale is about LLMs as producing the papers, not as their subject.

Re: Updated practice for review articles and position papers in ArXiv CS category

#103

Post by SyrupThinker »

hermannj314 wrote: post #91
amelius wrote: post #8

Maybe it's time for a reputation system. E.g. every author publishes a public PGP key along with their work. Not sure about the details but this is about CS, so I'm sure they will figure something out.

I didn't agree with this idea, but then I looked at how much HN karma you have and now I think that maybe this is a good idea.

Ignoring the actual proposal or user, just looking at karma is probably a pretty terrible metric. High karma accounts tend to just interact more frequently, for long periods of time. Often with less nuanced takes, that just play into what is likely to be popular within a thread. Having a Userscript that just places the karma and comment count next to a username is pretty eye opening.

Re: Updated practice for review articles and position papers in ArXiv CS category

#104

Post by JumpCrisscross »

yorwba wrote: post #97
JumpCrisscross wrote: post #96

Earlier quoted context omitted.

> Unless you can be fooled into trusting a fake endorser Wouldn’t most people subscribe to a default set of trusted citers?

If there's a default (I don't think there necessarily has to be one) there has to be somebody who decides what the default is. If most people trust them, that person is either very trustworthy or people just don't care very much.

> there has to be somebody who decides what the default is

Sure. This happens with ad blockers, for example. I imagine Elsevier or Wikipedia would wind up creating these lists. And then you’d have the same incentives as you have now for fooling that authority.

> or people just don't care very much

This is my hypothesis. If you’re an expert, you have your web of trust. If you’re not, it isn’t that hard to start from a source of repute.

Re: Updated practice for review articles and position papers in ArXiv CS category

#105

Post by uniqueuid »

eesmith wrote: post #92
uniqueuid wrote: post #79

Earlier quoted context omitted.

I got that suggestion recently talking to a colleague from a prestigious university. Her suggestion was simple: Kick out all non-ivy league and most international researchers. Then you have a working reputation system. Make of that what you will ...

Ahh, your colleague wants a higher concentration of "that comet might be an interstellar spacecraft" articles.

If your goal is exclusively reducing strain of overloaded editors, then that's just a side effect that you might tolerate :)

Re: Updated practice for review articles and position papers in ArXiv CS category

#106

Post by JadeNB »

Sharlin wrote: post #5

So what they no longer accept is preprints (or rejects…) It’s of course a pretty big deal given that arXiv is all about preprints. And an accepted journal paper presumably cannot be submitted to arXiv anyway unless it’s an open journal.

> And an accepted journal paper presumably cannot be submitted to arXiv anyway unless it’s an open journal.

Why not? I don't know about in CS, but, in math, it's increasingly common for authors to have the option to retain the copyright to their work.

Re: Updated practice for review articles and position papers in ArXiv CS category

#107

Post by [deleted] »

zerocrates wrote: post #102
catlifeonmars wrote: post #94

Earlier quoted context omitted.

Agree. Additionally, original title, "arXiv No Longer Accepts Computer Science Position or Review Papers Due to LLMs" is ambiguous. “Due to LLMs” is being interpreted as articles written by LLMs, which is not accurate.

No, the post is definitely complaining about articles written by LLMs: "In the past few years, arXiv has been flooded with papers. Generative AI / large language models have added to this flood by making papers – especially papers not introducing new research results – fast and easy to write." "Fast forward to present day – submissions to arXiv in general have risen dramatically, and we now receive hundreds of review…

[deleted]

Re: Updated practice for review articles and position papers in ArXiv CS category

#108

Post by lambdaone »

raddan wrote: post #67
jfengel wrote: post #42

Earlier quoted context omitted.

I had been kinda hoping for a web-of-trust system to replace peer review. Anyone can endorse an article. You can decide which endorsers you trust, and do some network math to find what you think is reading. With hashes and signatures and all that rot. Not as gate-keepy as journals and not as anarchic as purely open publishing. Should be cheap, too.

The problem with an endorsement scheme is citation rings, ie groups of people who artificially inflate the perceived value of some line of work by citing each other. This is a problem even now, but it is kept in check by the fact that authors do not usually have any control over who reviews their paper. Indeed, in my area, reviews are double blind, and despite claims that “you can tell who wrote this anyway” research…

I would have thought that those participants who are published in peer-reviewed journals could be be used as a trust anchor - see, for example, the Advogato algorithm as an example of a somewhat bad-faith-resistant metric for this purpose: https://web.archive.org/web/20170628063224/http://www.advoga...

Re: Updated practice for review articles and position papers in ArXiv CS category

#109

Post by dimava »

thomascountz wrote: post #2

The HN submission title is incorrect. > Before being considered for submission to arXiv’s CS category, review articles and position papers must now be accepted at a journal or a conference and complete successful peer review. Edit: original title was "arXiv No Longer Accepts Computer Science Position or Review Papers Due to LLMs"

refined title:

ArXiv CS requires peer review for surveys amid flood of AI-written ones

- nothing happened to preprints

- "summarization" articles always required it, they are just pointing at it out loud

Re: Updated practice for review articles and position papers in ArXiv CS category

#110

Post by pj_mukh »

Sharlin wrote: post #5

So what they no longer accept is preprints (or rejects…) It’s of course a pretty big deal given that arXiv is all about preprints. And an accepted journal paper presumably cannot be submitted to arXiv anyway unless it’s an open journal.

On a Sidenote: I’d a love a list of CLOSED journals and conferences to avoid like the plague.
Post reply on HN