Live data from Hacker News

Updated practice for review articles and position papers in ArXiv CS category

blog.arxiv.org

171–180 of 250 posts

Re: Updated practice for review articles and position papers in ArXiv CS category

#171
post #81
post #77

Earlier quoted context omitted.

An endorsement system would have to be finer grained than a whole article. Mark specific sections that you agree or disagree with, along with comments.

I mean if you skip the traditional publishing gates, you could in theory endorse articles that specifically bring out sections from other articles that you agree or disagree with. Would be a different form of article

Sounds a bit like the trails in Memex (1945).

Re: Updated practice for review articles and position papers in ArXiv CS category

#172
post #42
post #8

Maybe it's time for a reputation system. E.g. every author publishes a public PGP key along with their work. Not sure about the details but this is about CS, so I'm sure they will figure something out.

I had been kinda hoping for a web-of-trust system to replace peer review. Anyone can endorse an article. You can decide which endorsers you trust, and do some network math to find what you think is reading. With hashes and signatures and all that rot. Not as gate-keepy as journals and not as anarchic as purely open publishing. Should be cheap, too.

Suggest writing up a scope or PRD for this and sharing it on GitHub.

Re: Updated practice for review articles and position papers in ArXiv CS category

#173
post #89

There is a general problem with rewarding people for the volume of stuff they create, rather than the quality. If you incentivize researchers to publish papers, individuals will find ways to game the system, meeting the minimum quality bar, while taking the least effort to create the most papers and thereby receive the greatest reward. Similarly, if you reward content creators based on views, you will get view maximi…

What would a system that rewards people for quality rather than volume look like?

How would an online world that is optimized for humans, not algorithms, look like?

Should content creators get paid?

Re: Updated practice for review articles and position papers in ArXiv CS category

#174
post #89

There is a general problem with rewarding people for the volume of stuff they create, rather than the quality. If you incentivize researchers to publish papers, individuals will find ways to game the system, meeting the minimum quality bar, while taking the least effort to create the most papers and thereby receive the greatest reward. Similarly, if you reward content creators based on views, you will get view maximi…

> rewarding people for the volume ... rather than the quality. I suspect this is a major part of the appeal of LLMs themselves. They produce lines very fast so it appears as if work is being done fast. But that's very hard to know because number of lines is actually a zero signal in code quality or even a commit. Which it's a bit insane already that we use number of lines and commits as measures in the first place. T…

I think you will discover that few organizations use the size or number of edits as a metric of effort. Instead, you might be judged by some measure of productivity (such as resolving issues). Fortunately, language agents are actually useful at coding, when applied judiciously.

Re: Updated practice for review articles and position papers in ArXiv CS category

#175
post #101

A better policy might be for arXiv to do the following: 1. Require LLM produced papers to be attributed to the relevant LLM and not the person who wrote the prompt. 2. Treat submissions that misrepresent authorship as plagiarism. Remove the article, but leave an entry for it so that there is a clear indication that the author engaged in an act of plagiarism. Review papers are valuable. Writing one is a great way to g…

That doesn't solve the problem they're trying to solve, which is their all-volunteer staff is being flooded with LLM slop and doesn't have the time to artistically moderate.

If you want to blame someone, blame all the people LARPing as AI researchers.

Re: Updated practice for review articles and position papers in ArXiv CS category

#176

Earlier quoted context omitted.

> And an accepted journal paper presumably cannot be submitted to arXiv anyway unless it’s an open journal. You cannot upload the journal’s version, but you can upload the text as accepted (so, the same content minus the formatting).

I suspect that any editorial changes that happened as part of the journal's acceptance process - unless they materially changed the content - would also have to be kept back as they would be part of the presentation of the paper (protected by copyright) rather than the facts of the research.

No, in practice we update the preprint accordingly.

Re: Updated practice for review articles and position papers in ArXiv CS category

#177
post #42
post #8

Maybe it's time for a reputation system. E.g. every author publishes a public PGP key along with their work. Not sure about the details but this is about CS, so I'm sure they will figure something out.

I had been kinda hoping for a web-of-trust system to replace peer review. Anyone can endorse an article. You can decide which endorsers you trust, and do some network math to find what you think is reading. With hashes and signatures and all that rot. Not as gate-keepy as journals and not as anarchic as purely open publishing. Should be cheap, too.

So trivial to game

Re: Updated practice for review articles and position papers in ArXiv CS category

#178

it's clearly not sutainable to have the main website hosting CS articles not having any reviews or restrictions. (Except for the initial invite system) There were 26k submission in october: https://arxiv.org/stats/monthly_submissions Asking for a small amount of money would probably help. Issue with requiring peer reviewed journals or conferences is the severe lag, takes a long time and part of the advantage of arxiv…

I'll add the amount should be enough to cover at least a cursory review. A full review would be better. I just don't want to price out small players. The papers could also be categorized as unreviewed, quick check, fully reviewed, or fully reproduced. They could pay for this to be done or verified. Then, we have a reputational problem to deal with on the reviewer side.

> I'll add the amount should be enough to cover at least a cursory review.

You might be vastly underestimating the cost of such a feature

Re: Updated practice for review articles and position papers in ArXiv CS category

#179
post #47

i would like to understand what people get, or think they get, out of putting a completely AI-generated survey paper on arXiv. Even if AI writes the paper for you, it's still kind of a pain in the ass to go through the submission process, get the LaTeX to compile on their servers, etc., there is a small cost to you. Why do this?

Published papers are part of the EB-1 visa rubric so huge value in getting your content into these indexes: "One specific criterion is the ‘authorship of scholarly articles in professional or major trade publications or other major media’. The quality and reputation of the publication outlet (e.g., impact factor of a journal, editorial review process) are important factors in the evaluation”

Is arXiv a major trade publication?

I've never seen arXiv papers counted towards your publications anywhere that the number of your publications are used as a metric. Is USCIS different?

Re: Updated practice for review articles and position papers in ArXiv CS category

#180
post #173
post #89

There is a general problem with rewarding people for the volume of stuff they create, rather than the quality. If you incentivize researchers to publish papers, individuals will find ways to game the system, meeting the minimum quality bar, while taking the least effort to create the most papers and thereby receive the greatest reward. Similarly, if you reward content creators based on views, you will get view maximi…

What would a system that rewards people for quality rather than volume look like? How would an online world that is optimized for humans, not algorithms, look like? Should content creators get paid?

> Should content creators get paid?

I don't think so. Youtube was a better place when it was just amateurs posting random shit.

Post reply on HN