Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Updated practice for review articles and position papers in ArXiv CS category

blog.arxiv.org

Post reply on HN
Print view View original

81–90 of 250 posts

Re: Updated practice for review articles and position papers in ArXiv CS category

#81

Post by socksy »

nradov wrote: post #77
jfengel wrote: post #42

Earlier quoted context omitted.

I had been kinda hoping for a web-of-trust system to replace peer review. Anyone can endorse an article. You can decide which endorsers you trust, and do some network math to find what you think is reading. With hashes and signatures and all that rot. Not as gate-keepy as journals and not as anarchic as purely open publishing. Should be cheap, too.

An endorsement system would have to be finer grained than a whole article. Mark specific sections that you agree or disagree with, along with comments.

I mean if you skip the traditional publishing gates, you could in theory endorse articles that specifically bring out sections from other articles that you agree or disagree with. Would be a different form of article

Re: Updated practice for review articles and position papers in ArXiv CS category

#82

Post by yorwba »

nurettin wrote: post #72
jfengel wrote: post #42

Earlier quoted context omitted.

I had been kinda hoping for a web-of-trust system to replace peer review. Anyone can endorse an article. You can decide which endorsers you trust, and do some network math to find what you think is reading. With hashes and signatures and all that rot. Not as gate-keepy as journals and not as anarchic as purely open publishing. Should be cheap, too.

What prevents you from creating an island of fake endorsers?

Unless you can be fooled into trusting a fake endorser, that island might just as well not exist.

Re: Updated practice for review articles and position papers in ArXiv CS category

#83

Post by zackmorris »

tasuki wrote: post #59
zackmorris wrote: post #54

I always figured if I wrote a paper, the peer review would be public scrutiny. As in, it would have revolutionary (as opposed to evolutionary) innovations that disrupt the status quo. I don't see how blocking that kind of paper from arXiv helps hacker culture in any way, so I oppose their decision. They should solve the real problem of obtaining more funding and volunteers so that they can take on the increased volum…

That paper wouldn't be blocked. Have you read the thing?

Before being considered for submission to arXiv’s CS category, review articles and position papers must now be accepted at a journal or a conference and complete successful peer review.

Huh, I guess it's only a subset of papers, not all of them. My brain doesn't work that way, because I don't like assigning custom rules for special cases (edit: because I usually view that as a form of discrimination). So sometimes I have a blind spot around the realities of a problem that someone is facing, that don't have much to do with its idealization.

What I mean is, I don't know that it's up to arXiv to determine what a "review article and position paper" is. Because of that, they must let all papers through, or have all papers face the same review standards.

When I see someone getting their fingers into something, like muddying/dithering concepts, shifting focus to something other than the crux of an argument (or using bad faith arguments, etc), I view it as corruption. It's a means for minority forces to insert their will over the majority. In this case, by potentially blocking meaningful work from reaching the public eye on a technicality.

So I admit that I was wrong to jump to conclusions. But I don't know that I was wrong in principle or spirit.

Re: Updated practice for review articles and position papers in ArXiv CS category

#84

Post by an0malous »

bob1029 wrote: post #28

> The advent of large language models have made this type of content relatively easy to churn out on demand, and the majority of the review articles we receive are little more than annotated bibliographies, with no substantial discussion of open research issues. I have to agree with their justification. Since "Attention Is All You Need" (2017) I have seen maybe four papers with similar impact in the AI/ML space. The…

Isn’t the signal to noise problem what journals are supposed to be for? I thought arxiv was supposed to just be a record keeper, to make it easy to share papers and preprints.

Re: Updated practice for review articles and position papers in ArXiv CS category

#85

Post by an0malous »

Why not just reject papers authored by LLMs and ban accounts that are caught? arXiv’s management has become really questionable lately, it’s like they’re trying to become a prestigious journal and are becoming the problem they were trying to solve in the first place

Re: Updated practice for review articles and position papers in ArXiv CS category

#87

Post by skopje »

DalasNoin wrote: post #10

it's clearly not sutainable to have the main website hosting CS articles not having any reviews or restrictions. (Except for the initial invite system) There were 26k submission in october: https://arxiv.org/stats/monthly_submissions Asking for a small amount of money would probably help. Issue with requiring peer reviewed journals or conferences is the severe lag, takes a long time and part of the advantage of arxiv…

I think it worked well for metafilter: $1/1euro one-time charge to join. But that's probably worth it to spam Arxiv with junk.

Re: Updated practice for review articles and position papers in ArXiv CS category

#88

Post by catlifeonmars »

an0malous wrote: post #85

Why not just reject papers authored by LLMs and ban accounts that are caught? arXiv’s management has become really questionable lately, it’s like they’re trying to become a prestigious journal and are becoming the problem they were trying to solve in the first place

It’s articles (not papers) _about_ LLMs that are the problem, not papers written _by_ LLMs (although I imagine they are not mutually exclusive). Title is ambiguous.

Re: Updated practice for review articles and position papers in ArXiv CS category

#89

Post by efitz »

There is a general problem with rewarding people for the volume of stuff they create, rather than the quality.

If you incentivize researchers to publish papers, individuals will find ways to game the system, meeting the minimum quality bar, while taking the least effort to create the most papers and thereby receive the greatest reward.

Similarly, if you reward content creators based on views, you will get view maximization behaviors. If you reward ad placement based on impressions, you will see gaming for impressions.

Bad metrics or bad rewards cause bad behavior.

We see this over and over because the reward issuers are designing systems to optimize for their upstream metrics.

Put differently, the online world is optimized for algorithms, not humans.

Re: Updated practice for review articles and position papers in ArXiv CS category

#90

Post by tarruda »

an0malous wrote: post #85

Why not just reject papers authored by LLMs and ban accounts that are caught? arXiv’s management has become really questionable lately, it’s like they’re trying to become a prestigious journal and are becoming the problem they were trying to solve in the first place

> Why not just reject papers authored by LLMs and ban accounts that are caught?

Are you saying that there's an automated method for reliably verifying that something was created by an LLM?

Post reply on HN