Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Updated practice for review articles and position papers in ArXiv CS category

blog.arxiv.org

Post reply on HN
Print view View original

211–220 of 250 posts

Re: Updated practice for review articles and position papers in ArXiv CS category

#211

Post by noobermin »

miki123211 wrote: post #194
noobermin wrote: post #133

Earlier quoted context omitted.

Sure, just as long as we don't blame LLMs. Blame people, bad actors, systems of incentives, the gods, the devils, but never broach the fault of LLMs and their wide spread abuse.

LLMs are tools that make it easier to hack incentives, but you still need a person to decide that they'll use an LLM t do so. Blaming LLMs is unproductive. They are not going anywhere (especially since open source LLMs are so good.) If we want to achieve real change, we need to accept that they exist, understand how that changes the scientific landscape and our options to go from here.

everyone keeps claiming "they're here to stay" as if it's gospel. this constant drumbeat is rather tiresome and without much hard evidence.

Re: Updated practice for review articles and position papers in ArXiv CS category

#214

Post by epolanski »

efitz wrote: post #89

There is a general problem with rewarding people for the volume of stuff they create, rather than the quality. If you incentivize researchers to publish papers, individuals will find ways to game the system, meeting the minimum quality bar, while taking the least effort to create the most papers and thereby receive the greatest reward. Similarly, if you reward content creators based on views, you will get view maximi…

The prize in science is being cited/quoted, not publishing.

Sure, publishing on important papers has its weight, but not as much as getting cited.

Re: Updated practice for review articles and position papers in ArXiv CS category

#215

Post by js8 »

MangoToupe wrote: post #210
_jsmh wrote: post #173

Earlier quoted context omitted.

What would a system that rewards people for quality rather than volume look like? How would an online world that is optimized for humans, not algorithms, look like? Should content creators get paid?

Crazily, I think the easiest way is to remove any and all incentives, awards, finite funding, and allegedly merit-based positions. Allow anyone who wants to research to research. Natural recognition of peers seems to be the only way to my thinking. Of course this relies on a post-scarcity society so short of actually achieving communism we'll likely never see it happen.

You don't need postscarcity to do that. I was born in communist Czechoslovakia (my father was an academic). Government allocated jobs for academics and researchers, and they pretty much had tenure. So you could coast by being unproductive, or get by using your connections to the party members (the real currency in CSSR).

After 1989, most academics complained the system is not merit-based and practical (applied) enough. So we changed it to grants and publications metrics (modeled after the West). For a while, it worked.. until people found too much overbearing bureaucracy and some learned how to game the system again.

I would say, both systems have failure modes of a similar magnitude, although the first one is probably less hoops and less stress on each individual. (During communism, academia - if you could get there, especially technical sciences - was an oasis of freedom.)

Re: Updated practice for review articles and position papers in ArXiv CS category

#216

Post by PeterStuer »

epolanski wrote: post #214
efitz wrote: post #89

There is a general problem with rewarding people for the volume of stuff they create, rather than the quality. If you incentivize researchers to publish papers, individuals will find ways to game the system, meeting the minimum quality bar, while taking the least effort to create the most papers and thereby receive the greatest reward. Similarly, if you reward content creators based on views, you will get view maximi…

The prize in science is being cited/quoted, not publishing. Sure, publishing on important papers has its weight, but not as much as getting cited.

That might be the "prize" but the "bar" is most certainly in publish or perisch to work your way up the early academic carreer ladder. Every conference or workshop attendance needs a paper, regardless of wether you had any breakthrough. And early metrics are most often quantity based (at least 4 accepted journal articles), not citation based.

Re: Updated practice for review articles and position papers in ArXiv CS category

#217

Post by watwut »

hammock wrote: post #199
efitz wrote: post #89

There is a general problem with rewarding people for the volume of stuff they create, rather than the quality. If you incentivize researchers to publish papers, individuals will find ways to game the system, meeting the minimum quality bar, while taking the least effort to create the most papers and thereby receive the greatest reward. Similarly, if you reward content creators based on views, you will get view maximi…

> There is a general problem with rewarding people for the volume of stuff they create, rather than the quality. If you incentivize researchers to publish papers, individuals will find ways to game the system, I heard someone say something similar about the “homeless industrial complex” on a podcast recently. I think it was San Francisco that pays NGOs funds for homeless aid based on how many homeless people they ser…

Yeah, it is totally NGO that creates homelessness /s

Re: Updated practice for review articles and position papers in ArXiv CS category

#218

Post by jessriedel »

pbhjpbhj wrote: post #163
kergonath wrote: post #115

Earlier quoted context omitted.

> And an accepted journal paper presumably cannot be submitted to arXiv anyway unless it’s an open journal. You cannot upload the journal’s version, but you can upload the text as accepted (so, the same content minus the formatting).

I suspect that any editorial changes that happened as part of the journal's acceptance process - unless they materially changed the content - would also have to be kept back as they would be part of the presentation of the paper (protected by copyright) rather than the facts of the research.

As an outsider that's a reasonable thing to suppose based on a plain reading of copyright law, but in practice it's not true. Researchers update their preprint based on changes requested by reviewers and editors all the time. It's never an issue.

Re: Updated practice for review articles and position papers in ArXiv CS category

#219

Post by latentsea »

noobermin wrote: post #211
miki123211 wrote: post #194

Earlier quoted context omitted.

LLMs are tools that make it easier to hack incentives, but you still need a person to decide that they'll use an LLM t do so. Blaming LLMs is unproductive. They are not going anywhere (especially since open source LLMs are so good.) If we want to achieve real change, we need to accept that they exist, understand how that changes the scientific landscape and our options to go from here.

everyone keeps claiming "they're here to stay" as if it's gospel. this constant drumbeat is rather tiresome and without much hard evidence.

What evidence do you need exactly?

I think such statements are likely projections of people's own unwillingness to part with such tools given their own personal perceived utility.

I, for one, wouldn't give up LLMs. Too useful to me personally. So, I will always seek them out.

Re: Updated practice for review articles and position papers in ArXiv CS category

#220

Post by gus_massa »

noobermin wrote: post #211
miki123211 wrote: post #194

Earlier quoted context omitted.

LLMs are tools that make it easier to hack incentives, but you still need a person to decide that they'll use an LLM t do so. Blaming LLMs is unproductive. They are not going anywhere (especially since open source LLMs are so good.) If we want to achieve real change, we need to accept that they exist, understand how that changes the scientific landscape and our options to go from here.

everyone keeps claiming "they're here to stay" as if it's gospel. this constant drumbeat is rather tiresome and without much hard evidence.

If they go away, it's because they have been replaced by something better(worse) like LLLM or LLMM or whatever.

I'old enough to remember when GAN where going to be used to scam millions of people and flood social media with fake profiles.

Post reply on HN