Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Arxiv.org reaches a milestone and a reckoning

scientificamerican.com

Post reply on HN
Print view View original

51–60 of 91 posts

Re: Arxiv.org reaches a milestone and a reckoning

#51

Post by thrwn_frthr_awy »

WHA8m wrote: post #20
ahelwer wrote: post #9

Earlier quoted context omitted.

Because you cannot rely on the rich to fund common goods through voluntary donations. It is not a good idea and never will be. Other than some medical research, charity has never solved a single social ill.

Discard the last sentence. You're talking in absolute terms and that's leading nowhere. It's been some time, but this is what came to my mind immediately: https://en.wikipedia.org/wiki/Fuggerei That you can not rely on the rich to put their money in good use with the general public in mind is right. Does it happen? Yes. Does it happen often? Definitely not.

Right! And that’s really my point–there are so many people in tech with money and an ego that I’m surprised this isn’t done more. Pineapple person in the most recent memorable example.

Re: Arxiv.org reaches a milestone and a reckoning

#52

Post by SiempreViernes »

_def wrote: post #28

I'm in no way involved in academic research, and I don't now how it works in detail, but a question that popped into my head these days was: is there a place where people can publish papers even without any association with anything? Like, if I manage to do proper research, even without any qualifications on paper, would it still be useful? Does this make any sense?

Good research is always useful, where good means something between "reliable and flawlessly explained" and "plausible and very novel".

So on one end you could takes something like Newtons Principa, that synthesised a lot of the known facts and laws at the time into one single coherent framework, and on the other you can have Galileo recognising that he was seeing surface features on the moon and not just patterns on a flat disk and announcing this will some illustrations showing the principle and not what he actually saw.

As to the first part of the question, if you talk of "publishing" in academia you usually mean you want your work to be know by other credible researchers in the hope they find it useful.

The most reliable way to achieve that without credentials is to befriend someone respected in the field you are interested in before you send them your results, because they are more likely to have patience with any errors or confusion in your presentation that way.

There are of course some places that pride themselves on being completely free from filtering, but the concept of "spam folder" along should tell you how good a chance you have of getting any serious attention in those places.

Re: Arxiv.org reaches a milestone and a reckoning

#53

Post by WHA8m »

thrwn_frthr_awy wrote: post #51
WHA8m wrote: post #20

Earlier quoted context omitted.

Discard the last sentence. You're talking in absolute terms and that's leading nowhere. It's been some time, but this is what came to my mind immediately: https://en.wikipedia.org/wiki/Fuggerei That you can not rely on the rich to put their money in good use with the general public in mind is right. Does it happen? Yes. Does it happen often? Definitely not.

Right! And that’s really my point–there are so many people in tech with money and an ego that I’m surprised this isn’t done more. Pineapple person in the most recent memorable example.

Yes, I think we're on the same page here. Your main point was, that you can not rely on the rich to use their money to benefit others and I totally agree.

Re: Arxiv.org reaches a milestone and a reckoning

#54

Post by indigochill »

YeGoblynQueenne wrote: post #31

What you lose on arxiv is peer review. That might sound like a feature- you get to let others know of your results faster- but if you make a mistake it will not be caught until it's out in the wild (and possibly until everyone has already cited it and based their own work on it). With peer-review you know that someone will read your work that has the background to understand it and to catch errors. So what, you'll sa…

Borrowing from software engineering, you can publish code to a git repository and someone else can make a pull/merge request with a proposal to make fixes/improvements. There's still work involved in reviewing, revising, and merging those proposals, but it can be less work than needing to make every change yourself.

I wonder if academic papers could benefit from a similar process of communal development (though it flies in the face of the dead-tree format of academic papers, but we're talking about Arxiv here, not Elsevier). Even if suggestions don't get merged, having a repository that captures discussion around the central artifact is also useful (such as when a maintainer decides that a feature doesn't belong in a project).

Re: Arxiv.org reaches a milestone and a reckoning

#55

Post by dandanua »

YeGoblynQueenne wrote: post #31

What you lose on arxiv is peer review. That might sound like a feature- you get to let others know of your results faster- but if you make a mistake it will not be caught until it's out in the wild (and possibly until everyone has already cited it and based their own work on it). With peer-review you know that someone will read your work that has the background to understand it and to catch errors. So what, you'll sa…

> but there's no incentive to do the hard work to correct mistakes if your paper is already on arxiv and people are already citing it

Peer reviewing is essential, of course. But note that in most journals reviewers are not getting paid. They have even less incentives to find mistakes in your work than you are. I have no idea how such an utterly broken system emerged, where journals siphon billions of tax money exploiting the work of scientists.

Re: Arxiv.org reaches a milestone and a reckoning

#56

Post by kgwgk »

atomack wrote: post #44
YeGoblynQueenne wrote: post #31

What you lose on arxiv is peer review. That might sound like a feature- you get to let others know of your results faster- but if you make a mistake it will not be caught until it's out in the wild (and possibly until everyone has already cited it and based their own work on it). With peer-review you know that someone will read your work that has the background to understand it and to catch errors. So what, you'll sa…

The way we used arxiv worked well in physics, though this is 15 years ago now so might have changed since. arxiv was about distribution. It didn't replace peer review - articles were still submitted to journals and published there too. If an article was posted to arxiv and not a journal, the odds of a citation went down massively. And the journal it was submitted to was a factor in whether or not we read it. When art…

> arxiv was about distribution. It didn't replace peer review - articles were still submitted to journals and published there too.

I'm surprised that they no longer use the term "preprint" at all, at least it's nowhere to be found on the homepage or "about" section.

The consequences of this amnesia are hilarious: https://twitter.com/gustavnilsonne/status/138948729731431219...

> Why do we call it "preprints"? The term seems to imply that work is preliminary or unfinished. As far as I can tell, the term introduced by @arxiv , the first online repository for scientific manuscripts, is "e-print". Is "preprint" a marketing device invented by publishers?

Re: Arxiv.org reaches a milestone and a reckoning

#57

Post by monkeyjoe »

kgwgk wrote: post #56
atomack wrote: post #44

Earlier quoted context omitted.

The way we used arxiv worked well in physics, though this is 15 years ago now so might have changed since. arxiv was about distribution. It didn't replace peer review - articles were still submitted to journals and published there too. If an article was posted to arxiv and not a journal, the odds of a citation went down massively. And the journal it was submitted to was a factor in whether or not we read it. When art…

> arxiv was about distribution. It didn't replace peer review - articles were still submitted to journals and published there too. I'm surprised that they no longer use the term "preprint" at all, at least it's nowhere to be found on the homepage or "about" section. The consequences of this amnesia are hilarious: https://twitter.com/gustavnilsonne/status/138948729731431219... > Why do we call it "preprints"? The term…

I think it's because the publisher retains copyright. There is a limit on how "done" the manuscript can be and still be shared for free online. Some universities have started to fight back against this by limiting the scope of copyright restrictions that publishers can impose.

Re: Arxiv.org reaches a milestone and a reckoning

#58

Post by nuclearnice1 »

indigochill wrote: post #54
YeGoblynQueenne wrote: post #31

What you lose on arxiv is peer review. That might sound like a feature- you get to let others know of your results faster- but if you make a mistake it will not be caught until it's out in the wild (and possibly until everyone has already cited it and based their own work on it). With peer-review you know that someone will read your work that has the background to understand it and to catch errors. So what, you'll sa…

Borrowing from software engineering, you can publish code to a git repository and someone else can make a pull/merge request with a proposal to make fixes/improvements. There's still work involved in reviewing, revising, and merging those proposals, but it can be less work than needing to make every change yourself. I wonder if academic papers could benefit from a similar process of communal development (though it fl…

Some of the sentiment of your last sentence is embodied in open reviews.

https://openreview.net/group?id=ICLR.cc/2022/Conference&refe...

Re: Arxiv.org reaches a milestone and a reckoning

#59

Post by YeGoblynQueenne »

dandanua wrote: post #55
YeGoblynQueenne wrote: post #31

What you lose on arxiv is peer review. That might sound like a feature- you get to let others know of your results faster- but if you make a mistake it will not be caught until it's out in the wild (and possibly until everyone has already cited it and based their own work on it). With peer-review you know that someone will read your work that has the background to understand it and to catch errors. So what, you'll sa…

> but there's no incentive to do the hard work to correct mistakes if your paper is already on arxiv and people are already citing it Peer reviewing is essential, of course. But note that in most journals reviewers are not getting paid. They have even less incentives to find mistakes in your work than you are. I have no idea how such an utterly broken system emerged, where journals siphon billions of tax money exploi…

The incentive to freely review others' work is that others will also freely review your work.

This is quite separate than the exploitative aspects of the big scientific publishers. Reviewers are certainly exploited by publishers, because publishers make money from reviewers' free work. But reviewers would do the work for free anyway because that's what they expect others to do.

Re: Arxiv.org reaches a milestone and a reckoning

#60

Post by YeGoblynQueenne »

marginalia_nu wrote: post #46
Karellen wrote: post #41

Earlier quoted context omitted.

> in several fields, 50-90% of peer-reviewed studies fail to replicate. Is that a failure of peer-review though? Or is it more of a consequence that even if you do everything right you'll still get some false positives some of the time, and negative results tend not to get published at all? (i.e. the problem that pre-registration for medical trials hopes to solve - https://en.wikipedia.org/wiki/Preregistration_(scien…

> Is that a failure of peer-review though? It raises questions around using peer-review as a mark of scientific quality. If it is, why does peer reviewed science time and time again turn out to have enormous glaring quality problems?

I think this is like the difference between homeopathy and medicine. If you're sick and go to a doctor, you have a chance that the doctor will find a way to cure you. If you're sick and go to a homeopath, there's no chance that the homeopath will find a way to cure you. Similarly, peer review has a chance to catch errors before they make it to print, while absence of peer review has no such chance.

Peer review is not infallible, neither is it a mark of scientific quality, as you say. I appreciate that people tend to use it like that- you read, in articles in the lay press, that such-and-such study was "not published in a peer-reviewed journal" as if to say that we can't be sure of the quality of the study and that, conversely, if it was peer-reviewed, we could. That's not right. Peer review is not a sufficient condition to guarantee correct results. It's not even a necessary condition. Like I say in my comment above it is one mechanism of modern research practices that helps improve the quality of published papers.

Post reply on HN