Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Dozens of scientific journals have vanished from the internet

sciencemag.org

Post reply on HN
Print view View original

131–136 of 136 posts

Re: Dozens of scientific journals have vanished from the internet

#131

Post by Nemo_bis »

nurbl wrote: post #43
Jaxkr wrote: post #26

Let’s be real here: was anything of value really lost? Any important work was likely cited, paraphrased, or duplicated elsewhere. Anyone disagree?

Part of the point of citing another article is that you don't then have to repeat all of it. So if B is cited by A, and B is no longer available, it's not really possible to read and understand A either. And likely A used information in B to justify some claim, which is now weaker.

That's pretty common either way. See Rekdal (2014), "Academic urban legends". https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4232290

Re: Dozens of scientific journals have vanished from the internet

#132

Post by Vinnl »

bnewbold wrote: post #121
Vinnl wrote: post #98

Earlier quoted context omitted.

> If folks want to help, it would be great to have a "youtube-dl for open access papers". I think Unpaywall is already trying to do this? Or at least, for every DOI they index, they try to include a link to the direct article if known.

Unpaywall is very helpful! However, even for direct PDF links, publishing platforms will often do things like check for a session cookie; if you don't have the correct cookie you get bounced back to the landing page, where you need find and follow another link. This isn't super complicated to work around (persist a cookie jar, use a headless browser, etc), but it doesn't work out-of-the-box with our crawlers, the sam…

I knew there had to be a reason, I couldn't imagine you not knowing about Unpaywall :) Thanks for elaborating!

Re: Dozens of scientific journals have vanished from the internet

#133

Post by garfieldnate »

cxr wrote: post #94
bnewbold wrote: post #67

At the Internet Archive, we are working on this exact problem, and have been in communication with the pre-print's authors. We have built open infrastructure (open source, open data) tracking "preservation coverage", for example: https://fatcat.wiki/coverage/search?q=is_oa%3Atrue+year%3A%3... and are working to improve crawling. There is a "save paper now" feature, as well as an API for bots. Organizations like DOAJ,…

> If folks want to help, it would be great to have a "youtube-dl for open access papers". Zotero has an existing set of "translators". And somewhat related to this request: I scratched out some notes last year about how to get more out of "zero-obligation communities" (like the pool of prospective contributors in open source) " rel="nofollow">https://www.colbyrussell.com/2019/02/15/what-happened-in-jan... . The long…

Wintergatan's Martin recently transitioned a lot of the work for his new marble machine to volunteers, and he essentially followed this pattern, casting himself into the product owner role to harness the power of volunteers. It seems to be working pretty well!

Re: Dozens of scientific journals have vanished from the internet

#134

Post by cinquemb »

bnewbold wrote: post #122
cinquemb wrote: post #97

Earlier quoted context omitted.

> as well as a long tail of small publishers that don't use simple/common mechanisms like OAI-PMH and the `citation_pdf_url` HTML meta tag to identify fulltext content. The OAI-PMH ecosystem sadly is not very complete or helpful for the use case of mirroring. Most of orgs/journals/conferences in this group just don't have the resources to be able do this, nor maintain something like this. It was funny last year when…

From what I have seen, the least technically resourced journals often use hosted platforms or free software like OJS (basically wordpress for journals), which comes with features like HTML meta tags and OAI-PMH by default. The trickier cases are when folks write their own platforms, or even write their own raw HTML with no templating, in which case adding tags to all landing pages or supporting an API would be a rela…

> From what I have seen, the least technically resourced journals often use hosted platforms or free software like OJS (basically wordpress for journals), which comes with features like HTML meta tags and OAI-PMH by default.

Yeah, and there are a lot of issues with OJS and the meta tagging and google being able to crawl a lot of these (not to mention site uptime where lots of sites go down for long stretches and google just assumes the site was taking offline permanently if kept down for a while). Esp when the meta tag locales don't map to the actual language used in the papers themselves (i.e. journal admins enter meta data tagged as EN but use indonesian for the text and the text of the paper) or like non iso locale codes in metadata fields, etc.

Right now, when journals/orgs/conferences want help getting indexed in google and have oai endpoints, we pass all their content through locale detect stuff and don't trust their meta data by default.

Re: Dozens of scientific journals have vanished from the internet

#135

Post by jrochkind1 »

Nemo_bis wrote: post #130
jrochkind1 wrote: post #12

Earlier quoted context omitted.

Say more? All of scihub is mirrored at archive.org? Where do I find out more about this?

You don't. shhhh

How do we know it's actually a thing at all, and not just something someone made up?

Re: Dozens of scientific journals have vanished from the internet

#136

Post by Nemo_bis »

jrochkind1 wrote: post #135
Nemo_bis wrote: post #130

Earlier quoted context omitted.

You don't. shhhh

How do we know it's actually a thing at all, and not just something someone made up?

It's not quite a secret, you can see yourself if you look closely enough.
Post reply on HN