Live data from Hacker News

Sci-Bay: Google Scholar plus Sci-Hub

sci-bay.org

21–30 of 114 posts

Re: Sci-Bay: Google Scholar plus Sci-Hub

#21
post #9
post #6

How did you integrate scholar in sci-bay ? Does scholar have an API ? And about the future, how do you see google responding?

I don't know how they are doing it, but Google Scholar does not have an API, and scraping is against their TOS. > Don’t misuse our Services. For example, don’t interfere with our Services or try to access them using a method other than the interface and the instructions that we provide. Despite this, there is scholar.py [0], which can extract files from Google Scholar, though it explicitly doesn't work around the rat…

or try to access them using a method other than the interface

Unless this actually exploits something and hacks into Google's servers to get to the content, which would be something quite different, it wouldn't really be distinguishable from someone manually visiting the site in a browser, volume aside.

IMHO the pervasive attitude today of somehow requiring permission or an explicitly sanctioned "API" to access what is otherwise publicly accessible data is rather troubling for the freedom and flexibility of the Web as a whole. It encourages walled-garden content models and centralisation.

Re: Sci-Bay: Google Scholar plus Sci-Hub

#22
post #14

Google Scholar definitely and intentionally offers no API. I don't see this lasting long...

At a glance it looks like it's really just a proxy, that was limited to scholar.google.com and mutates the page slightly (adds a header, sci-hub links). Does google generally block proxy servers?

I don't know if they generally do, but I'm sure they can/will if they want to.

Re: Sci-Bay: Google Scholar plus Sci-Hub

#24

Can someone help dispel my ethical concerns over using papers like this? Eg sci-hub. Any and all arguments, on both sides are very welcome.

Well, the research and the researchers are all almost entirely funded with public funds from around the world. The fact that they are forced for professional reasons to funnel all their work into a for-profit publishing industry for distribution is a historical anachronism that dies hard.

In a broad sense, there's only very weak analogy to music or film piracy in terms of the process of creative work. Everyone has already paid for scientists to do this science. Scientists and institutions have even paid the journals to publish their work in almost all cases, and they won't reap any reward if you read the work through the legitimate distribution channel. You only pay the publisher, not a starving researcher.

Personally, I'd feel very happy if someone read my papers by any means necessary. That was the whole point of writing them, setting aside the practical annoyance of simply trying to keep my career afloat.

Edit: The lifetime value of a single paper to the journal can actually be priced, since they started charging researchers for open access a few years ago (often in addition to other fees). This number is usually in the $2-5K range per paper, in my field.

Re: Sci-Bay: Google Scholar plus Sci-Hub

#25

Can someone help dispel my ethical concerns over using papers like this? Eg sci-hub. Any and all arguments, on both sides are very welcome.

A timely request:

http://slatestarcodex.com/2018/03/19/the-dark-rule-utilitari...

"So this is my argument that Sci-Hub can be ethical. Universalized it would destroy the system – but the system is bad and needs to be destroyed. And although this would break the law, a very slight amount of law-breaking might be a beneficial solution to inadequate equilibria that could be endorsed even when universalized."

Re: Sci-Bay: Google Scholar plus Sci-Hub

#27
post #14

Google Scholar definitely and intentionally offers no API. I don't see this lasting long...

At a glance it looks like it's really just a proxy, that was limited to scholar.google.com and mutates the page slightly (adds a header, sci-hub links). Does google generally block proxy servers?

They don't block startpage and they've been around for a while.

Re: Sci-Bay: Google Scholar plus Sci-Hub

#30

Can someone help dispel my ethical concerns over using papers like this? Eg sci-hub. Any and all arguments, on both sides are very welcome.

It allows for open-access of the epitome of human progress, which is something unavailable to low-income communities. It's copying data, not stealing from a museum, so it's not as if any journal is losing much, if anything.

There's no real ethical downside - allowing people who wouldn't normally have access to research helps to reduce inequality, and allows human progress to advance a lot quicker than if it weren't open. There are brilliant people of which have no access to research, who could - and would - more likely do something with it than not.

Given that we all pay for science, [0] from people in the lowest drudges of poverty, to people in the highest skyscrapers in New York, we should all have access to it. Imagine you paid taxes meant to improve roads in your area, but ended up going to a place hundreds of miles away from you. You'll never get to use these roads, and you'll be told you're unethical and unlawful if you even consider using them. It wouldn't be very fair, would it?

[0] https://undsci.berkeley.edu/article/who_pays

Post reply on HN