A plan to mine the world’s research papers

Maybe I missed it, but the article doesn't seem to explain exactly how Malamud's group compiled its database.

Throttling is a major problem with the naive approach of throwing wget on a publisher site. The publisher detects a bot on its network downloading everything in sight and either slows data transfer to a trickle or just shuts down access to it.

The publishers may not win on copyright, but they may try to make a case based on criminality if Malamud's team actively took steps to circumvent throttling and defeat the defenses of the hosting sites. Especially if the publisher knows what to look for in its logs.

Do you happen to know if it's only SciHub or it also includes Libgen? If it's only SciHub that's a lot of papers... ^__^;

In my day job, I'm often tasked with implementing algorithms from recently published physics papers. In order to do so, I normally have to read through at least 10 related papers (both cited papers and cited by papers) in order to have a clear idea in my head about what is going on. Even then, I often have to discuss what I have read with 2 or 3 other people and try out many different approaches before we finally understand what the paper meant.

This is because few papers include enough details to reproduce what they are doing. Many of these papers have mistakes or typos in their equations. Many of the math equations are also under specified. For example, an equation may list a sum over an index, but then in the text there may be a whole paragraph that describes what that index means (there is nothing wrong with this, but it would make it hard for AI to "just use the equations").

At the end of this whole process, which can take a week, about half of the time we choose not to implement the algorithm for one of several reasons:

* The authors misrepresented their work and it does not perform as well as claimed (e.g. the chosen examples are special cases that make their approach look better than the state-of-the-art approach)

* The work is not reproducible from the information in the paper

* The amount of work to implement it is far greater than an initial reading of the paper would suggest, due to additional details that were left out of their discussion

So, all of this is to conclude that understanding scientific papers is at the very limit of human ability for a group of PhDs in the field. I do not think that until we have much more powerful AI that it has any hope of making sense of this mess.

edit: P.S. I am guilty of these same mistakes when publishing. I understand the deadlines and pressure to publish that leads to these issues. It is a huge amount of work to fully document and publish all the details needed to reproduce some new algorithm .