In memory of Aaron Swartz: a collection of PDFs from PDFtribute

Cool stuff, have you seen this yet?

http://pdftribute.net/

Looking through all the files that are uploaded, there are a lot more non-English documents than I expected; I randomly clicked on 2. It's amazing how there is support from around the world.

We're working with the creator of edward.io to get some kind of integration between the two sites, and also working on a search/index/analysis tool for pdftribute.net

Cool effort.

But ... its a score to jstor. It's unorganized.

But ... science if full of noise and crappy publications these days anyway. Lots of ways to do the same thing, unprooven and only exists because everybody has to publish to stay relevant.

Now: How to really improve science ? My suggestion: A big python framework for each field of study. That has implementations of the real algorithms and models for comparison and benchmarking and even real life implementation.

See as example in the robotics field, ROS ( Robotics Operating System) . Ros is like a basis glue framework where universities and individuals can publish their code. Its decentralized, it has simulators so that scientists do not need to own the physical robots and can even compare(diff) results and algorithms in a very fast way.

The simulator can have a embedded browser + wiki + quora that explains X.

evolution: physical paper -> PDF -> simulator.

As long as these PDFs are exposed publicly (and linked to, which a tweet with or without #pdftribute will take care of), they will mostly be indexed by Google Scholar, which does a decent job of extracting metadata using heuristics etc.

Of course, it would be much better if people started embedding machine-readable metadata in PDFs (totally possible, see for example http://code.google.com/p/pdfmeat/), and if there was some agreed-upon format for bibliographic microformats, that could be embedded in websites listing articles.

We also eventually need an open alternative to Google Scholar. GS is great, and I use it every day (and love that you can output BibTex for example), but it has no API (and will never have one because of deals with publishers), actively resists automatic access, is a black-box in terms of how data is gathered, etc. Think of "Open Scholar" to Google Scholar as analogous to OSM vs GMaps. OSM might not look as pretty, or be as consistent in the beginning, but it enables a whole range of applications that GMaps doesn't. (And at least GMaps does have a fairly good API, even if it charges for overuse, GS has nothing).

(These are just some thoughts I've made, as I've been experimenting with an open scholar workflow, trying to share as much of the "byproduct" of the research, including rich notes and summaries, my own bibliography with links to OA pubs where they exist etc: http://reganmian.net/wiki/researchr:start).

Another thing I've found working on my project, where I try to expose OA links to as many pubs as possible, and regularly rescan to see if they are still available (and still OA), is how quickly documents disappear... Hosting on private pages is convenient, but fragile. Ideally, people would upload papers to university repositories, subject repositories like Arxiv.org, etc.