Live data from Hacker News

Aaronsw indicted for hacking MIT network to download millions of JSTOR docs

documentcloud.org

271–280 of 324 posts

Re: Aaronsw indicted for hacking MIT network to download millions of JSTOR docs

#271

This reminded me of Jacques Mattheij's list of ideas for startups (see http://jacquesmattheij.com/My+list+of+ideas+for+when+you+are... ): "(45) OpenPapers A place where all academic research that has been funded in part by public funds is published, journals be damned. Hopefully with deep pockets to fight off the lawsuits."

arXiv seems to fit this bill, and has a reasonable bulk data access policy http://arxiv.org/help/bulk_data

Re: Aaronsw indicted for hacking MIT network to download millions of JSTOR docs

#272

Earlier quoted context omitted.

50k is cheap as hell. UMass Boston had 125,000 JSTOR downloads last year. Let's assume we pay $50k (close enough), that's 40 cents an article. This is cheap as hell compared to other databases that charge more than that for a single search!

It's further harming the possibilities of being an independent scholar or even reading academic literature outside academia, though, which I think is a significant detriment to academia. I'm not as worried about the $50k a university library has to pay, but if you're an independent scholar for even one year, it becomes clear how harmful to the research community JSTOR is. Journals that used to sell archive access to…

Many university libraries allow to you join as a "friend" of the library, which typically costs ~$100/year and gives you full access to all library resources.

Re: Aaronsw indicted for hacking MIT network to download millions of JSTOR docs

#273

Earlier quoted context omitted.

I see your point, but you could just as easily walk into your local university, use their network as a guest, and access millions of dollars worth of content for free. Most databases allow you to download PDFs so you can download a bunch of articles and take them home, legally. Proxying through your grad student friends is breaking the law.

Even among state universities access to electronic journals for guests is becoming increasingly rare (usually prohibited in the license agreement). Also if you look to much into how scholarly communications works, the idea of charging 50k is insane since that vast majority of the cost of journals are actually paid for by universities in the first place. edit: although I do agree with your point that in with the curre…

But many state schools let you join the library even if you're not a student. e.g. http://www.lib.iastate.edu/info/6197 - $20/year for a non-affiliated member.

Re: Aaronsw indicted for hacking MIT network to download millions of JSTOR docs

#274

What Aaron did sounds seriously sketchy (sneaking into MIT wiring closets, trying to download the entire database, etc.), a fact that Demand Progress and several commenters here seem to be ignoring. Defending his actions would require a very strong, multi-pronged version of the argument "if it's physically / technologically possible, it must be ok." Can MIT legally limit guest access to its network? Can JSTOR limit a…

Thanks for the dissenting opinion. Is this worse than the sort of thing that goes on in the early days of most startups, including our most revered? People around here have a lot of respect for pg, rtm, tlb and their startup Viaweb - go back to "Founders at Work" and read how they got computer time needed to get the startup going. That kind of thing is practically universal in startups. The good ones, anyway. So Aaro…

I wonder if it has anything to do with the fact that he's in charge of a PAC rather than a startup. IOW these charges could be politically motivated.

Re: Aaronsw indicted for hacking MIT network to download millions of JSTOR docs

#275

Earlier quoted context omitted.

I see your point, but you could just as easily walk into your local university, use their network as a guest, and access millions of dollars worth of content for free. Most databases allow you to download PDFs so you can download a bunch of articles and take them home, legally. Proxying through your grad student friends is breaking the law.

Even among state universities access to electronic journals for guests is becoming increasingly rare (usually prohibited in the license agreement). Also if you look to much into how scholarly communications works, the idea of charging 50k is insane since that vast majority of the cost of journals are actually paid for by universities in the first place. edit: although I do agree with your point that in with the curre…

I would be impressed if it's an actual trend that guests were being blocked from accessing licensed content.

I work in this field and I can say that almost all databases and journals have IP authentication, generally with the entire campus IP range white listed. If the vendors don't implement that, they generally use password protected accounts. There are some alternatives such as uploading a list of barcodes, Referring URLs (I'm not kidding ProQuest for example allows this), Athens, Shibboleth, and a few others. These, however, are not commonly implemented by libraries because they are not widely adopted by vendors and require additional IT support that libraries simply don't have.

Therefore, unless libraries proxy all their users through their proxy servers (They don't for on-campus users. Usually, all links will go through the proxy, but if on campus the proxy will redirect you directly to the database to avoid the overhead), it would be near impossible to enforce the restriction of guests. I would wager that in many of these scenarios, it's all smoke and mirrors, and that access is really there if you know what you're doing (e.g. just visit the vendor site while on-campus without going through the library links).

Re: Aaronsw indicted for hacking MIT network to download millions of JSTOR docs

#276
post #220
post #151

I don't understand why the US Attorney is suggesting he be charged with such ridiculously inflated charges. Reviewing the Indictment report, "JSTOR did not permit users... to download all of the articles from any particular issue of a journal." Further, "JSTOR notified its users of these rules, and users accepted these rules when they chose to obtain and use JSTOR’s content." So basically JSTOR is claiming Aaron viol…

Frankly, I think most of the claims in your post are bonkers, but this one sticks out: This inconveniencing of other users could have been avoided and the blame for how JSTOR allocates resources lies with the architects of JSTOR. "During November and December, 2010, Swartz used the "ghost laptop" (i.e., the Acer laptop) at MIT to make over two million downloads from JSTOR. This is more than one hundred times the numb…

Well, it does seem that if JSTOR serves 7000 institutions around the world, having that number effectively jump to 7100 for a couple of months shouldn't be a big drama. Also, it's pretty surprising to me that the average number of downloads at MIT was about 1.4 per person per month. (2 million / 100 / 12000 people / 2 months.) I would have expected at least one order of magnitude higher.

Re: Aaronsw indicted for hacking MIT network to download millions of JSTOR docs

#277

I'm probably Advocating Crime but, couldn't a bunch of people coordinate and do what he was doing, over a year or two, distributed, from several universities? You totally could. The more, the less noticeable, and punishable. edit: it wouldn't surprise me if something like that showed up; the problem has been highlighted, the legal issues made clearer, and JSTOR bloodied. Go Aaron.

JSTOR might watermark PDFs to prevent this, as some other similar services do.

Re: Aaronsw indicted for hacking MIT network to download millions of JSTOR docs

#278

Earlier quoted context omitted.

Even among state universities access to electronic journals for guests is becoming increasingly rare (usually prohibited in the license agreement). Also if you look to much into how scholarly communications works, the idea of charging 50k is insane since that vast majority of the cost of journals are actually paid for by universities in the first place. edit: although I do agree with your point that in with the curre…

I would be impressed if it's an actual trend that guests were being blocked from accessing licensed content. I work in this field and I can say that almost all databases and journals have IP authentication, generally with the entire campus IP range white listed. If the vendors don't implement that, they generally use password protected accounts. There are some alternatives such as uploading a list of barcodes, Referr…

I've been at at least one university (a smallish liberal-arts college) that used a separate foo_guest wifi network for guests, which dumped people into an IP range that wasn't in the database-access whitelist (enrolled students and faculty would connect to a different wifi network, using their username/password, that was whitelisted). I agree that that isn't usual practice, though, despite being technically required by some database agreements.

Re: Aaronsw indicted for hacking MIT network to download millions of JSTOR docs

#279
post #103

Demand Progress PAC's website is down, but they released a statement: (from: http://webcache.googleusercontent.com/search?q=cache:9k5ryiX... ) Cambridge, MA– Moments ago, Aaron Swartz, former executive director and founder of Demand Progress, was indicted by the US government. As best as we can tell, he is being charged with allegedly downloading too many scholarly journal articles from the Web. The government conten…

[deleted]

Re: Aaronsw indicted for hacking MIT network to download millions of JSTOR docs

#280
post #231
post #216

Earlier quoted context omitted.

Are you sure? Maybe not in the world, but I'm pretty sure all large public libraries in the US do subscribe to these kinds of databases.

I admit that I don't have statistics [edit: on libraries], but most libraries in the world are not large or in the US, and JSTOR's prices for a "small" library in "the rest of the world" are much, much larger than [edit: wrong — comparable to or perhaps a bit larger than, but not much, much larger than] their entire budget. Check out http://support.jstor.org/csp/PriceCalculator/ . This code (for Chrome) gives me a ye…

It's sad that you have to write javascript code to do that! (But also cool that you did. :)

"Complete Current Scholarship Collection" for 22751.90 is a duplicate of all the things above it. So I think some of the entries have been double counted.

The real price for most libraries may about 1/2 or less of your estimate (they won't be interested in everything). And 20,000 to 40,000 is (well, shouldn't) be a lot of money for a public library.

That's the salary for a single employee! I would expect a library to have at least 5 employees, plus a budget to buy books.

Also I would expect a small library to have only a subset of the papers, and for serious research you would need to "go into the city".

Post reply on HN