How Web Scraping Is Revealing Lobbying and Corruption in Peru

I received a C&D from a Fortune 100 asserting that I was accessing their site in violation of the CFAA, among numerous other silly claims. I'm a 1 person company. I did have some revenues, but they were about even with my full-time job (this was a side project); certainly not enough to satisfy the retainers that lawyers wanted before they'd even think about taking me on. I eventually found a lawyer who agreed to help a little bit for a $2k retainer, but as you'd expect for that rate, I can't get much out of him.

I wasn't doing anything egregious. The product I offered did not compete with their products; it actually made it easier for the consumer to spend money with them. The data I was gathering is mere factual data and is not subject to copyright (though, as in Ticketmaster v. RMG, this alone will not protect from copyright infringement claims). Their site is the single place that this factual data can be found.

Their Terms of Use forbids access by either manual or automated processes; thus, it makes it illegal for anyone to use their site at all, and precludes any solution based on MTurk or similar. It also forbids any access for "commercial use". Combined, this means they can sue you and make you stop using their site basically whenever they want for any reason. They could've done this anyway because the CFAA protects them from any "unauthorized" access.

If I were to actually dispute this company's claims and refuse to comply with their C&D, they would sue me. This would've cost me millions of dollars in legal fees before the case was through, which is irrelevant to them but obviously well outside of my reach. There's a good chance they would've gotten an injunction legally forbidding me from continuing to offer my service almost immediately, so then I'd have been stopped from offering my product AND I would've had a pending lawsuit against me, which would've asserted some absurd dollar amount of damage, and, if Facebook v Power Ventures is any indication, there would've been a good chance that I would've been held personally liable for it.

It doesn't matter that their claims are all dependent on interpretation and grey area. What matters is that if you don't have $30-$40 million dollars sitting around, you can't take the risk of a lawsuit from a big company. Gotta earmark $1-10 million for legal fees (depending on what kind of lawyers you get; the opposing party in my case has one of the most expensive law firms in the country); set aside $5-10 million in case you lose and have to pay damages, set aside some chunk of money to continue to bear the cost of maintaining and running the business despite the legal pressure and despite the likelihood that you've been legally disallowed from selling your primary money maker pending resolution of the case, which will likely drag on for a minimum of 3-5 years, and up to 10 years is not really unheard of. Gotta have the extra $20 mil+ so that you don't pour more than 50% of your net worth into something that is very possibly a losing battle.

My lawyer advises me that the various workarounds I devised could be construed as conspiracy and aiding and abetting, even though I would no longer be making any requests to the complainant's servers at all. This also wouldn't stop the complainant from suing me for past damages or to stop the practice they dislike, even if I'm doing it through means that totally obviate the need to access any of their servers.

If I were to continue operating, the only option would be to leave the U.S. entirely for a jurisdiction that doesn't enforce U.S. judgments (since I would be sued in the US and lose by default; my lawyer indicates that merely moving my company overseas is insufficient), and not return until the statue of limitations expires on the judgment that would get registered. Even this is not foolproof because the activity would have to be obviously and unequivocally legal in the new host jurisdiction so that the company's lawsuit in that jurisdiction wouldn't get anywhere, the jurisdiction would have to decline to enforce judgments on U.S. persons, and they'd have to be impervious to attempts by one of the world's largest companies to influence their legal system. I haven't found such a jurisdiction yet. Some are kinda-sorta close (but not really).

I couldn't reply because I was submitting too fast so instead of replying I added to my original point which was that 3taps took the heat for Padmapper. The fact padmapper didn't get slapped with CFAA, meant 3taps took the major heat and like you are going on about CFAA as being the biggest blunt force, I don't see why it makes his response look weird. He even wrote that padmapper was not the subject of a CFAA, 3taps was. It makes sense that he can't talk in detail about the case for legal reasons.

I hate that HN does that to anyone. It should be reversed only for spam bots and obvious bad faith participants, not someone with an unpopular opinion trying to have a conversation. I've encountered it before too. Sorry that it happened to you. You may want to lodge a complaint with dang so that he understands it's not a good mechanism.

I definitely think that on the outset, it looks weird that 3Taps ended up taking PadMapper's heat, but I think that 3Taps wanted to become a generalized thing-as-a-service vendor. It's possible that PadMapper wasn't 3Taps's only customer for the CL feeds. As PadMapper wasn't contacting CL's computers without authorization, it makes sense that CL had to change the target to 3Taps. At that point, PadMapper would've seen that scraping CL meant a near-impossible legal challenge for a startup and been wise enough not to implement their own solution.

This is all just speculation, but I doubt that 3Taps stuck its neck out for the sole benefit of PadMapper.

We weren't doing anything remotely nefarious with the non-copyrightable data we gathered. Some details are intentionally unclear. You can continue to make your own inferences on these.

Big companies don't send polite emails asking you to pretty please stop. They just let their lawyers deal with the whole kit and kaboodle.

It is illegal, or close enough to illegal, to scrape from practically any company in the U.S., because "unauthorized access" is a floating definition; as soon as that company makes a decision that they don't want you doing that thing you do anymore, you're doing something illegal; their change of heart can make your previously fine action a crime. The Terms of Use for almost all companies state as much. The statute does not state any required notice period or method, so you'd have to argue to the relevant magistrate that you didn't have reasonable knowledge that your scrape was unauthorized. This is the crux on which all scraping cases have hung, and the results are usually not favorable at all to the scrapers, although 1 or 2 recent decisions are sort of hopeful. Also note that this is only the CFAA portion; these suits usually allege a bunch of other torts too, which have proven similarly difficult to beat.

Scrapinghub's existence depends essentially on luck; first, that they won't get sued, and second, if they do get sued, that they'll get a sympathetic judge who will find that no contract was entered due to insufficient notice. That is not likely due to the nature of scrapinghub's operations (see Register.com v. Verio). The fact that some people are able to scrape and get away without being sued doesn't change the legal reality or the dubiousness of investing in a company with such a large risk profile.

The knife CEO analogy fails because CFAA claims are NOT about how the data is used. They are about the method used to obtain the data. The entity exceeding authorized computer access or accessing a computer without authorization -- in this case, that is scrapinghub, kimono, et al -- is the entity that has committed the violation of the CFAA. In your knife analogy, if the knife company had illegally acquired the metals used to manufacture the knife, it would be the culpable party, not the end user that bought its knives. The data that scrapinghub goes out, obtains, crafts and packages according to customer specifications ("make this page on craigslist a CSV file that auto-updates every 5 minutes") is the metal that the knife company goes out, obtains, crafts and packages according to customer specifications ("make this metal a sharp cutting utensil").

The person using the data that results from CFAA violations may be doing other illegal things, but in almost all of these types of cases, they're not violating the CFAA if they're not the ones accessing the computer that supplies the data.

I'm really not sure what you're arguing about anymore. The CFAA isn't a real law because the person gathering the data isn't necessarily the one putting it to use? I don't understand.

okay I think you are honestly trolling now. nicely played and good bye.