Live data from Hacker News

9th Circuit holds that scraping a public website does not violate the CFAA [pdf]

cdn.ca9.uscourts.gov

201–210 of 293 posts

Re: 9th Circuit holds that scraping a public website does not violate the CFAA [pdf]

#201

Earlier quoted context omitted.

Not allowing the CFAA to be (ab)used to attempt to make scraping illegal makes sense. However, how is it reasonable to force a web site to serve its contents to a third-party company, without being allowed to make a decision whether to serve it or not? Serving the web site costs money, and the scraper surely isn't going to generate ad income...

Does their robots.txt say don't crawl this part of the site? If it does, this ruling is catastrophic. If it doesn't then there is hope.

> If it does, this ruling is catastrophic

I know it is a generally considered bad form to ask, but did you read much of the ruling? I feel like a lot of people on this thread are just going off of Animats' comment and haven't spent much time looking at the opinion.

I didn't read the whole thing, but skimmed through it and read what seemed to be the relevant parts of the argument. (Including the bit that talks about LinkedIn's robots.txt)

The ruling doesn't really support your claim of catastrophe and doesn't claim to pass any sort of final judgement.

The judge makes a specific point about not reading too much into him upholding the injunction saying:

>> These appeals generally provide “little guidance” because “of the limited scope of our review of the law” and “because the fully developed factual record may be materially different from that initially before the district court.”

Re: 9th Circuit holds that scraping a public website does not violate the CFAA [pdf]

#202

Earlier quoted context omitted.

I mean it should. That is also a huge anti-competitive action that just isn't pursued by anyone yet: google makes money of scraping and denies scrapers scraping them - that's all sorts of messed up. The problem is that someone would have to sue google first and no one will do that unless there's big business incentive and big business can already scrape the shit out of google. This is the weird thing about web-scrapi…

Google respects robots.txt, so it might be hard to prove that they are accessing websites without their implied consent. Also, their own robots.txt contains "Disallow: /search". So, there is arguably no inconsistency, either. But, what does this new ruling mean for robots.txt?

I think OP is getting at the nature of the relationship is kinda imbalanced. Consider basically most of their website is off limits: https://www.google.com/robots.txt

Re: 9th Circuit holds that scraping a public website does not violate the CFAA [pdf]

#204
post #63

Earlier quoted context omitted.

Regardless, there is legitimate value in the collection, cleaning, interlinking, and presentation of existing data. How that is interpreted by the law is one thing but merely because the data came from a variety of other public/private sources doesn't mean it derived all of its value externally.

For sure, but they shouldn't be hypocritical about it. If they don't consider themselves content parasites, they shouldn't consider people scraping their site to be content parasites, either. (Some sites really are just parasites, though.)

There's nothing hypocritical about it. Googlebot respects robots.txt configured on pages it scrapes. Google in turn expects that their own robots.txt will be respected. What's the issue?

https://www.google.com/robots.txt

Re: 9th Circuit holds that scraping a public website does not violate the CFAA [pdf]

#205

Earlier quoted context omitted.

Google respects robots.txt, so it's arguably not the same as scraping a website without their consent.

Most sites don't have their main data/functionality in the Disallow section though.

Sure, but they could if they wanted to and that's their own business.

Re: 9th Circuit holds that scraping a public website does not violate the CFAA [pdf]

#206

Earlier quoted context omitted.

The data are certainly not free of copyright. Data can contain user picture, or even small essay describing the job, life of a user though linkedin is not the copyright holder. Moreover these are personal data, and I'm not sure that the scraper has the original user right to collect the data. In Europe, the scrapper may face issues related to GDPR.

Facts can't be copyrighted, so such things as whether or not a person worked for a certain company, or went to a certain school, are unprotected, and with this ruling can be scraped, at least in the U.S. Others things common on LinkedIn, as you rightly point out, are protected--but by copyright law, not the CFAA. So a scraper acting in good faith would have to be careful about what they used if they wanted to respect…

I never said that fact can be copyrighted, I said that most of the things people put around in their profile can be. I was responding to the claim that the data were not under copyright made above. If you just scrap name, company, position, this is fine, but I highly doubt that they just do that. This lawsuit can have tons of side effects.

Re: 9th Circuit holds that scraping a public website does not violate the CFAA [pdf]

#208

Earlier quoted context omitted.

As a real-world analogue: you can indeed be guilty of trespassing on someone's property even if you don't have to jump over any fences or pick any locks to get there. In some places, they don't even have to have a "no trespassing" sign. Simply being present on someone else's property without an invitation from them is illegal, and no, an open door does not count as an invitation.

> an open door does not count as an invitation. But if you have a someone living there who lets anyone in if they ask, it would be pretty hard to argue they are trespassing. A user agent must ask for every page with an http request . If the server responds with 200 OK , it’s pretty hard to argue that it isn’t letting (or even inviting) you in.

This doesn't cover you if you lied to get the invitation. If a robots.txt file denies some but not all user agents, setting your user agent to indicate that your request originates from a source it does not originate from is clearly a circumvention of an access control.

Re: 9th Circuit holds that scraping a public website does not violate the CFAA [pdf]

#209
post #196

Earlier quoted context omitted.

Ugh, yeah, the more I think about this ruling, the less I like it. It's actually pretty insane to force a site to serve content. I think both parties are in the wrong here - HiQ for assuming they're entitled to receive a response from LinkedIn's webservers, and LinkedIn for abusing the CFAA to try to deny service rather than figure out a technical solution to their business problem. In my view: * The data is public,…

From reading the opinion, I think the argument goes something like this: > First, LinkedIn does not contest hiQ’s evidence that contracts exist between hiQ and some customers, including eBay, Capital One, and GoDaddy > Second, hiQ will likely be able to establish that LinkedIn knew of hiQ’s scraping activity and products for some time. LinkedIn began sending representatives to hiQ’s Elevate conferences in October 201…

So, if you want to block someone from your service, you need to be able to prove that it is for a legitimate business purpose.

Moreover it seems, 'this harms a competitor of ours' is not considered a legitimate business purpose, but anti-competitive behavior.

Re: 9th Circuit holds that scraping a public website does not violate the CFAA [pdf]

#210
post #196

Earlier quoted context omitted.

Ugh, yeah, the more I think about this ruling, the less I like it. It's actually pretty insane to force a site to serve content. I think both parties are in the wrong here - HiQ for assuming they're entitled to receive a response from LinkedIn's webservers, and LinkedIn for abusing the CFAA to try to deny service rather than figure out a technical solution to their business problem. In my view: * The data is public,…

From reading the opinion, I think the argument goes something like this: > First, LinkedIn does not contest hiQ’s evidence that contracts exist between hiQ and some customers, including eBay, Capital One, and GoDaddy > Second, hiQ will likely be able to establish that LinkedIn knew of hiQ’s scraping activity and products for some time. LinkedIn began sending representatives to hiQ’s Elevate conferences in October 201…

That’s quite ... crazy.

Be restaurant. Be on Deliveroo. Be getting low margins because of high fees.

So basically you can’t decide not to use Deliveroo any more, to improve margina (“secure an exonomic advantage”). I mean, you can cancel Deliveroo, but only as long as you’re not “inducing a breach of their contract”. So only a matter of time before Deliveroo writes a contract “we’re obligated to deliver food for you from said restaurant”.

Post reply on HN