Live data from Hacker News

9th Circuit holds that scraping a public website does not violate the CFAA [pdf]

cdn.ca9.uscourts.gov

161–170 of 293 posts

Re: 9th Circuit holds that scraping a public website does not violate the CFAA [pdf]

#161
post #22

This action does more than that. The court left the preliminary injunction against LinkedIn in place: "The district court granted hiQ’s motion. It ordered LinkedIn to withdraw its cease-and-desist letter, to remove any existing technical barriers to hiQ’s access to public profiles, and to refrain from putting in place any legal or technical measures with the effect of blocking hiQ’s access to public profiles." So Lin…

"If this case is eventually decided in favor of hiQ, scrapers can no longer be blocked."

If the case is decided in favour of hiQ, then, absent an injunction, what would prevent a website from blocking a scraper? Maybe the website could still block unless and until the scraper gets her lawyers to file an injunction.

Another interpretation is that if hiQ wins, then in the 9th Circuit's jurisdiction websites serving public information they neither own nor exclusively license may no longer try to to use the CFAA and/or copyright law to threaten scrapers.

Re: 9th Circuit holds that scraping a public website does not violate the CFAA [pdf]

#162
post #64

Earlier quoted context omitted.

If you have to login to access the data, that data isn't publicly available, and unlikely to be protected by this decision. Page 31 of the filing specifically differentiates this case from one regarding Facebook: "While Power Ventures was gathering user data that was protected by Facebook’s username and password authentication system, the data hiQ was scraping was available to anyone with a web browser."

Wait, doesn't Linkedin have pretty strong authentication safeuguards to view user profiles. if you google someone and click their linkedin without being logged in to LinkedIn, you're always directed to sign up or log in to it. Twitter is the only example I can think of among the large social media sites that doesn't require you to be logged in to see profiles

Nope. You can see public profiles without being logged in. Certainly the case here in the UK; not sure about elsewhere.

Re: 9th Circuit holds that scraping a public website does not violate the CFAA [pdf]

#163
post #22

This action does more than that. The court left the preliminary injunction against LinkedIn in place: "The district court granted hiQ’s motion. It ordered LinkedIn to withdraw its cease-and-desist letter, to remove any existing technical barriers to hiQ’s access to public profiles, and to refrain from putting in place any legal or technical measures with the effect of blocking hiQ’s access to public profiles." So Lin…

Not allowing the CFAA to be (ab)used to attempt to make scraping illegal makes sense. However, how is it reasonable to force a web site to serve its contents to a third-party company, without being allowed to make a decision whether to serve it or not? Serving the web site costs money, and the scraper surely isn't going to generate ad income...

Isn't the issue of being selective on who can view the content? If I, random Joe User views the publicly available content you have no issue. But if someone scrapes that data them you'd want to charge them. Unless I click on the ad, the act of using your bandwidth doesn't change based on who the viewer is. You'd want to apply fees based on the future use of the data rather than on your actual costs.

Re: 9th Circuit holds that scraping a public website does not violate the CFAA [pdf]

#164
post #22

This action does more than that. The court left the preliminary injunction against LinkedIn in place: "The district court granted hiQ’s motion. It ordered LinkedIn to withdraw its cease-and-desist letter, to remove any existing technical barriers to hiQ’s access to public profiles, and to refrain from putting in place any legal or technical measures with the effect of blocking hiQ’s access to public profiles." So Lin…

Not allowing the CFAA to be (ab)used to attempt to make scraping illegal makes sense. However, how is it reasonable to force a web site to serve its contents to a third-party company, without being allowed to make a decision whether to serve it or not? Serving the web site costs money, and the scraper surely isn't going to generate ad income...

> However, how is it reasonable to force a web site to serve its contents to a third-party company, without being allowed to make a decision whether to serve it or not?

Your statement makes absolutely no sense. That's not how internet works. If you serve something publicly you don't get to cherry pick who sees it.

Not only it makes no sense technically it's also a huge anti-competitive case.

Re: 9th Circuit holds that scraping a public website does not violate the CFAA [pdf]

#165
post #22

This action does more than that. The court left the preliminary injunction against LinkedIn in place: "The district court granted hiQ’s motion. It ordered LinkedIn to withdraw its cease-and-desist letter, to remove any existing technical barriers to hiQ’s access to public profiles, and to refrain from putting in place any legal or technical measures with the effect of blocking hiQ’s access to public profiles." So Lin…

Does this prevent Google from returning captchas if you use a robot to scrape the search result pages, as they currently do?

I mean it should. That is also a huge anti-competitive action that just isn't pursued by anyone yet: google makes money of scraping and denies scrapers scraping them - that's all sorts of messed up.

The problem is that someone would have to sue google first and no one will do that unless there's big business incentive and big business can already scrape the shit out of google.

This is the weird thing about web-scraping. Big companies can get around protections quite easily - it's the small scripts and average users that get hurt by them. No one is going to tell you this because people would stop buying Cloudflare's "anti-bot 99% effective anti-ddos money saving package" which is complete bullshit.

Re: 9th Circuit holds that scraping a public website does not violate the CFAA [pdf]

#166

Earlier quoted context omitted.

Ahh so if a company leaks data it's the viewer's fault, not the companies?

Nobody leaked any data here. These were public profiles that were "controlled" by a robots.txt file. The judge appears to question whether robots.txt is sufficient to prevent scraping, or if a proper authorization step would be required. The best real-world analogy I can come up with... I post a No Trespassing sign on my garden, but don't fence/gate the property. Is it ok to access the property and take my tomatoes?…

Where I live in order to be trespass it has to be an enclosed space. If you don’t fence your property then it is not tesspass.

Re: 9th Circuit holds that scraping a public website does not violate the CFAA [pdf]

#167

Earlier quoted context omitted.

Does this prevent Google from returning captchas if you use a robot to scrape the search result pages, as they currently do?

I mean it should. That is also a huge anti-competitive action that just isn't pursued by anyone yet: google makes money of scraping and denies scrapers scraping them - that's all sorts of messed up. The problem is that someone would have to sue google first and no one will do that unless there's big business incentive and big business can already scrape the shit out of google. This is the weird thing about web-scrapi…

Google only scrapes sites that allow it by their robots.txt file so I don’t think their policy is as hypocritical as you are making it sound.

Re: 9th Circuit holds that scraping a public website does not violate the CFAA [pdf]

#168

Earlier quoted context omitted.

Does this prevent Google from returning captchas if you use a robot to scrape the search result pages, as they currently do?

I mean it should. That is also a huge anti-competitive action that just isn't pursued by anyone yet: google makes money of scraping and denies scrapers scraping them - that's all sorts of messed up. The problem is that someone would have to sue google first and no one will do that unless there's big business incentive and big business can already scrape the shit out of google. This is the weird thing about web-scrapi…

Google respects robots.txt, so it might be hard to prove that they are accessing websites without their implied consent.

Also, their own robots.txt contains "Disallow: /search". So, there is arguably no inconsistency, either.

But, what does this new ruling mean for robots.txt?

Re: 9th Circuit holds that scraping a public website does not violate the CFAA [pdf]

#169

Considering the kind of private scraping and selling tactics LinkedIn has been chronically guilty of (and not just the ordinary "growth hack" stuff: "LinkedIn violated data protection by using 18M email addresses of non-members to buy targeted ads on Facebook" [1]), it's satisfying to see LinkedIn lose this. [1] https://techcrunch.com/2018/11/24/linkedin-ireland-data-prot...

I feel like this is a really common theme I've seen several times. Something like "Music Lyric site X sues Google for embedding their lyrics in the results directly" which is funny because site X got the lyrics by scraping them from other sites. Plus Google only exists from scraping content, but I believe their TOS includes "don't scrape our content". I find it really funny that the scrapers are battling scrapers - l…

Google respects robots.txt, so it's arguably not the same as scraping a website without their consent.

Re: 9th Circuit holds that scraping a public website does not violate the CFAA [pdf]

#170

Earlier quoted context omitted.

Not allowing the CFAA to be (ab)used to attempt to make scraping illegal makes sense. However, how is it reasonable to force a web site to serve its contents to a third-party company, without being allowed to make a decision whether to serve it or not? Serving the web site costs money, and the scraper surely isn't going to generate ad income...

> However, how is it reasonable to force a web site to serve its contents to a third-party company, without being allowed to make a decision whether to serve it or not? Your statement makes absolutely no sense. That's not how internet works. If you serve something publicly you don't get to cherry pick who sees it. Not only it makes no sense technically it's also a huge anti-competitive case.

Of course you get to choose. You can reject requests based on their user agent, their IP address, the owner or likely geographic location of the IP address, and many other possibilities.
Post reply on HN