Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

9th Circuit holds that scraping a public website does not violate the CFAA [pdf]

cdn.ca9.uscourts.gov

Post reply on HN
Print view View original

81–90 of 293 posts

Re: 9th Circuit holds that scraping a public website does not violate the CFAA [pdf]

#81

Post by tanderson92 »

dickjocke wrote: post #78
meritt wrote: post #64

Earlier quoted context omitted.

If you have to login to access the data, that data isn't publicly available, and unlikely to be protected by this decision. Page 31 of the filing specifically differentiates this case from one regarding Facebook: "While Power Ventures was gathering user data that was protected by Facebook’s username and password authentication system, the data hiQ was scraping was available to anyone with a web browser."

Wait, doesn't Linkedin have pretty strong authentication safeuguards to view user profiles. if you google someone and click their linkedin without being logged in to LinkedIn, you're always directed to sign up or log in to it. Twitter is the only example I can think of among the large social media sites that doesn't require you to be logged in to see profiles

Even twitter requires you to be logged in to view "replies" on a user's profile.

Re: 9th Circuit holds that scraping a public website does not violate the CFAA [pdf]

#82

Post by colechristensen »

inbland wrote: post #72

What if hiQ were to then go and sell the information that's been scraped from public profiles? Would this be considered illegal?

I would assume the data would still be covered by copyright meaning they could use that data and maybe create and sell derivative works, but not just scrape and publish.

Re: 9th Circuit holds that scraping a public website does not violate the CFAA [pdf]

#83

Post by [deleted] »

mirimir wrote: post #79
freehunter wrote: post #37

Earlier quoted context omitted.

I have a scraper for a site that used to offer an API for their publicly available site but removed the API with no warning. The info is still available to the general public, but only through their website. I created a scraper for the public page, but shortly after they switched to loading some public information through Javascript so my HTML scraper couldn't see it anymore. I ended up having to write an application…

Chrome headless does that really well. > google-chrome --headless --run-all-compositor-stages-before-draw --virtual-time-budget=25000 --print-to-pdf='foo.pdf' URL Edit: Plus > pdftotext -raw foo.pdf

[deleted]

Re: 9th Circuit holds that scraping a public website does not violate the CFAA [pdf]

#84

Post by manfredo »

tptacek wrote: post #75
manfredo wrote: post #52

Earlier quoted context omitted.

I think the issue is not "negligence of the victim" so much as, "took no steps to make the information private". If I don't lock my door and someone goes into my house, opens my filing cabinet, and copies my financial info then they're still a criminal. They took information from a place that was unambiguously meant to be private. But if I staple a copy of my financial info to a telephone pole, surely those who read…

To take the case at issue, when your immediate reaction to discovering an unprotected URL is to scrape it, discuss on an IRC channel how you're going to monetize it, and then go to the media to announce your security vulnerability discovery, you are going to find it difficult to make the argument that you believed you had authorized access.

By that same line of reasoning, one could argue that changing your url parameter in that twitter chatroom website is a privilege escalation attack that allows users to access protected information.

Absence of authentication means all access is authorized, otherwise just typing in random urls is a crime.

Re: 9th Circuit holds that scraping a public website does not violate the CFAA [pdf]

#85

Post by supernova87a »

This case is so ridiculous on multiple fronts that although this procedural ruling (injunction) seems technically correct (to allow the case to proceed to actual court), it could just as well have been thrown out with no difference in or ultimate harm to the parties.

First, LinkedIn makes the claim that its users have a right to privacy against scraping by such a 3rd party. That's laughable. As the court saw, their whole business model is made on people sharing their profiles broadly and mostly to the public.

Secondly, HiQ claims that LinkedIn's efforts to stop it from using the data are tortious interference. That's bold -- suppose someone is taking your assets (you believe illegally) and selling them to others -- can you imagine the gall that the person taking your assets can sue you for interfering with their subsequent sale of your assets?

Finally, that LinkedIn resorted to using the computer fraud and anti-terrorism statutes to make their argument is ridiculous.

So much craziness to go around. I would've just tossed the case, but I guess there is the whole bit about due process... Maybe HiQ will fail anyway at the next substantive trial, but what a waste of time.

Re: 9th Circuit holds that scraping a public website does not violate the CFAA [pdf]

#86

Post by geggam »

Pfhreak wrote: post #53
freehunter wrote: post #37

Earlier quoted context omitted.

I have a scraper for a site that used to offer an API for their publicly available site but removed the API with no warning. The info is still available to the general public, but only through their website. I created a scraper for the public page, but shortly after they switched to loading some public information through Javascript so my HTML scraper couldn't see it anymore. I ended up having to write an application…

> There is no reason why your page should refuse to load plain text without Javascript enabled. Sure there is. You prefer writing javascript and you want to serve your site through a CDN. You might not think that's a good reason, but that's certainly a reason.

Until the ADA comes along and demands you create an accessible to the blind site.

I've often wondered when the laws would start to be applied and I think its coming

Re: 9th Circuit holds that scraping a public website does not violate the CFAA [pdf]

#87

Post by slg »

dx87 wrote: post #34

Would that ruling mean that sites could no longer refuse to show content based on how they're accessed? For example, sites that won't load if the browser is in headless mode, or sites that depend on javascript as a way of blocking wget/curl.

Captchas would be another technology that might be fall under that "technical barriers" terminology. I don't do much scraping, but I think most of us would enjoy never having to "identify the traffic lights in these photos".

Re: 9th Circuit holds that scraping a public website does not violate the CFAA [pdf]

#88

Post by tptacek »

manfredo wrote: post #84
tptacek wrote: post #75

Earlier quoted context omitted.

To take the case at issue, when your immediate reaction to discovering an unprotected URL is to scrape it, discuss on an IRC channel how you're going to monetize it, and then go to the media to announce your security vulnerability discovery, you are going to find it difficult to make the argument that you believed you had authorized access.

By that same line of reasoning, one could argue that changing your url parameter in that twitter chatroom website is a privilege escalation attack that allows users to access protected information. Absence of authentication means all access is authorized, otherwise just typing in random urls is a crime.

You can indeed argue that. Typing random URLs can indeed be a crime.

Re: 9th Circuit holds that scraping a public website does not violate the CFAA [pdf]

#89

Post by henryfjordan »

jolmg wrote: post #76
CosmicShadow wrote: post #50

Even if LinkedIn loses and scrapers can no longer be blocked, they still just switched to putting all profiles behind an authwall, or at least it's very hard to not get an authwall. So could HiQ even carry on if they won anyway?

I'm not very familiar with neither LinkedIn nor HiQ, but what would be the problem with logging in before scraping?

The reason the pages are public to begin with is that Google will only scrape public pages for search indexing. LinkedIn wants to provide the pages ONLY to google, so they tried telling hiQ to stop scraping without any physical blockers (so as to not impede google's scraper).

If LinkedIn loses this case they (and others) might try to get Google to change their policy (either use auth or some whitelisted IP addresses or something).

Re: 9th Circuit holds that scraping a public website does not violate the CFAA [pdf]

#90

Post by [deleted] »

txcwpalpha wrote: post #49
codedokode wrote: post #48

Earlier quoted context omitted.

this is bad comparison because when scraping a site, you don't cross any borders, you just send and receive information. You can compare this to a phone call or to talking to someone.

A website or server is property, just like land is. Accessing it is no different than accessing any other piece of property. Opening a website is, for all intents and purposes, the same as crossing a border. To take it a step further, the information on said website is also personal property, and accessing the information without permission is also trespassing. Specifically, this is called trespass to chattels [1] (t…

[deleted]
Post reply on HN