Does this have implications for other scraping? (E.g. https://techcrunch.com/2019/05/20/instagram-influencer-celeb... )
Or this other case https://gizmodo.com/instagram-boots-ad-partner-hyp3r-for-rep...
9th Circuit holds that scraping a public website does not violate the CFAA [pdf]
171–180 of 293 posts
Re: 9th Circuit holds that scraping a public website does not violate the CFAA [pdf]
#172Earlier quoted context omitted.
Not allowing the CFAA to be (ab)used to attempt to make scraping illegal makes sense. However, how is it reasonable to force a web site to serve its contents to a third-party company, without being allowed to make a decision whether to serve it or not? Serving the web site costs money, and the scraper surely isn't going to generate ad income...
> However, how is it reasonable to force a web site to serve its contents to a third-party company, without being allowed to make a decision whether to serve it or not? Your statement makes absolutely no sense. That's not how internet works. If you serve something publicly you don't get to cherry pick who sees it. Not only it makes no sense technically it's also a huge anti-competitive case.
It makes sense and it is how the internet works. Servers cherry pick who sees their content all the time. Scrapers are often blocked, as are entire IP address ranges. Things like Selenium server scrapers can be (approximately) detected and often are denied access.
I’m not sure about being anti-competitive. Serving a website is an action in which you open up your resources for others to access. My friend runs an open source stock market tracking website for free. He started getting hit with scrapers from big hedge funds and fintech companies a couple of months back. This costs him around $50-100 a month to serve all of these scrapers.
Re: 9th Circuit holds that scraping a public website does not violate the CFAA [pdf]
#173> LinkedIn has taken steps to protect the data on its website from what it perceives as misuse or misappropriation. The instructions in LinkedIn’s “robots.txt” file—a text file used by website owners to communicate with search engine crawlers and other web robots—prohibit access to LinkedIn servers via automated bots, except that certain entities, like the Google search engine, have express permission from LinkedIn f…
>except that certain entities, like the Google search engine, have express permission from LinkedIn for bot access
how does this work technically? i just tried crawling a friend's profile using curl and set my user agent to Google's bot and it still was blocked.
Re: 9th Circuit holds that scraping a public website does not violate the CFAA [pdf]
#174Earlier quoted context omitted.
A friend of mine from grad school was very involved in legal issues related to cfaa stuff. According to him, weev really got screwed because he failed "the punk test", which discouraged lawyers from wanting to use him as a test case.
Curious question: what is "the punk test"?
"will this person be negatively perceived by a jury of their peers, or, especially, a jury of 3 federal judge? And, will this person say dumb shit that torpedos my case for reasons other than its merits"
Re: 9th Circuit holds that scraping a public website does not violate the CFAA [pdf]
#175This action does more than that. The court left the preliminary injunction against LinkedIn in place: "The district court granted hiQ’s motion. It ordered LinkedIn to withdraw its cease-and-desist letter, to remove any existing technical barriers to hiQ’s access to public profiles, and to refrain from putting in place any legal or technical measures with the effect of blocking hiQ’s access to public profiles." So Lin…
"If this case is eventually decided in favor of hiQ, scrapers can no longer be blocked." If the case is decided in favour of hiQ, then, absent an injunction , what would prevent a website from blocking a scraper? Maybe the website could still block unless and until the scraper gets her lawyers to file an injunction. Another interpretation is that if hiQ wins, then in the 9th Circuit's jurisdiction websites serving pu…
Scraper bots have genders now?
Re: 9th Circuit holds that scraping a public website does not violate the CFAA [pdf]
#176Earlier quoted context omitted.
The real issue here is somewhere between both you and GP. What is required to trigger the CFAA? Does accessing a page the site owner doesn't want you to violate the CFAA or do you need to hack through access controls?
As a real-world analogue: you can indeed be guilty of trespassing on someone's property even if you don't have to jump over any fences or pick any locks to get there. In some places, they don't even have to have a "no trespassing" sign. Simply being present on someone else's property without an invitation from them is illegal, and no, an open door does not count as an invitation.
> an open door does not count as an invitation.
But if you have a someone living there who lets anyone in if they ask, it would be pretty hard to argue they are trespassing.
A user agent must ask for every page with an http request. If the server responds with 200 OK, it’s pretty hard to argue that it isn’t letting (or even inviting) you in.
Re: 9th Circuit holds that scraping a public website does not violate the CFAA [pdf]
#177Earlier quoted context omitted.
I would assume the data would still be covered by copyright meaning they could use that data and maybe create and sell derivative works, but not just scrape and publish.
Pictures and such are covered by copyright, but mere facts are not (at least in the US): https://en.wikipedia.org/wiki/Feist_Publications,_Inc.,_v._R... .
Collections of facts are copyrightable. Which is why you can go out and make a map of your local area but you may not copy the data from google maps. You may end up with the exact same data and that is ok because you both copied the same facts but if there is a mistake on google maps (Perhaps placed as a trap) then you can be caught if your map has the same mistake.
Re: 9th Circuit holds that scraping a public website does not violate the CFAA [pdf]
#178> LinkedIn has taken steps to protect the data on its website from what it perceives as misuse or misappropriation. The instructions in LinkedIn’s “robots.txt” file—a text file used by website owners to communicate with search engine crawlers and other web robots—prohibit access to LinkedIn servers via automated bots, except that certain entities, like the Google search engine, have express permission from LinkedIn f…
>except that certain entities, like the Google search engine, have express permission from LinkedIn for bot access how does this work technically? i just tried crawling a friend's profile using curl and set my user agent to Google's bot and it still was blocked.
IP range whitelisting? MASSL?
EDITED: corrected auto-correct.
Re: 9th Circuit holds that scraping a public website does not violate the CFAA [pdf]
#179Earlier quoted context omitted.
I feel like this is a really common theme I've seen several times. Something like "Music Lyric site X sues Google for embedding their lyrics in the results directly" which is funny because site X got the lyrics by scraping them from other sites. Plus Google only exists from scraping content, but I believe their TOS includes "don't scrape our content". I find it really funny that the scrapers are battling scrapers - l…
> Plus Google only exists from scraping content, but I believe their TOS includes "don't scrape our content". Yes. This is EXTREMELY frustrating. Of all companies to prevent scraping, Google is the most ironic. Especially since their goal is to organize the world's information, it shocks me that there's no way to get access to this organized information from machine to machine.
I d0n't really think it's inappropriate or ironic. I can easily imagine naive scrapers essentially DDOSing google.
Re: 9th Circuit holds that scraping a public website does not violate the CFAA [pdf]
#180Earlier quoted context omitted.
I'm confused. Why would a blind person be any less likely to use JavaScript?
To test for a blind persons ability to render your website a good method is a CLI browser. Neither the blind persons device or a CLI browser will render javascript
Why on earth would you think that? The screen readers tie into modern browsers like Safari or Chrome.