71–80 of 293 posts
This is actually bad, would not it be better if sites would be allowed to block crawlers? I don't see what is the legal basis for forbidding to ban scrapers. Is there a law that a site must serve pages for anyone?
This action does more than that. The court left the preliminary injunction against LinkedIn in place: "The district court granted hiQ’s motion. It ordered LinkedIn to withdraw its cease-and-desist letter, to remove any existing technical barriers to hiQ’s access to public profiles, and to refrain from putting in place any legal or technical measures with the effect of blocking hiQ’s access to public profiles." So Lin…
Leaving the injunction in place is insane and a huge oversight. It amounts to making web pages carriers that cannot select who they serve. It should have said only that there is nothing judicially wrong with scraping but also not limited the rights of a service.
Earlier quoted context omitted.
> There is no reason why your page should refuse to load plain text without Javascript enabled. Sure there is. You prefer writing javascript and you want to serve your site through a CDN. You might not think that's a good reason, but that's certainly a reason.
What does a CDN has to do with it?
As soon as your application bundle is rendered on your servers dynamically then only part of your site can be delivered via CDN.
Basically going all-JS gives you an app model where your sever side code doesn’t even know or care about HTML or the web or whatever. It just pushes JSON or whatever around and it largely client independent.
Great model when you need to support iOS, Android, web, desktop.
Earlier quoted context omitted.
Yes. That's the general rule--negligence of a victim does not negate the culpability of the criminal. "It was easy to commit the crime" is not a defense. If you find yourself with access to something you think you're not supposed to have access to, you're supposed to do the right thing.
I think the issue is not "negligence of the victim" so much as, "took no steps to make the information private". If I don't lock my door and someone goes into my house, opens my filing cabinet, and copies my financial info then they're still a criminal. They took information from a place that was unambiguously meant to be private. But if I staple a copy of my financial info to a telephone pole, surely those who read…
Even if LinkedIn loses and scrapers can no longer be blocked, they still just switched to putting all profiles behind an authwall, or at least it's very hard to not get an authwall. So could HiQ even carry on if they won anyway?
Earlier quoted context omitted.
> There is no reason why your page should refuse to load plain text without Javascript enabled. Sure there is. You prefer writing javascript and you want to serve your site through a CDN. You might not think that's a good reason, but that's certainly a reason.
What does a CDN has to do with it?
For many SPA's the only actual html is a header, container div, and a call to the app's js. Ignoring the header there might only be say half a dozen lines total.
Earlier quoted context omitted.
Scrapers generally, sure. Not sure about scrapers on other social-media sites like Facebook, though. The question being: if just having access to the network isn't enough to grant you access to the data of a specific profile, but instead you have to aggregate samples from a bunch of people in the network in order to see "through their eyes" to the data on the profiles of their friends and friends-of-friends, is that…
If you have to login to access the data, that data isn't publicly available, and unlikely to be protected by this decision. Page 31 of the filing specifically differentiates this case from one regarding Facebook: "While Power Ventures was gathering user data that was protected by Facebook’s username and password authentication system, the data hiQ was scraping was available to anyone with a web browser."
Twitter is the only example I can think of among the large social media sites that doesn't require you to be logged in to see profiles
Would that ruling mean that sites could no longer refuse to show content based on how they're accessed? For example, sites that won't load if the browser is in headless mode, or sites that depend on javascript as a way of blocking wget/curl.
I have a scraper for a site that used to offer an API for their publicly available site but removed the API with no warning. The info is still available to the general public, but only through their website. I created a scraper for the public page, but shortly after they switched to loading some public information through Javascript so my HTML scraper couldn't see it anymore. I ended up having to write an application…
> google-chrome --headless --run-all-compositor-stages-before-draw --virtual-time-budget=25000 --print-to-pdf='foo.pdf' URL
Edit: Plus
> pdftotext -raw foo.pdf