Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Congrats! Web scraping is legal! (US precedent)

parsers.me

Post reply on HN
Print view View original

341–350 of 409 posts

Re: Congrats! Web scraping is legal! (US precedent)

#341

Post by munk-a »

Supermancho wrote: post #251
lovehashbrowns wrote: post #204

Earlier quoted context omitted.

Your analogy doesn't hold. Your backyard is private property. The data that LinkedIn publishes is intended for the public. That's why Google can index the pages and give you results from LinkedIn.

> Your analogy doesn't hold. It does, in the US. You're likely making an inconsistent comparison. Property ownership has nothing to do with visual access. You cannot legally be barred from casually (involuntarily) perceiving something. It's reasonable to put up physical barriers to reduce what is casually perceived. It's a very good analogy.

However it doesn't hold - as your neighbor I can't bar you from putting up a fence because it'll intrude on my view of your property... granted people try to do that _all the time_ but I think it's commonly understood that putting up a fence for privacy is allowed.

It's also not a great analogy for this case because another party is given continued easy access to view my backyard while the first party is denied - and the analogy breaks down here because, as a neighbor, I have no inherent right to view your private life at least as much as any of your other neighbors.

Re: Congrats! Web scraping is legal! (US precedent)

#342

Post by nickjj »

1MoreThing wrote: post #337
nickjj wrote: post #333

I guess this means the web is lawfully seen as belonging to the public which seems both good and bad, because while it might be publicly accessible as a whole, each individual site is privately owned. I'm pretty sure with any brick and mortar business, as a business owner you are legally allowed to disallow service to anyone without having to provide a reason. That would be the equivalent of allowing to block bots /…

Yeah, but this is slightly different. This is about a business owner trying to control who can look through the windows.

> Yeah, but this is slightly different. This is about a business owner trying to control who can look through the windows.

Is it any different tho?

I don't think there's a clean analogy for window shopping on the web.

If you visit example.com to take a look at the site then you are really inside of example.com's business / store front. Their servers are spending resources to serve your traffic. The doors have been opened.

I also think a brick and mortar business has the right to prevent you from (literally) window shopping too because if you enter the land that they own (or are renting) and they do not want to serve you then you are trespassing on private property.

Re: Congrats! Web scraping is legal! (US precedent)

#343

Post by mic47 »

I strongly disagree that not allowing scraping protection on social network is a good thing (reasons mostly from this thread https://news.ycombinator.com/item?id=22182144 )

What I would do on Linkedin side is:

1. Split public setting into 2 settings: public for everyone including scraping, and second choice that make it public for Linkedin users (aka banning scraping [1])

2. Everyone on Linkedin who have current setting public would go though dialog to choose which one, with default setting to ban scraping (which is hard to criticize, since that option is protecting privacy more).

3. If you allow scraping, the form should warn you about consequences, and make you to acknowledge them (which is again a good practice, if you want to expose your information to EVERYONE on the internet).

4. Now you have users consent to protect their data from scraping.

Sounds like dark pattern? Kind of yes, but for good thing and it's hard to argue against it, since it increases privacy.

[1] Idea here is that I can allow anyone on Linkedin to see my profile, whether it is potential employer, or someone who met me at the conference (i.e. to improve my Linkedin experience). But I do not want that data to be harvested, by third party for any reason (specially, because they would then use it to to send me spam / advertising / prefill my profile on different website / ...).

edit: formatting

Re: Congrats! Web scraping is legal! (US precedent)

#344

Post by thomasjames »

papln wrote: post #284
TrickyRick wrote: post #32

Earlier quoted context omitted.

Wasn't the point that they were scraping public profiles, i.e. they never signed up and thereby never signed any ToS?

ToS aren't signed. They are terms of service, not license agreements. ToS merely need to be disclosed, and the site cna decline service if you violate the terms. A license agreement may subject you to penalties for violating the agreement.

Thanks for the clarification.

Re: Congrats! Web scraping is legal! (US precedent)

#345

Post by dariosalvi78 »

Tactic wrote: post #213
kabacha wrote: post #11

The toxicity towards web-scraping is really what makes me lose hope in the current web. People want their data to be public and all of the benefits that comes with public data but then they want to chose who gets to see it - it's a complete and utter paradox. This precedent doesn't really mean much but is definitely step in the right direction.

The issue here for some, if not many, is a matter of scale. It is one thing if an end-user, whom I am trying to service, comes to my site and gets my publicly available data. Maybe I monetize with ads, maybe not. It doesn't matter, that is the audience I am trying to service, regardless of size. But when you scrape it my load goes up dramatically. A load I have to pay for. It is analogous to the privacy debates going…

The contention was not about load, it was about using the data.

Re: Congrats! Web scraping is legal! (US precedent)

#346

Post by divbzero »

In the past the CFAA has been wielded haphazardly and even maliciously [1]. As noted by others these cases are far from over, but hopefully we’re shifting towards a healthier balance between open web and private information.

[1]: https://en.wikipedia.org/wiki/Aaron_Swartz

Re: Congrats! Web scraping is legal! (US precedent)

#347

Post by [deleted] »

kaveh_h wrote: post #94

One question to those who dislike web scraping as they deem it infringes copyright laws. Given that Google scrapes LinkedIn public profiles and add data from it to it’d index and when presenting search results, is it then not discrimination that Microsoft tries to block HiQ and not google?

[deleted]

Re: Congrats! Web scraping is legal! (US precedent)

#348

Post by nickjj »

schwartzworld wrote: post #340
nickjj wrote: post #333

I guess this means the web is lawfully seen as belonging to the public which seems both good and bad, because while it might be publicly accessible as a whole, each individual site is privately owned. I'm pretty sure with any brick and mortar business, as a business owner you are legally allowed to disallow service to anyone without having to provide a reason. That would be the equivalent of allowing to block bots /…

> as an owner you are legally allowed to disallow service to anyone without having to provide a reason as a website owner you absolutely have the right to limit who does or doesn't visit your site, but that's up to you to enforce. Scraping is literally equivalent to reading a giant public billboard and writing what it says somewhere else. How could that be illegal?

> Scraping is literally equivalent to reading a giant public billboard and writing what it says somewhere else. How could that be illegal?

If you treat the website you're visiting as a privately owned business then you're trespassing on private property by scraping their site, assuming that site is trying to prevent that behavior.

If you don't treat the website as a privately owned business, then what do you classify it as? It can't be considered in the public domain because someone owns, operates and pays for the resources to make the site work. It is the site owner's private property IMO. By having the site public, they are inviting the world to check it out but they should have the right to disallow service.

Also, if the internet is supposed to be distributed and each site is an independent node on the system, how is that any different than an independently owned brick and mortar business operating in some location? In this case the physical world is "the internet".

It's not a clear cut thing, and I hate the idea of censorship, but I can't see this case's outcome becoming the norm. There's too many loop holes. Like, is going to your site 50 thousand times a second hoping to get new information from the public billboard a legal move?

Or, to put it another way, I'm pretty sure if you were able to mind control people and you commanded a billion people to flood a physical business so that it could not operate and serve its customers this would be quickly seen as an unlawful move in the physical world. You would probably get shut down by the state or government too for disrupting service for neighboring businesses and citizens.

Re: Congrats! Web scraping is legal! (US precedent)

#349

Post by shkkmo »

Ugh, this is yet another poorly written coverage of a court case that is more complicated than simple snippets and one liners allow.

This absolutely does not say that web scraping is legal (the ruling specifically notes other ways that it could be illegal).

It doesn't even say for sure that web scraping isn't illegal under the CFAA. This is a ruling about a preliminary injuction and as such all it determines is that HiQ has a good enough arguement and valid enough concerns that a preliminary injuction is warranted until LinkedIn wins a suit against HiQ to force them to stop scraping.

Take the time, and read the linked ruling. Don't trust poorly written and researched articles from companies that stand to gain from a particular interpretation.

Re: Congrats! Web scraping is legal! (US precedent)

#350

Post by hombre_fatal »

JackRabbitSlim wrote: post #222
powrtoch wrote: post #38

"HiQ only takes information from public LinkedIn profiles. By definition, any member of the public has the right to access this information. Most importantly, the appeals court also upheld a lower court ruling that prohibits LinkedIn from interfering with hiQ’s web scraping of its site." Surely I'm not reading this correctly. This would seem to suggest that websites are not legally allowed to prevent bots from crawli…

https://www.eff.org/cases/hiq-v-linkedin LinkedIn aint the victim here...

This is the part I disagree with:

> hiQ also asked the court to prohibit LinkedIn from blocking its access to public profiles while the court considered the merits of its request. hiQ won a preliminary injunction against LinkedIn in district court, and LinkedIn appealed.

Whether LinkedIn is the good guy or bad guy here doesn't matter when the decision creates precedence for the rest of us.

Surely a healthier precedent is that we can respond arbitrarily to requests and have no obligation to the requester. So what if I want to randomize the html structure on every request or block requests from Tor because 100% of them are abuse? Can someone take me to court on the grounds that either is effectively "blocking" their scraping syndicate? Why not?

I feel like once CFAA is off the table (which I do agree with), the cat and mouse game is a fair middle ground. Keep web scraping a sport!

Post reply on HN