Live data from Hacker News

LinkedIn: It’s illegal to scrape our website without permission

arstechnica.com

181–190 of 303 posts

Re: LinkedIn: It’s illegal to scrape our website without permission

#181
post #100

Earlier quoted context omitted.

Hmm, then that becomes a copyright issue I suppose.

The type of data being discussed here (factual data about people) cannot be copyrighted - i.e. the fact John Doe is a Software Engineer for ACME Inc is not copyrightable.

Compilations can sometimes be protected.

Re: LinkedIn: It’s illegal to scrape our website without permission

#182

Unpopular opinion: when you make a HTTP request you're asking the server to give you information. The server has the right to say no. IMHO, LinkedIn doesn't have a right to stop scraping after the fact, but they have the right to take technical steps to stop scrapers from accessing their site.

I don't think you've characterized this accurately. When you make an HTTP request to LinkedIn you are accessing their service. There is a long history of this relationship, you plug your house into the sewer line and you connect to the sewer service. You connect to the power pole and connect to the electricity service. You connect to the telephone pole and connect to the telephone service. Every service has "terms of…

(I am not a lawyer.) As far as I understand the legal precedents involved, random terms of services for websites are not effective in this scenario as the public profiles do not require having any account or other relationship. This actually went to court, and because Zappos didn't force users to click through a terms of service to access their service, the terms of service was invalid.

As for their ability to control what you do with the information: there might be a limited license on the data granted from users to LinkedIn that is not transferrable, so maybe you couldn't build a service that redistributed that information, but I don't see why obtaining and holding it would be illegal.

As for the analogies to power and telephone and such, those are built on property owned by a local government and there are usually other extra laws related to them: it isn't due to some common law position that you can't mess with their stuff. Here, I am not a lawyer, but I am a government official with a particular interest in sewage; here is a link to the sewer use ordinances form our local sanitation district: pay particular attention to 2.03.

http://goletawest.org/wp-content/uploads/2012/04/Ordinance-N...

Re: LinkedIn: It’s illegal to scrape our website without permission

#183

Unpopular opinion: when you make a HTTP request you're asking the server to give you information. The server has the right to say no. IMHO, LinkedIn doesn't have a right to stop scraping after the fact, but they have the right to take technical steps to stop scrapers from accessing their site.

Isn't the server giving you permission to view the data? Not give?

EG, if it returns an image - it doesn't imply I can use the image anywhere I want.

Re: LinkedIn: It’s illegal to scrape our website without permission

#184

Earlier quoted context omitted.

I don't think you've characterized this accurately. When you make an HTTP request to LinkedIn you are accessing their service. There is a long history of this relationship, you plug your house into the sewer line and you connect to the sewer service. You connect to the power pole and connect to the electricity service. You connect to the telephone pole and connect to the telephone service. Every service has "terms of…

When you make a connection to the city sewers or to the power company, there is some kind of pre-connection step where the terms are presented and you agree to those terms. With HTTP and LinkedIn, there is no such step. There's no pre-connection agreement. LinkedIn could present such an agreement on first connection, but they do not.

You agree to the terms of service when you sign up.

If you're talking about making anonymous requests to their service, they only allow a few of those before they stop showing you profiles. If you circumvent that protection, it's a bit more like hooking a cable up to a power line (illegal) or dumping your commercial waste in the sewer (illegal).

Re: LinkedIn: It’s illegal to scrape our website without permission

#185

Unpopular opinion: when you make a HTTP request you're asking the server to give you information. The server has the right to say no. IMHO, LinkedIn doesn't have a right to stop scraping after the fact, but they have the right to take technical steps to stop scrapers from accessing their site.

I don't think you've characterized this accurately. When you make an HTTP request to LinkedIn you are accessing their service. There is a long history of this relationship, you plug your house into the sewer line and you connect to the sewer service. You connect to the power pole and connect to the electricity service. You connect to the telephone pole and connect to the telephone service. Every service has "terms of…

I disagree with your analogy. To me, the key word in "HTTP request" is request. A request is something that can be granted or not.

Re: LinkedIn: It’s illegal to scrape our website without permission

#186

Earlier quoted context omitted.

I don't think you've characterized this accurately. When you make an HTTP request to LinkedIn you are accessing their service. There is a long history of this relationship, you plug your house into the sewer line and you connect to the sewer service. You connect to the power pole and connect to the electricity service. You connect to the telephone pole and connect to the telephone service. Every service has "terms of…

When you make a connection to the city sewers or to the power company, there is some kind of pre-connection step where the terms are presented and you agree to those terms. With HTTP and LinkedIn, there is no such step. There's no pre-connection agreement. LinkedIn could present such an agreement on first connection, but they do not.

That argument has been tried in a variety of ways and been shot down in court repeatedly. (there are parallels to tenants not agreeing to the terms of their internet connection where the landlord provided it).

LinkedIn has two things that they do which protect them; First, they specify they disallow access in their robots.txt file. While not a binding agreement per se it is the default mechanism that is accepted by the community for apriori identifying whether or not automated access is possible. Second, when they detect an access pattern that violates their terms of service they actively block the access proactively notify the source of the violation.

The sad truth is that web scraping has been around since the very beginnings of the Web back in 1993 and this question has been litigated in every way that you might choose to argue it, the body of case law is enough to fill at least two volumes in the reference section of the library.

There is no legal or ethical basis for scraping the web without permission. And if it isn't explicitly allowed by a site the presumption is that it is disallowed (no 'open door' exception).

Re: LinkedIn: It’s illegal to scrape our website without permission

#187

Unpopular opinion: when you make a HTTP request you're asking the server to give you information. The server has the right to say no. IMHO, LinkedIn doesn't have a right to stop scraping after the fact, but they have the right to take technical steps to stop scrapers from accessing their site.

Legally, I think scrappers should respect robots.txt

Re: LinkedIn: It’s illegal to scrape our website without permission

#188
post #60

Earlier quoted context omitted.

Does the pool have any recourse if you proceed to bypass the ban? Do you have to re-enter to pool to bypass it, or does sending in confederates with their own badges to continue your work also bypassing the ban? How about sending in new motorized cars? The analogy is starting to break down, but I think it's still instructive for the problem of applying a simple first principles approach.

There is a legal concept known as "attractive nuisance"[1]. If I have a pool and neighborhood kids come to play and someone gets hurt, it's my fault. Even if I was away from my house and never gave permission (or explicitly forbade them from swimming), if I don't have proper access controls in place, the courts say it is too tempting for the neighbors to just come over and swim. I need to put up a locking gate to kee…

Laws like that are ridiculous. You can see that by looking at how the reasoning does not expand to certain areas.

For example, if a woman walks down a dark alley wearing short skirts and gets raped, it isn't her fault. I mean can you imagine if we said "well, she was just an attractive nuisance!" The judge would throw the book at you.

Re: LinkedIn: It’s illegal to scrape our website without permission

#189
post #155

When I make my CV public on LinkedIn I expect it to be public. Even for bots. That is the whole point of 'public.'

LinkedIn's purpose is not to help you as a worker. It's purpose is to scrape together as much personal information about people as possible to make money from it.

Likewise Facebook doesn't exist to help connect people together to create a more personal, connected world. It exists to connect people so that they can get the users to share as much personal information as possible, so they can profit from that information.

It's always important to remember that on social media you are the product. You have to weigh what you benefits it is really providing vs. what you are giving up.

Re: LinkedIn: It’s illegal to scrape our website without permission

#190

Earlier quoted context omitted.

I don't think you've characterized this accurately. When you make an HTTP request to LinkedIn you are accessing their service. There is a long history of this relationship, you plug your house into the sewer line and you connect to the sewer service. You connect to the power pole and connect to the electricity service. You connect to the telephone pole and connect to the telephone service. Every service has "terms of…

I disagree with your analogy. To me, the key word in "HTTP request" is request . A request is something that can be granted or not.

Perhaps more clearly would be any HTTP request that LinkedIn believes is in violation of their terms of service will be denied. It can be hard to know when the first request arrives if it is someone scraping the site or not, but once it is clear that it is someone scraping they actively deny all future requests. If they could know that the request coming in was going to be a scrape and not a page view they would preemptively deny it.
Post reply on HN