If I spammed people as much as LinkedIn I wouldn't be throwing stones. They completely ignore the law.
LinkedIn: It’s illegal to scrape our website without permission
221–230 of 303 posts
Re: LinkedIn: It’s illegal to scrape our website without permission
#222Earlier quoted context omitted.
LinkedIn takes plenty of technical precautions to block scraping. I’ve built bots that scrape them in the past, it’s surprisingly difficult as LinkedIn is very good at determining you’re a bot and blocking you. So it’s hard to argue that a service which is scraping LinkedIn is doing it without knowledge that they are going against LinkedIn’s wishes. Whether or not this is illegal is up to the courts to determine, and…
> So it’s hard to argue that a service which is scraping LinkedIn is doing it without knowledge that they are going against LinkedIn’s wishes. As far as I can tell, no one has made that argument, so I'm not sure why you feel the need to rebut it. I think it all pretty much boils down to this quote from the article: > LinkedIn's position disturbs Orin Kerr, a legal scholar at George Washington University. "You can't p…
> no one has made that argument, so I'm not sure why you feel the need to rebut it.
The title is "It’s illegal to scrape our website without permission". So that argument is implied in the headline, at least.
As for Orin Kerr, I'm sure he'd agree that there are private parts of the internet (my payment information being an obvious example). Just because something is deployed to the internet doesn't mean it is "published to the world" as he claims.
Re: LinkedIn: It’s illegal to scrape our website without permission
#223Earlier quoted context omitted.
I don't think you've characterized this accurately. When you make an HTTP request to LinkedIn you are accessing their service. There is a long history of this relationship, you plug your house into the sewer line and you connect to the sewer service. You connect to the power pole and connect to the electricity service. You connect to the telephone pole and connect to the telephone service. Every service has "terms of…
I disagree with your analogy. To me, the key word in "HTTP request" is request . A request is something that can be granted or not.
I understand the appeal of this argument, but there are clearly cases where a computationally valid request and response is illegal despite the fact that the server "chose" to satisfy the request. An obvious example would be any exploit where an attacker can construct a particular request and get access to someone else's private data.
Re: LinkedIn: It’s illegal to scrape our website without permission
#224Earlier quoted context omitted.
Actually, no. It's more like complaining about others selling tickets to view said painting from the sidewalk. HiQ repackages and sells data it scrapes from LinkedIn.
Actually, no. It's like taking pictures of the paintings from the street, and reselling those pictures. If they don't like that... that the painting down. Simple.
Although I agree with you in principle, I think your analogy is flawed. The nature of a web request is that you're asking in the first place, and the serve has to serve you. If you're taking a photo, you're not asking anything; data transfer has a cost, too.
Re: LinkedIn: It’s illegal to scrape our website without permission
#225Sometimes I wish we could start over.
Re: LinkedIn: It’s illegal to scrape our website without permission
#226Earlier quoted context omitted.
> So it’s hard to argue that a service which is scraping LinkedIn is doing it without knowledge that they are going against LinkedIn’s wishes. As far as I can tell, no one has made that argument, so I'm not sure why you feel the need to rebut it. I think it all pretty much boils down to this quote from the article: > LinkedIn's position disturbs Orin Kerr, a legal scholar at George Washington University. "You can't p…
> no one has made that argument, so I'm not sure why you feel the need to rebut it. The title is "It’s illegal to scrape our website without permission". So that argument is implied in the headline, at least. As for Orin Kerr, I'm sure he'd agree that there are private parts of the internet (my payment information being an obvious example). Just because something is deployed to the internet doesn't mean it is "publis…
"Published to the world" is a bit off; I'd compare it more to recording telephone conversations with corporations' public service numbers. They always tell you that you are being recorded (for "training and quality purposes") so they give up their right to not be recorded in turn.
If a company paid a million people to call into an information service hotline and each request one fact from it—and then the company recorded and compiled the answers into their own database to start their own information service—is that illegal?
Re: LinkedIn: It’s illegal to scrape our website without permission
#227Earlier quoted context omitted.
> They shouldn't have to serve that content to people who aren't actually their users. So put it behind a password. It's not reasonable to expect to get only the benefits of publicly-searchable data without any of the drawbacks.
The policy is also harmful to innovation. If a Google-like competitor started, all Google would have to do to crush them is demand big-name sites formally prohibit the competitor from accessing their content or risk being delisted. And magically, it becomes impossible/illegal to build a duck-duck-go.
Isn't that monopolistic behaviour? I suspect that it might violate some laws, though I don't know anything in particular. On the other hand, why couldn't DDG just do what Google does and say "robots.txt prevents us from getting a description for this site"?
On one hand I'm against the cartel beahviour of Google doing something like that, but on the other hand, if Google asks and the other company agrees to block DDG, why shouldn't that be allowed?
Re: LinkedIn: It’s illegal to scrape our website without permission
#228Earlier quoted context omitted.
> So it’s hard to argue that a service which is scraping LinkedIn is doing it without knowledge that they are going against LinkedIn’s wishes. As far as I can tell, no one has made that argument, so I'm not sure why you feel the need to rebut it. I think it all pretty much boils down to this quote from the article: > LinkedIn's position disturbs Orin Kerr, a legal scholar at George Washington University. "You can't p…
> no one has made that argument, so I'm not sure why you feel the need to rebut it. The title is "It’s illegal to scrape our website without permission". So that argument is implied in the headline, at least. As for Orin Kerr, I'm sure he'd agree that there are private parts of the internet (my payment information being an obvious example). Just because something is deployed to the internet doesn't mean it is "publis…
That is not Kerr's argument. A key point (which he has developed to a very detailed degree, as both a law professor and actively defending people, such as Weev) is the access controls in place.
He's not just bloviating; before you disagree with him, reviewing his arguments is worthwhile. (I do disagree with him in part, and agree with his reasoning but don't like the outcomes in part, but in any case, he's a pretty accomplished lawyer, and I'm not any kind of lawyer, so there's that.)
Re: LinkedIn: It’s illegal to scrape our website without permission
#229Earlier quoted context omitted.
> there is no legal way to scrape a web site without its explicit permission to do so. Google scrapes several of my sites and I've never given Google explicit permission to do so.
If your robots.txt file is /allow then you did. If you have no robots.txt file then it's an open question. If you put a /deny into your robots.txt file Google will stop scraping your site. The implicit contract is that you let them scrape because you want to show up in their search results which will send you traffic. If you don't care about Google traffic then set /deny in your robots.txt and get back the bandwidth…
> If you have no robots.txt file then it's an open question.
Only for definitions of explicit I must be unfamiliar with.
If the presence of a robots.txt makes one's intent for a given resource explicit one way or the other, the lack of one (and the lack of some communication in some other channel) must mean there is no explicit permission.
Re: LinkedIn: It’s illegal to scrape our website without permission
#230Earlier quoted context omitted.
I disagree with your analogy. To me, the key word in "HTTP request" is request . A request is something that can be granted or not.
True, but in making the request, you will provide information on who is making that request. If you say, "I am a bot!", and they grant you permission, your request is legal. But if you say, 'I am NOT a bot', like spoofing a browser's user agent string, but you are a bot, then you are requesting access under a pretense, in order to circumvent their terms of service. Kinda feels morally wrong, and illegal.
That argument works, insofar as it does, only for more recognizable bots and browsers. If I write a client of some sort that identifies itself as:
Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Snackmaster Pro/666.0.666
What do you do?
I also tell my browser to lie about what it is sometimes, due to sites that are malfunctioning, but whose owners choose to document the errors instead of fixing them with "Use Chrome" (or IE, or whatever) checks.
Is that 'kinda' illegal or morally wrong (two very different things)?
If so, that seems like a belief that all sorts of browser defaults are 'kinda' wrong and/or illegal to change. Javascript? Lying about installed fonts/screen dimensions/whatever? Refusing to keep nonsession cookies between sessions? That slope would seem to get pretty slippery...