Earlier quoted context omitted.
> If you have no robots.txt file then it's an open question. Only for definitions of explicit I must be unfamiliar with. If the presence of a robots.txt makes one's intent for a given resource explicit one way or the other, the lack of one (and the lack of some communication in some other channel) must mean there is no explicit permission.
That is correct, for what it was worth IBM's legal team came down on the side of 'assume deny' and Google was (at the time I was there) 'assume allow.'
LinkedIn: It’s illegal to scrape our website without permission
281–290 of 303 posts
Re: LinkedIn: It’s illegal to scrape our website without permission
#282Earlier quoted context omitted.
> But LinkedIn seems within their rights here. Congress writes bad laws all the time. So LinkedIn might be within their rights, but that doesn't mean they should have those rights. It's bad for innovation to allow for selective discrimination like this. LinkedIn is perfectly happy to allow Google, Yahoo, Bing, and many, many more companies to scrape their content and use it for personal profit. Giving them the option…
The question is, how would you better define hacking? To make the letter of the law match the spirit? "Exceeding authorised access" is an early attempt that predates modern internet usage. You can't just say "anything the computer lets you do, is legal" because code exploits are just a computer following the (poorly-written) instructions in its code.
In the 80s, it was reasonable to assume that connecting to some port on a remote machine owned by another person or company could constitute unauthorized access. But today, billions of people connect to ports on remote machines thousands of times a day for completely legitimate reasons, so it's reasonable to assume that data that can be accessed by just asking nicely over the internet is considered intended for public consumption.
It seems permissive, but I think that's a crucial component. If some company makes accidentally makes their S3 buckets public, it's completely unfair to say that accessing that information is illegal, especially when they are serving up other information in public S3 buckets which they want people to access.
Re: LinkedIn: It’s illegal to scrape our website without permission
#283Earlier quoted context omitted.
> The number of photos is irrelevant to the analogy Actually, the size of the data and the number of requests is very relevant. More data means more information, means more money. It also means more bandwidth and processing power required to process requests. You're not taking a photo of the bike, you're asking the bike to give you a photo of it. > it's just dishonest/misleading to pretend that copying data is ever a…
If LinkedIn are being that negatively affected by a single scraper, they should deal with it - block it, only allow a specific number of requests from an IP per day, anything that doesn't involve lawsuits. The problem is them trying to pretend that publicly visible content is really private if they say so, without them trying to protect it in any real way. "The hurt occurs when people benefit from the work the origin…
With this, I agree 100%.
> No, the straw man is pretending that a copy is the same as theft. Theft is theft because someone is depriving you of the original, not because you imagine you might have had more sales if the copy didn't exist. There's a reason why there are different words for different things, and pretending that a copy is the same as taking a physical object it a lie. Period.
That's just pedantry. The debate isn't between "copy" and "theft", it's between "theft" and "copyright infringement".
> But, you put the price up too high, so I opted not to buy it. Maybe borrow the CD from a friend, or listen to something else. Or, you decided I couldn't buy it in the format or region I wanted. There are real issues, but pretending that a copy = a lost sale is utter bull that's been debunked time and time again, yet is regularly repeated by people trying to inject emotional arguments instead of facts.
This is wrong on so many levels, I'm not sure there's any point in continuing this debate. Are you accusing me of using emotional blackmail instead of facts because I point out that "you can clone my work, but I can't clone my food"?
I'm not using myself as an example because I want pity. I'm doing it because it's easier in writing, and because I'm a software developer.
My work takes hours of hours of time and effort (not accounting the hours I spent in school). If it' ok for everyone to clone my work, I won't make any money from it. We still live in a society where goods and services are exchanged with money. I exchanged my hours of work for no money, but I can't exchange no money for basic living necessities such as food. There's no feelings involved here. In the current economy, work going in, and no food coming out is not a viable business model. And if nobody payed for digital content, there would be a lot less digital content.
> pretending that a copy = a lost sale is utter bull that's been debunked time and time again
This is another straw man. Whether or not an illegal copy is or isn't a lost sale is irrelevant. You don't have the right to make that copy in the first place. If everyone made illegal copies, there would be no sales. So then why should only some be entitled to illegal copies? There isn't a distinction between people who can make copies and people who must pay for copies, so either everyone must pay for copies or no one must pay for copies. That's how law and economy work. You can't make exceptions by yourself. Either everyone is allowed, or no one is allowed. And for digital content that is for sale, no one is allowed illegal copies. If laws are made that allow poor people to receive goods for free, these laws must address both digital and physical goods.
Re: LinkedIn: It’s illegal to scrape our website without permission
#284Earlier quoted context omitted.
There is a legal concept known as "attractive nuisance"[1]. If I have a pool and neighborhood kids come to play and someone gets hurt, it's my fault. Even if I was away from my house and never gave permission (or explicitly forbade them from swimming), if I don't have proper access controls in place, the courts say it is too tempting for the neighbors to just come over and swim. I need to put up a locking gate to kee…
"An unlocked car is too tempting for some people to just walk past and not take it." I am saddened by this.
Re: LinkedIn: It’s illegal to scrape our website without permission
#285Earlier quoted context omitted.
The website can't really kick you out though, it can only kick your agent out and you can trivially create a thousand more. The website can politely ask you to stop just like the pool, but it can't actually do anything if you ignore it.
Except block your IP address, or your user agent, or the pattern your software makes when it connects. Yes, that will cause potential issues for other people, which is why they tend not to do that, but if you trivially create a thousand more agents, and potentially trigger a degradation of service, how are you different to the people who block junctions at traffic lights? I'm not keen on inconveniencing people, and "…
That's why it might be reasonable for laws around this sort of thing to be different in the virtual world.
Re: LinkedIn: It’s illegal to scrape our website without permission
#286Earlier quoted context omitted.
>moral compass The issue I see with this is in 2 part. First, any issue that comes down to "moral compass" is inherently dangerous. We can find many examples of the simple concept that what to one person is Good is to another Evil. In this case, I think Linkedin shareholders would not appreciate calls to mess with the site, or with people having trouble jobhunting because the site is going down repeatedly due to DDOS…
everything comes down to a moral compass of some kind. Your comment expects some sort of objective measurement of good and evil, but I don't see any. The law can be, and is often, in the wrong. Most people seem to think this law (if it is held up in court) is wrong and should be changed. You might say, oh well, we have a democratic right to change or influence our laws. But a princeton study has found no correlation…
Re: LinkedIn: It’s illegal to scrape our website without permission
#287Earlier quoted context omitted.
To the extent to which that is the case, though, it isn't due to the terms of service; and that is also a case of how you are using the data for later, which is a separate question from the scraping and collection process: it is very clear to me that a search engine is operating on the legal equivalent of thin ice, particularly with details like snippets and synthesis ;P. Whether the CFAA applies (as indicated in thi…
> it is very clear to me that a search engine is > operating on the legal equivalent of thin ice, We may be saying similar things but from a metaphor I think of search engines operating on 'thick' ice. It has been litigated so much that there is a bevy of case law to refer to at all levels. Eric Goldman's blog used to have a pretty good list of the number of suits of various kind and the searchengine blog covered man…
Re: LinkedIn: It’s illegal to scrape our website without permission
#288Unpopular opinion: when you make a HTTP request you're asking the server to give you information. The server has the right to say no. IMHO, LinkedIn doesn't have a right to stop scraping after the fact, but they have the right to take technical steps to stop scrapers from accessing their site.
Have no doubt this will be unpopular, but I think LinkedIn is right. Most news sites publish to the world. But scraping a news site's content and monetizing it yourself is not ok. Legally, it violates intellectual property law. But laws aside, I assume most people would agree, if someone spent the time researching and writing an article, they should have the right to monetize it and nobody else. In this case, IP law…
Re: LinkedIn: It’s illegal to scrape our website without permission
#289Unpopular opinion: when you make a HTTP request you're asking the server to give you information. The server has the right to say no. IMHO, LinkedIn doesn't have a right to stop scraping after the fact, but they have the right to take technical steps to stop scrapers from accessing their site.
Have no doubt this will be unpopular, but I think LinkedIn is right. Most news sites publish to the world. But scraping a news site's content and monetizing it yourself is not ok. Legally, it violates intellectual property law. But laws aside, I assume most people would agree, if someone spent the time researching and writing an article, they should have the right to monetize it and nobody else. In this case, IP law…
Re: LinkedIn: It’s illegal to scrape our website without permission
#290Earlier quoted context omitted.
I wasn’t aware that robots.txt had any legal meaning. Am I wrong?
> Am I wrong? I hate to be the bearer of bad new but...maybe. A reading of the Computer Fraud and Abuse Act could make robots.txt legally enforceable. And given the government's approach to CFAA cases a very aggressive interpretation, under the right circumstances (for example, when it provides evidence that the scraper knew that scrapint was not authorized), seems like a real possibility. Among the many other things…
The interesting question is are you violating the CFAA in a way that will cause the executive branch to exercise their discretion to prosecute, and moreso, is the CFAA even constitutional.