Meanwhile LinkedIn was happy to scrape its users' email archives and contact lists. It would appear there's some karma at play here...
That is an interesting fact.. LinkedIn broke the ruling against scraping they are relying on to prevent scraping.
LinkedIn: It’s illegal to scrape our website without permission
171–180 of 303 posts
Re: LinkedIn: It’s illegal to scrape our website without permission
#172Earlier quoted context omitted.
That's a pretty literalist, one-size fits all approach to policy. I don't think it's a good framework to use for applying ethics considerations. If I can walk near a pool, should I also be able to run? Is running anything more than faster walking? If I'm allowed to be around the pool walking with my entry ID, should I also be allowed to place my ID on a little motorized car and make it dart around the pool really fas…
The way HTTP works is there is a request and response. If LinkedIn does not want to provide a response, they don't have to. If they don't want to accept requests, then maybe the World Wide Web isn't for them.
So was Napster. Turns out, law>technology.
Re: LinkedIn: It’s illegal to scrape our website without permission
#173Unpopular opinion: when you make a HTTP request you're asking the server to give you information. The server has the right to say no. IMHO, LinkedIn doesn't have a right to stop scraping after the fact, but they have the right to take technical steps to stop scrapers from accessing their site.
I don't think that is an unpopular opinion. It also implies that LinkedIn is wrong, because of their HTTP server says yes, they want to retroactively say no.
Re: LinkedIn: It’s illegal to scrape our website without permission
#174Earlier quoted context omitted.
The owner makes money from the admission fee clients pay to see the bike. Taking photos and selling albums defeats that purpose.
But the owner is offering the images up for free to the public. hiQ is taking those publicly-available photos and annotating them with, "red bike", "pink bike", "broken bike", "professional bike", etc. It's clearly a value-add and not theft.
It depends.
You have to keep in mind that an entire generation was brainwashed that personal data isn't that "personal", so Google, FB and the rest can have amazing profits.
Most of these discussions are stained by general unawareness of the privacy and copyright law.
Ofc because the value of the data supplier (usually a single person, etc) these never reaches the courts, which just reinforces the ongoing misconceptions.
If you really want to test this, try copying the content from Google, Facebook hiQ or whoever that's big enough to go after you.
But people somehow believe that it's okay for businesses to do what regular persons aren't allowed to.
Re: LinkedIn: It’s illegal to scrape our website without permission
#175It would be very good to have one or a few solid legal decisions relating to scraping. Today, as far as I know, the entire segment is still in limbo.
It is important to understand that, legal or not, in the US anyone can sue you for anything. In another comment people were discussing the legality of following or ignoring "robots.txt". I tend to be pragmatic about this stuff. If you fabricate your own legal interpretation and end-up being sued by LinkedIn, it could end-up costing you $250K.
When facing large corporations law firms often ask for sizable retainers ($100K+) and proof of cash-on-hand to go beyond that, $250K total not being an unusual number. They don't do this out of greed. They do it because litigation at those levels can be very expensive. If you only have $100K you could find yourself burning through all of it quickly. If you don't have more cash to continue you'll lose the lawsuit and the $100K you spent will be burned for no reason at all. In other words, the law firm is protecting you by asking you to have enough cash to litigate.
A few years ago we started to develop a very extensive product based on obtaining data from Amazon. Some of the data is available through their API and other data had to be scraped in various forms. The product is extremely valuable yet the issues pertaining to scraping made me decide to put it on hold. Even if you can make millions a year the prospect facing a monster like Amazon in a lawsuit, as improbable as it might be, is scary enough to go look for other pastures.
Re: LinkedIn: It’s illegal to scrape our website without permission
#176It would be interesting to put a TOS into your scraper by the pragma or useragent field that says if you return data to the scraper you accept the TOS.
Re: LinkedIn: It’s illegal to scrape our website without permission
#177Unpopular opinion: when you make a HTTP request you're asking the server to give you information. The server has the right to say no. IMHO, LinkedIn doesn't have a right to stop scraping after the fact, but they have the right to take technical steps to stop scrapers from accessing their site.
I don't think you've characterized this accurately. When you make an HTTP request to LinkedIn you are accessing their service. There is a long history of this relationship, you plug your house into the sewer line and you connect to the sewer service. You connect to the power pole and connect to the electricity service. You connect to the telephone pole and connect to the telephone service.
Every service has "terms of service" which are the conditions that you are allowed to access the service and what you may do with the service once being granted access. For example, if you start pouring toxic waste into your sewer, you will find that the city will both disconnect you from the service and they will fine you for violating the terms of service you nominally agreed to when being hooked up.
In LinkedIn's case, they allow you to access their service, with HTTP, to render a page in a browser for viewing of that page. Full stop. Any other use of the data you acquire over HTTP, or any other method of acquiring said data over HTTP is disallowed by the terms of service.
Not only does LinkedIn have a legal right to stop scraping after the fact, they have literally centuries of common law in support of that position.
Re: LinkedIn: It’s illegal to scrape our website without permission
#178Meanwhile LinkedIn was happy to scrape its users' email archives and contact lists. It would appear there's some karma at play here...
That is an interesting fact.. LinkedIn broke the ruling against scraping they are relying on to prevent scraping.
Scraping would be forbidden via TOS; theoretically it would be the users giving access to linked in (who actually is bound by the terms) who would be liable, not linked in.
Re: LinkedIn: It’s illegal to scrape our website without permission
#179Unpopular opinion: when you make a HTTP request you're asking the server to give you information. The server has the right to say no. IMHO, LinkedIn doesn't have a right to stop scraping after the fact, but they have the right to take technical steps to stop scrapers from accessing their site.
I mostly agree. the traditional method [1] should be honored. but clearly, there are some bad agents out there that ignore the rules and mess things up for everyone. They are pretty clear about their expectations: # Notice: The use of robots or other automated means to access LinkedIn without # the express permission of LinkedIn is strictly prohibited. [1] https://www.linkedin.com/robots.txt
puuuh, somebody should explain the concept of user-agent groups to them (would heavily simplify their robots.txt)
Re: LinkedIn: It’s illegal to scrape our website without permission
#180Unpopular opinion: when you make a HTTP request you're asking the server to give you information. The server has the right to say no. IMHO, LinkedIn doesn't have a right to stop scraping after the fact, but they have the right to take technical steps to stop scrapers from accessing their site.
I don't think you've characterized this accurately. When you make an HTTP request to LinkedIn you are accessing their service. There is a long history of this relationship, you plug your house into the sewer line and you connect to the sewer service. You connect to the power pole and connect to the electricity service. You connect to the telephone pole and connect to the telephone service. Every service has "terms of…
When you make a connection to the city sewers or to the power company, there is some kind of pre-connection step where the terms are presented and you agree to those terms.
With HTTP and LinkedIn, there is no such step. There's no pre-connection agreement. LinkedIn could present such an agreement on first connection, but they do not.