141–150 of 303 posts
IMHO, LinkedIn doesn't have a right to stop scraping after the fact, but they have the right to take technical steps to stop scrapers from accessing their site.
Unpopular opinion: when you make a HTTP request you're asking the server to give you information. The server has the right to say no. IMHO, LinkedIn doesn't have a right to stop scraping after the fact, but they have the right to take technical steps to stop scrapers from accessing their site.
I do a lot of crawler services as well. The last one, I did 13 crawlers to keeping comparing prices of all drugstore products online. I'm selling it to drugstore ecommerces who wants to know when their competitor prices and when they're doing promotions, if competitors prices are higher... Well, I just automated the job of a person that was doing this job manually every single day, looking into competitor's site If i…
Your very example of Google Maps in Germany - well, turns out Google had to give people the option to opt out: https://europe.googleblog.com/2010/10/how-many-german-househ...
Same goes for web sites - robots.txt exists for a reason. If your crawler ignores that, well, I'd suggest talking to a lawyer.
Unpopular opinion: when you make a HTTP request you're asking the server to give you information. The server has the right to say no. IMHO, LinkedIn doesn't have a right to stop scraping after the fact, but they have the right to take technical steps to stop scrapers from accessing their site.
That being said, I hate LinkedIn as a company and I fully support anyone trying to mess with them. They are not a social network, they are a sleazy website that convinces people to willingly provide personal information which they then turn around and sell at ridiculously high prices. Even if you are legitimately using LinkedIn as an end-user, it’s easy to get blocked for using it too much and being forced to pay just to interact with people on the site.
Unpopular opinion: when you make a HTTP request you're asking the server to give you information. The server has the right to say no. IMHO, LinkedIn doesn't have a right to stop scraping after the fact, but they have the right to take technical steps to stop scrapers from accessing their site.
# Notice: The use of robots or other automated means to access LinkedIn without
# the express permission of LinkedIn is strictly prohibited.
I do a lot of crawler services as well. The last one, I did 13 crawlers to keeping comparing prices of all drugstore products online. I'm selling it to drugstore ecommerces who wants to know when their competitor prices and when they're doing promotions, if competitors prices are higher... Well, I just automated the job of a person that was doing this job manually every single day, looking into competitor's site If i…
I would suggest occasionally actually informing yourself of the wider world. Your very example of Google Maps in Germany - well, turns out Google had to give people the option to opt out: https://europe.googleblog.com/2010/10/how-many-german-househ... Same goes for web sites - robots.txt exists for a reason. If your crawler ignores that, well, I'd suggest talking to a lawyer.
Unpopular opinion: when you make a HTTP request you're asking the server to give you information. The server has the right to say no. IMHO, LinkedIn doesn't have a right to stop scraping after the fact, but they have the right to take technical steps to stop scrapers from accessing their site.
Earlier quoted context omitted.
I would suggest occasionally actually informing yourself of the wider world. Your very example of Google Maps in Germany - well, turns out Google had to give people the option to opt out: https://europe.googleblog.com/2010/10/how-many-german-househ... Same goes for web sites - robots.txt exists for a reason. If your crawler ignores that, well, I'd suggest talking to a lawyer.
I wasn’t aware that robots.txt had any legal meaning. Am I wrong?
Unpopular opinion: when you make a HTTP request you're asking the server to give you information. The server has the right to say no. IMHO, LinkedIn doesn't have a right to stop scraping after the fact, but they have the right to take technical steps to stop scrapers from accessing their site.
LinkedIn takes plenty of technical precautions to block scraping. I’ve built bots that scrape them in the past, it’s surprisingly difficult as LinkedIn is very good at determining you’re a bot and blocking you. So it’s hard to argue that a service which is scraping LinkedIn is doing it without knowledge that they are going against LinkedIn’s wishes. Whether or not this is illegal is up to the courts to determine, and…
So it's fine to mess with them, even illegally, just because you don't like them?
* convinces people to willingly provide personal information
Convincing is not forcing, and in fact, you say "willingly" yourself. Any business convinces you to willingly give them money or other value.
* sell at ridiculously high prices
To respect to what? Prices are determined by the market. We are not talking about a life-saving medicine or health care, on which there could be some debate.
We are talking about a company selling information, which has value and which it acquires through an infrastructure that takes a lot of money to run. Value that customers are willing to pay for.
* Even if you are legitimately using LinkedIn as an end-user, it’s easy to get blocked
For what definition of legitimately? Yours? Since it's their business, they can define what is a legitimate free use and what can be a paid one.
* They are not a social network
So based on what I pointed out, they are not a social network only because they are not free? Do all social network have to be free and give your data for advertisers to be social networks?
Earlier quoted context omitted.
I would suggest occasionally actually informing yourself of the wider world. Your very example of Google Maps in Germany - well, turns out Google had to give people the option to opt out: https://europe.googleblog.com/2010/10/how-many-german-househ... Same goes for web sites - robots.txt exists for a reason. If your crawler ignores that, well, I'd suggest talking to a lawyer.
I wasn’t aware that robots.txt had any legal meaning. Am I wrong?
I hate to be the bearer of bad new but...maybe. A reading of the Computer Fraud and Abuse Act could make robots.txt legally enforceable. And given the government's approach to CFAA cases a very aggressive interpretation, under the right circumstances (for example, when it provides evidence that the scraper knew that scrapint was not authorized), seems like a real possibility.
Among the many other things CFAA criminalizes, it makes it a crime to "intentionally access[] a computer without authorization or exceed[] authorized access . . and thereby obtain[] information from any protected computer;"
A "protected computer" is, among other things, a computer "which is used in or affecting interstate or foreign commerce or communication." That would probably cover just about any server on the Internet.