I guess for LinkedIn it could matter most, because their only livelihood is CV and employer companies Data, if you steal a poor man's bread they are gonna be upset. It was dead as a social network long ago if it ever was, now their only asset is the data, so I guess their right to protect their only asset is legit.
LinkedIn: It’s illegal to scrape our website without permission
151–160 of 303 posts
Re: LinkedIn: It’s illegal to scrape our website without permission
#152Earlier quoted context omitted.
I would suggest occasionally actually informing yourself of the wider world. Your very example of Google Maps in Germany - well, turns out Google had to give people the option to opt out: https://europe.googleblog.com/2010/10/how-many-german-househ... Same goes for web sites - robots.txt exists for a reason. If your crawler ignores that, well, I'd suggest talking to a lawyer.
I wasn’t aware that robots.txt had any legal meaning. Am I wrong?
I'd talk to a lawyer. There are presumably some legal arguments that might apply. There definitely have been cases revolving around it, but I have no idea what the outcomes were.
From a moral point of view: robots.txt states an intent, and intentionally ignoring that is not a nice thing.
And often, "not a nice thing" translates into legal action. So, if this matters to you, you should find an expert :)
Re: LinkedIn: It’s illegal to scrape our website without permission
#153it is incredible how many global companies are successfully scraping LI as part of their core business. you would be suprised
Re: LinkedIn: It’s illegal to scrape our website without permission
#154Earlier quoted context omitted.
LinkedIn takes plenty of technical precautions to block scraping. I’ve built bots that scrape them in the past, it’s surprisingly difficult as LinkedIn is very good at determining you’re a bot and blocking you. So it’s hard to argue that a service which is scraping LinkedIn is doing it without knowledge that they are going against LinkedIn’s wishes. Whether or not this is illegal is up to the courts to determine, and…
* That being said, I hate LinkedIn as a company and I fully support anyone trying to mess with them So it's fine to mess with them, even illegally, just because you don't like them? * convinces people to willingly provide personal information Convincing is not forcing, and in fact, you say "willingly" yourself. Any business convinces you to willingly give them money or other value. * sell at ridiculously high prices…
This might be one of the best replies on HN that I have seen. Spot on.
Re: LinkedIn: It’s illegal to scrape our website without permission
#155When I make my CV public on LinkedIn I expect it to be public. Even for bots.
That is the whole point of 'public.'
Re: LinkedIn: It’s illegal to scrape our website without permission
#156Earlier quoted context omitted.
LinkedIn takes plenty of technical precautions to block scraping. I’ve built bots that scrape them in the past, it’s surprisingly difficult as LinkedIn is very good at determining you’re a bot and blocking you. So it’s hard to argue that a service which is scraping LinkedIn is doing it without knowledge that they are going against LinkedIn’s wishes. Whether or not this is illegal is up to the courts to determine, and…
* That being said, I hate LinkedIn as a company and I fully support anyone trying to mess with them So it's fine to mess with them, even illegally, just because you don't like them? * convinces people to willingly provide personal information Convincing is not forcing, and in fact, you say "willingly" yourself. Any business convinces you to willingly give them money or other value. * sell at ridiculously high prices…
> So it's fine to mess with them, even illegally, just because you don't like them?
If it's not illegal, then scummy behaviour against a scummy company doesn't exactly set my moral compass off. You reap what you sow.
Re: LinkedIn: It’s illegal to scrape our website without permission
#157As far as I'm concerned, if the information is publicly visible on a site (not behind a login) and as long as the scraping doesn't cause performance issues or generate costs on the site, then it should be perfectly fine to do.
> As far as I'm concerned, if the information is publicly visible on a site then it should be perfectly fine to do.
There, I fixed it.
Re: LinkedIn: It’s illegal to scrape our website without permission
#158I can't even view LinkedIn with a web browser, how the hell do robots get in?
Re: LinkedIn: It’s illegal to scrape our website without permission
#159I guess for LinkedIn it could matter most, because their only livelihood is CV and employer companies Data, if you steal a poor man's bread they are gonna be upset. It was dead as a social network long ago if it ever was, now their only asset is the data, so I guess their right to protect their only asset is legit.
Their desire to protect it is legit, but it doesn't necessarily give them a right to do so.
A long time ago, companies that were compiling phonebooks also thought that they could use e.g. copyright to prevent others from using that compiled data for commercial gain. They were wrong, because copyright doesn't protect "mere aggregation" of data, as courts have ruled.
In this case, LinkedIn is not arguing on the basis of copyright, so it's a different legal argument. But the essence of the case is they same - they want to have a business model around aggregating other people's data and then providing access to that data, while limiting what people who access it can do with it. They don't have a right to this business model. If technical means of restricting access don't work, or if adopting them means that they drive most of their customers away, tough luck.
Re: LinkedIn: It’s illegal to scrape our website without permission
#160What if I build a new public search engine and LinkedIn blocks me because I am not Google or Bing?
They could stop potential (and much needed) competitors.