Ironic that Linkedin hates people who scrape their site which is nothing but user generated content yet they do the same.
One rule for them, another for us. Same old, same old.
Tell HN: LinkedIn is automatically importing job postings into company profiles
111–120 of 166 posts
Re: Tell HN: LinkedIn is automatically importing job postings into company profiles
#112A bit of tangent, but it reminds me of a funny story: I discovered one day that I am getting recommended "Long distance truck driver" jobs through email and some of the re-targeted ads. First, I ignored them, but in a week or two, it became apparent that it was not just a coincidence. After some research, I realized that I had listed "Device Driver" programming as a skill on my LinkedIn and personal website. I suspec…
Haha I got suggested for "freight" and "shipping" due to having "Cargo" (the MediaWiki SQL wrapper, not the package manager) listed on mine.
Re: Tell HN: LinkedIn is automatically importing job postings into company profiles
#113Scraping for me, but not for thee. https://news.bloomberglaw.com/privacy-and-data-security/link...
LinkedIn may be less hypocritical than it first appears: February 2022: LinkedIn sue Mantheos. April 18, 2022: TechCrunch runs an article [0] stating that a U.S. appellate court affirms the legality of scraping publicly available data. So IIUC, LinkedIn was consistently playing to the apparent law. EDIT: oops, forgot the link! [0] https://techcrunch.com/2022/04/18/web-scraping-legal-court/
I was working at a recruiting startup years ago and we got a cease & desist letter from LI asking us to stop crawling their public data.
Initially crawling them was very easy (we had ~100m profiles). With time crawling became harder and harder on the technical front. They obviously invested millions of $ in technology to stop being crawled.
Re: Tell HN: LinkedIn is automatically importing job postings into company profiles
#114Earlier quoted context omitted.
Web scraping is legal, US appeals court reaffirms (April 2022) [1]: > Using a “gate-up, gate-down” analogy, the Supreme Court said that when a computer or website’s gates are up — and therefore information is publicly accessible — no authorization is required. > The Ninth Circuit, in referencing the Supreme Court’s “gate-up, gate-down” analogy, ruled that “the concept of ‘without authorization’ does not apply to publ…
Scraping is one thing; reproduction is another. I can scrape the Times but I don't get to mirror their site on my own domain.
I've always wondered how this isn't more of an issue for the common archivers and news sites...
Re: Tell HN: LinkedIn is automatically importing job postings into company profiles
#115Everyone needs to put the following in their robots.txt to give Linkedin a taste of their own medicine. User-agent: LinkedinBot Disallow: / # Notice: If you would like to crawl us, # please email whitelist-crawl@domain.com to apply # for white listing. *taken from linkedin.com/robots.txt
HN messes up whitespace, so you'll have to indent this to make if preserve the newlines.
Re: Tell HN: LinkedIn is automatically importing job postings into company profiles
#116LinkedIn's motto should be "Making Derivatives out of Career aspirations"
seriously, everything you do on it are fired off in ALL the tangents like a jelly fish and just desperately tries to tangle onto something. Could be poisonous I don't know, does feel like it when it's done indiscriminately like that
Re: Tell HN: LinkedIn is automatically importing job postings into company profiles
#117Preface: I hate LinkedIn Do we know that in fact LinkedIn is scraping? Could it be that the LinkedIn data science team is erroneously matching company names with job titles that are added to its website? E.g., someone from YMCombinator posts a page to LinkedIn and puts the application link to angel.co. Then LinkedIn mistakenly sees YMCombinator as YCombinator and adds the logo (no scraping involved).
LinkedIn may or may not be scraping. Just remember that scraping is a pejorative word for what Google does, also known as "indexing".
Re: Tell HN: LinkedIn is automatically importing job postings into company profiles
#118Earlier quoted context omitted.
When I decide to click on the link then I accept the other side will know I clicked on that URL. What I tried to say was that I am always a bit worried that by clicking on a link there will be a chain of redirects or some click-jacking technique (or anything else) that in result will take over one of my accounts which I'm keeping logged in on my main browser (i.e. google, or LinkedIn)
I'm not particularly sure I understand. You're worried that because the URL is long that it might somehow take over your google account? I'm not quite sure I understand the correlation between "long URL" and redirects/clickjacking.
Looks like I structured what I tried to say in wrong way, sorry about that, English is not my mother tongue and sometimes I try to say too much in one sentence. I generally don't trust hyperlinks sent to me when pointing at source I have never seen before. I open such URLs in private session.
Re: Tell HN: LinkedIn is automatically importing job postings into company profiles
#119Everyone needs to put the following in their robots.txt to give Linkedin a taste of their own medicine. User-agent: LinkedinBot Disallow: / # Notice: If you would like to crawl us, # please email whitelist-crawl@domain.com to apply # for white listing. *taken from linkedin.com/robots.txt
Ah yes, surely they will always respect the holy robots.txt
Re: Tell HN: LinkedIn is automatically importing job postings into company profiles
#120Glassdoor does this too.
It’s particularly frustrating because there’s an Israeli company with a very similar name as ours, so all of their out-of-country posts constantly get attached to our company’s profile (along with a bunch of negative reviews complaining about the cleanliness of the on-site bathrooms, yet we are a 100% remote company with no offices)
It was a major struggle to get those reviews removed.