Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

U.S. judge says LinkedIn cannot block startup from public profile data

reuters.com

Post reply on HN
Print view View original

31–40 of 301 posts

Re: U.S. judge says LinkedIn cannot block startup from public profile data

#31

Post by CosmicShadow »

devrandomguy wrote: post #11
DanBlake wrote: post #2

This seems very at-odds with previous rulings (specifically, relating to craigslists many past dealings). Strikes me as being very unlikely to stand up to appeal. Also, linkedin will likely modify their websites behavior (make you click to agree before you view a profile) which would create a binding 'click wrap' stopping companies from scraping them.

That click wrap contract is kind of an interesting thing on it's own, for those of us who only enable JS when absolutely necessary. If I never see the agreement, and I am not specifically avoiding it, does it still apply to me?

There was a big ruling in Canada about this specifically around MLS, the big real estate monopoly we have, so that if you go to their sites to search for homes, like you'd see at Realtor.ca, you have to click through a clickwrapper to access any data, and even if you automate past that, the fact that a human would have to click it means that it's illegal to scrape since you are forced as a human to agree to a TOS before you view.

Re: U.S. judge says LinkedIn cannot block startup from public profile data

#33

Post by swiley »

djsumdog wrote: post #27
danschumann wrote: post #10

Being a programmer not a lawyer, I like the idea of more rights for scrapers. I don't want to see the internet partitioned away and owned by a few companies, especially when that information is often called a "public profile".

It gets into the incredibly murky water of how the web works. You're just issuing a request and getting things back. Sometimes in a web browser, sometimes not. But the content itself may still be copyright. You can't just take it, even though for now, the publisher/server is allowing you to view it for free. But what if you only chose to view some of the content (e.g. block ads). What if you apply your own styles to…

Maybe the whole idea of copyright is flawed and harmful?

Re: U.S. judge says LinkedIn cannot block startup from public profile data

#34

Post by revelation »

djsumdog wrote: post #27
danschumann wrote: post #10

Being a programmer not a lawyer, I like the idea of more rights for scrapers. I don't want to see the internet partitioned away and owned by a few companies, especially when that information is often called a "public profile".

It gets into the incredibly murky water of how the web works. You're just issuing a request and getting things back. Sometimes in a web browser, sometimes not. But the content itself may still be copyright. You can't just take it, even though for now, the publisher/server is allowing you to view it for free. But what if you only chose to view some of the content (e.g. block ads). What if you apply your own styles to…

There is no grey area. You can not copyright facts. If you download ("scrape") a webpage and then extract the facts, whatever you downloaded only exists in volatile memory. So there is no claim there. The only claim you can make is on the download itself, hence what LinkedIn chose.

Re: U.S. judge says LinkedIn cannot block startup from public profile data

#35

Post by isalmon »

DanBlake wrote: post #2

This seems very at-odds with previous rulings (specifically, relating to craigslists many past dealings). Strikes me as being very unlikely to stand up to appeal. Also, linkedin will likely modify their websites behavior (make you click to agree before you view a profile) which would create a binding 'click wrap' stopping companies from scraping them.

The biggest reason why they have not done this so far is SEO. If you introduce the 'click wrap' - other crawlers like Google won't be able to crawl it, so their traffic will decrease overnight.

Re: U.S. judge says LinkedIn cannot block startup from public profile data

#36

Post by onion2k »

revelation wrote: post #34
djsumdog wrote: post #27

Earlier quoted context omitted.

It gets into the incredibly murky water of how the web works. You're just issuing a request and getting things back. Sometimes in a web browser, sometimes not. But the content itself may still be copyright. You can't just take it, even though for now, the publisher/server is allowing you to view it for free. But what if you only chose to view some of the content (e.g. block ads). What if you apply your own styles to…

There is no grey area. You can not copyright facts. If you download ("scrape") a webpage and then extract the facts, whatever you downloaded only exists in volatile memory. So there is no claim there. The only claim you can make is on the download itself, hence what LinkedIn chose.

I've seen plenty of LinkedIn profiles that could be classified as fiction.

Re: U.S. judge says LinkedIn cannot block startup from public profile data

#37

Post by devrandomguy »

CosmicShadow wrote: post #31
devrandomguy wrote: post #11

Earlier quoted context omitted.

That click wrap contract is kind of an interesting thing on it's own, for those of us who only enable JS when absolutely necessary. If I never see the agreement, and I am not specifically avoiding it, does it still apply to me?

There was a big ruling in Canada about this specifically around MLS, the big real estate monopoly we have, so that if you go to their sites to search for homes, like you'd see at Realtor.ca, you have to click through a clickwrapper to access any data, and even if you automate past that, the fact that a human would have to click it means that it's illegal to scrape since you are forced as a human to agree to a TOS bef…

Ah yes, that. MLS compliance was a source of many tickets for me, in a previous job, in Canada. The employer didn't even want me to waste time trying to learn it all, just follow the compliance officer.

IIRC, this stuff varies quite a bit from region to region, even within a single metropolitan area. Attempting to simultaneously comply with multiple independently developed rulebooks was ... fun.

I can't wait for shipyard startups to disrupt the housing market. /s

Re: U.S. judge says LinkedIn cannot block startup from public profile data

#38

Post by sbarre »

smegel wrote: post #28
danschumann wrote: post #10

Being a programmer not a lawyer, I like the idea of more rights for scrapers. I don't want to see the internet partitioned away and owned by a few companies, especially when that information is often called a "public profile".

If a website puts something on the public internet, it should not even be aware if it is being accessed by a scraper or a human. Maybe we should just ban User Agent strings and be done with it.

You call it the "public internet" but it's most definitely not a public space or anything like it.

Private entities own and operate all(most of) the servers, services and conduits, and that does need to be paid for and maintained.

I'm not saying I agree with Linkedin in this particular scenario, but this is about two commercial for-profit entities arguing over money, so let's not make it about something it's not.

Re: U.S. judge says LinkedIn cannot block startup from public profile data

#39

Post by revelation »

opaque wrote: post #20

Does anyone know how they do this scraping from a technical standpoint. The articles allude to it being the same as data Google/Bing spiders, which can clearly access more data that average internet IP for making their result summaries. I had assumed big sites whitelisted specific crawler IP ranges or User-Agents for the search giants. Do they somehow spoof this?

I don't think they do any such thing, if anything they are rotating IPs/user agents to avoid being limited or blocked.

Google requires sites to send the crawler the same content as someone clicking a link on a Google results page would see, so even if some sites get creative covering it up with blurred boxes and similar dark patterns, the data is there in the markup.

Re: U.S. judge says LinkedIn cannot block startup from public profile data

#40

Post by dawnerd »

isalmon wrote: post #35
DanBlake wrote: post #2

This seems very at-odds with previous rulings (specifically, relating to craigslists many past dealings). Strikes me as being very unlikely to stand up to appeal. Also, linkedin will likely modify their websites behavior (make you click to agree before you view a profile) which would create a binding 'click wrap' stopping companies from scraping them.

The biggest reason why they have not done this so far is SEO. If you introduce the 'click wrap' - other crawlers like Google won't be able to crawl it, so their traffic will decrease overnight.

They'd most certainly whitelist the google ips.
Post reply on HN