Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

U.S. judge says LinkedIn cannot block startup from public profile data

reuters.com

Post reply on HN
Print view View original

241–250 of 301 posts

Re: U.S. judge says LinkedIn cannot block startup from public profile data

#241

Post by opaque »

revelation wrote: post #39
opaque wrote: post #20

Does anyone know how they do this scraping from a technical standpoint. The articles allude to it being the same as data Google/Bing spiders, which can clearly access more data that average internet IP for making their result summaries. I had assumed big sites whitelisted specific crawler IP ranges or User-Agents for the search giants. Do they somehow spoof this?

I don't think they do any such thing, if anything they are rotating IPs/user agents to avoid being limited or blocked. Google requires sites to send the crawler the same content as someone clicking a link on a Google results page would see, so even if some sites get creative covering it up with blurred boxes and similar dark patterns, the data is there in the markup.

I haven't checked the markup, but if you try and hit a linkedin profile page you just get forwarded to a login page. Perhaps if you don't follow the forwarding?

Not sure how this complies with google's requirements, I suspect if you're big enough you get a custom arrangement. However, that doesn't explain how hiQ are getting the data.

Re: U.S. judge says LinkedIn cannot block startup from public profile data

#242

Post by [deleted] »

chalst wrote: post #232
c3534l wrote: post #169

Earlier quoted context omitted.

Great. So in a business that makes money primarily by collecting and selling user data, they're now required to give it away for free.

Correction: user-generated data. LI get the content from their users for free.

[deleted]

Re: U.S. judge says LinkedIn cannot block startup from public profile data

#243

Post by pjc50 »

The question of "data protection" hasn't been discussed enough here - it may well be that Linkedin has no case against HiQ, but if HiQ is scraping people's PII in the EU they are required to have the permission of the data subjects.

How this interacts with Safe Harbour I have no idea.

Re: U.S. judge says LinkedIn cannot block startup from public profile data

#244

Post by vntok »

chalst wrote: post #232
c3534l wrote: post #169

Earlier quoted context omitted.

Great. So in a business that makes money primarily by collecting and selling user data, they're now required to give it away for free.

Correction: user-generated data. LI get the content from their users for free.

I dont understand how you came to think that. What about the cost of hosting and serving the html forms (cpu and bandwidth), processing the content update requests, storing the resulting data structures? What about the marketing costs of making the brand known both by individuals and companies? The wages associated to any of the operations i have just mentioned?

What definition of "free" are you using here?

Re: U.S. judge says LinkedIn cannot block startup from public profile data

#245

Post by brango »

IANAL but I suspect the forthcoming GDPR will make this illegal for EU customer data. It gives users greater control over their data, so I wouldn't be surprised to find that it corresponds with this judgement, i.e. if a user determines data to be publicly available, it must be made to be so.

Anyone know whether this is right?

And BTW in case you're not aware, if you hold data from any EU citizens you'll be required to comply with the GDPR regardless of where you're located.

Re: U.S. judge says LinkedIn cannot block startup from public profile data

#246

Post by joncrocks »

jaredklewis wrote: post #211
anandsuresh wrote: post #65

Earlier quoted context omitted.

This is a tricky issue that has more to do with user psychology than technology. While the data is public, most users do not understand the persistence characteristics of data, especially in the presence of 3rd parties. In a world where there are no (persistent) copies made by third-parties, the user still is in control of the visibility of their data by updating their profile directly on LinkedIn to show/hide pieces…

Though I disagree, I think basically understand your argument regarding why people should have to respect the wishes of a party when dealing with information that the party made public. What I cannot understand is how such a system could reasonably be enforced. Let's say John Doe posts his resume on a job board. If I print out his resume, but he later updates it, am I now somehow in the wrong for retaining the old co…

With regard to persistence, I don't think you can argue that the scale of what can now practically be stored hasn't changed significantly.

Given that a larger amount of information can now be stored easily, the economics of what is stored is different, and one would expect this would lead to the storage of more information over time. If you can store in a single HD what would have previously taken a large room, for a tiny fraction of the cost, the bar for storing the information vs. discarding it is much much lower.

I also think it's not only storage, but searching of large datasets that's another reason that information changing over time. Again, if finding information in a large blob of data is easier and as discussed above, is much cheaper, then this will also lead to people storing information for later retrieval. As you say, the fact that all this storage is not networked together means that this information is now easily retrievable from anywhere, only intensifying the impact.

Re: U.S. judge says LinkedIn cannot block startup from public profile data

#247

Post by codedokode »

> U.S. District Judge Edward Chen in San Francisco granted a preliminary injunction request brought by hiQ Labs, and ordered LinkedIn to remove within 24 hours any technology preventing hiQ from accessing public profiles.

That is actually dangerous. Why some startup or some judge can tell me to whom I can serve content and to whom I cannot?

Re: U.S. judge says LinkedIn cannot block startup from public profile data

#248

Post by jakubbalada »

mtokunaga wrote: post #207

This type of decision might also impact Yelp or any others in similar businesses. Currently their API limits a top few reviews per business via their API, and also prohibits "scraping" of data in other means. I was going to do some experiments with larger datasets from businesses in a region, but quickly found that's not possible.

You might try Apifier for that, we've recently scraped more than 150k reviews for 27k restaurants in London.

Here's a community crawler you can use: https://www.apifier.com/community/crawlers/Yonny/bcYqH-api-u...

Re: U.S. judge says LinkedIn cannot block startup from public profile data

#249

Post by kakarot »

ikeboy wrote: post #206
bigtones wrote: post #6

This is just a preliminary injunction and the court has not even heard or ruled on this case. They just allowed HiQ to access the data while they wait for the scheduled court hearing to begin. The court may eventually rule very differently once they have heard all the evidence presented and weighed up existing applicable case law. The judge who issued this injunction - Edward Chen, is also the judge presiding over th…

Forcing LinkedIn to provide the data and remove any locks is pretty harsh. Imagine I sued a bank under the theory that the locks on their vault doors are illegally preventing me from opening them. As part of the pleliminary injunction, the judge rules that the bank must remove all locks until the case is decided. Point is, it's not just "maintain the status quo", it's "give them your data for free". IANAL but I don't…

Misleading analogy. Physical goods such as a bank's money are both not reproducible and not normally available for the general public to just take

Re: U.S. judge says LinkedIn cannot block startup from public profile data

#250

Post by fastier »

vntok wrote: post #244
chalst wrote: post #232

Earlier quoted context omitted.

Correction: user-generated data. LI get the content from their users for free.

I dont understand how you came to think that. What about the cost of hosting and serving the html forms (cpu and bandwidth), processing the content update requests, storing the resulting data structures? What about the marketing costs of making the brand known both by individuals and companies? The wages associated to any of the operations i have just mentioned? What definition of "free" are you using here?

I agree there seems to be a problems with means and investments: if I invest to create a website with valuable data, and anyone can come and scrape my data and start their own or a competing business, riding on top of my investment, isn't that a disincentive to me?
Post reply on HN