Live data from Hacker News

LinkedIn: It’s illegal to scrape our website without permission

arstechnica.com

31–40 of 303 posts

Re: LinkedIn: It’s illegal to scrape our website without permission

#31
""The CFAA makes it a crime to "access a computer without authorization or exceed authorized access." Courts have been struggling to figure out what this means ever since Congress passed it more than 30 years ago.""

The law by itself is ok, but I suspect lawmakers were referring to accessing a single personal workstation, probably not taking into account a cluster of servers containing public accessible data.

Re: LinkedIn: It’s illegal to scrape our website without permission

#32
post #12

Earlier quoted context omitted.

The CFAA, which makes it a crime to access a computer system without proper permission.

That's a load of controversial hogwash. If you have permission to view the page, then you have permission to scrape the page. Anyone can make a LinkedIn account, ergo anyone can scrape any public profile.

> If you have permission to view the page, then you have permission to scrape the page.

This is false. Web pages expect scrapers to respect robots.txt and users to respect the terms and conditions. From what I understand, the CFAA law was not meant for these violations, and the fight is whether or not this law can be used to win the trial. However, other laws can be put into place to address these situations.

Re: LinkedIn: It’s illegal to scrape our website without permission

#34
post #10

I'm following this story with a lot of interest. I've done (and still do!) a lot of data crawling/scraping. In the past I've worked on so-called "alternative data" collection and analysis for financial forecasting. Without going into too much detail, a lot of hedge funds have teams constantly searching for kernels of data that can contribute some kind of signal for market movements. This data can come in the form of…

"satellite imagery for oil tankers" - Interesting.

If hedge funds floating drones above oil tankers, I'd guess they'd be accussed of corporate espionage / spying / invasion of privacy?

Ok, so oil tankers are big and "in the clear". What if $TANKERCORP floats big parachute balloons above its tankers to imply "looking past these is unauthorized viewing"?

Then if a HedgeFund gets a clever angle on a satellite photo.. is that the equivalent of breaking a lock, or violating CFAA?

Re: LinkedIn: It’s illegal to scrape our website without permission

#35

If you post it on a public network, it is defacto, public. If you don't want it scraped, take it down, or put it behind a login. If the user provides the login to a scraper, then the scraper has permission.

What if the user is provided with permission to view medical records - your medical records for example? Would it be ok to scrape and sell those? What about children's school profiles and records?

Re: LinkedIn: It’s illegal to scrape our website without permission

#36

"HiQ scrapes data about thousands of employees from public LinkedIn profiles, then packages the data for sale to employers worried about their employees quitting". cute.

I'd always been suspicious of linking up with my current employer/coworkers, now I guess it was a valid concern.

Re: LinkedIn: It’s illegal to scrape our website without permission

#37

Earlier quoted context omitted.

The CFAA, which makes it a crime to access a computer system without proper permission.

Providing the information on a publicly accessible page provides permission. It's like posting a letter on a public noticeboard and demanding that certain people not read it. If you don't want it read, take it down.

There is a distinction between a person reading one page and a scraper reading all pages.

First, the scraper consumes a lot more bandwidth (which some websites might have real trouble with). Second, the aggregated scraped data is a lot more valuable than what any single person receives from reading a few pages.

Websites could (and sometimes do) limit the number of views and the frequency of accesses from the same IP, which allows users to read the page and discourages scrapers without "taking it down".

Re: LinkedIn: It’s illegal to scrape our website without permission

#38

>One plausible reading of the law—the one LinkedIn is advocating—is that once a website operator asks you to stop accessing its site, you commit a crime if you don't comply. I'm fine with that. But I'm also fine if they do the like the article suggests and require anything non-scrapable to be behind an account prompt, even if everyone with a account can access it. I don't think it's fair to make Linked In foot the bi…

> They shouldn't have to serve that content to people who aren't actually their users.

So put it behind a password. It's not reasonable to expect to get only the benefits of publicly-searchable data without any of the drawbacks.

Re: LinkedIn: It’s illegal to scrape our website without permission

#39
post #21

Earlier quoted context omitted.

How is scraping data shady? Is it also shady to walk into a store and look at things without buying anything?

>Is it also shady to walk into a store and look at things without buying anything? Not making a claim about scraping, but it's maybe more apt to compare to walking into a store and writing a list of everything for sale their costs.

I don't see a problem with that. Market transparency is a good thing, as it allows the market to be efficient.
Post reply on HN