Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Congrats! Web scraping is legal! (US precedent)

parsers.me

Post reply on HN
Print view View original

61–70 of 409 posts

Re: Congrats! Web scraping is legal! (US precedent)

#61

Post by dec0dedab0de »

amelius wrote: post #21
JaceLightning wrote: post #10

Earlier quoted context omitted.

Probably not. Facts aren't copyrightable but creative works are. So prices on Amazon.com are facts. User reviews are creative so probably copyrighted. Similarly the videos on YouTube are copyrighted. However the number of views and the number of likes are probably scrapable.

Why would a review be a copyrightable creative work, while a LinkedIn resume wouldn't be?

I think perhaps the layout, cover letter, and maybe any flourishing notes are copyrightable, but the actual details of work experience and education are not.

Re: Congrats! Web scraping is legal! (US precedent)

#62

Post by manaskarekar »

Wowfunhappy wrote: post #29
kabacha wrote: post #11

The toxicity towards web-scraping is really what makes me lose hope in the current web. People want their data to be public and all of the benefits that comes with public data but then they want to chose who gets to see it - it's a complete and utter paradox. This precedent doesn't really mean much but is definitely step in the right direction.

I want web scraping to be legal—but, is it really contradictory to say "I want this data to be accessible to real humans only"? Any person can post on Hacker News. However, if someone made a bot to post to Hacker News, I think most of us would be pretty upset.

Makes me wonder, how many users on HN are actually very convincing bots?

Re: Congrats! Web scraping is legal! (US precedent)

#63

Post by ashman5 »

AznHisoka wrote: post #5

Linkedin is taking this to the Supreme Court: https://www.law360.com/articles/1237505/linkedin-will-go-to-... No ultimate decision was ever made, and no, this doesn't make web scraping 100% legal. Wake me up when there's a new announcement because anyone interested in this already know this old news.

5-4 overturn?

Re: Congrats! Web scraping is legal! (US precedent)

#64

Post by AlgorithmicTime »

Nitramp wrote: post #36

German copyright has the concept of a "Datenbankwerk" (since the 90s). E.g. the telephone book contains lots of boring facts that are each in themselves not copyrightable. However the collection in itself is copyrightable, as it required substantial effort to create. It seems odd that US copyright law wouldn't have a similar provision, or that it doesn't apply here?

Facts aren't copyrightable under US law. You could copy all of the phone numbers out of the phone book and put them in your own phone book without risking any sort of copyright infringement.

Re: Congrats! Web scraping is legal! (US precedent)

#65

Post by DougWebb »

kabacha wrote: post #11

The toxicity towards web-scraping is really what makes me lose hope in the current web. People want their data to be public and all of the benefits that comes with public data but then they want to chose who gets to see it - it's a complete and utter paradox. This precedent doesn't really mean much but is definitely step in the right direction.

One of my clients is involved in property tax collection and reporting. Property Tax records are public info, and their website allows looking up the records for any property without a login. However, the data behind this website it the _source_ of the public records, and not the public records themselves (which would be local government databases).

For years now we've been in an arms race with someone using a botnet to scrape all of the account information for a particular county. My client doesn't care so much about the data; it's the server load that's a problem. Normal activity for this site is a few dozen account searches per minute, but when the botnet gets through our blockade it sends hundreds of search requests per second, overwhelmimg the site. The operator of the botnet has NEVER tried to contact my client to ask for an efficient api to access the data, which they'd probably provide for a minimal fee.

Data hosting isn't free, even if the data is.

Re: Congrats! Web scraping is legal! (US precedent)

#66

Post by jakelazaroff »

Let’s not pretend this is a pure win. There are good uses of web scraping, like Archive.org trying to preserve the web. But what HiQ is doing is looking at public LinkedIn profiles and then snitching to employers if they think an employee is searching for a new job.

It’s easy to blanket say “web scraping is legal, do what you will“. The tricky part is protecting people’s public data while not giving a huge moat to giant corporations who control it.

Re: Congrats! Web scraping is legal! (US precedent)

#67

Post by giarc »

powrtoch wrote: post #38

"HiQ only takes information from public LinkedIn profiles. By definition, any member of the public has the right to access this information. Most importantly, the appeals court also upheld a lower court ruling that prohibits LinkedIn from interfering with hiQ’s web scraping of its site." Surely I'm not reading this correctly. This would seem to suggest that websites are not legally allowed to prevent bots from crawli…

Your second point is interesting. I suspect the contract between hiQ and some company is that hiQ provides info on public profiles, and if LinkedIn removes all public profiles by requiring a login the contract would become moot. Just the same if I was to change my profile settings from public to private, hiQ wouldn't be in breach of their contract (nor would I).

Re: Congrats! Web scraping is legal! (US precedent)

#68

Post by ncallaway »

kabacha wrote: post #44
pc86 wrote: post #35

Earlier quoted context omitted.

> Isn't data just data ? No. At the risk of just repeating the comment you didn't understand, creative works are not "just data" - they are copyrightable works that the owner has control over who can use them, not just for profit, but for any reason with few exceptions. You don't just get to drop someone else's work product into your algorithm without their permission.

> You don't just get to drop someone else's work product into your algorithm without their permission. Why not?

Because copyright law exists.

Re: Congrats! Web scraping is legal! (US precedent)

#69

Post by onetimemanytime »

>>Most importantly, the appeals court also upheld a lower court ruling that prohibits LinkedIn from interfering with hiQ’s web scraping of its site. This fundamentally changes the balance of power in dealing with such cases in the future.

This, I don't agree with. I agree that it's not fraud to send a bot to scrape public info but the site should every right to block a bot or person.

Re: Congrats! Web scraping is legal! (US precedent)

#70

Post by kerkeslager »

kabacha wrote: post #11

The toxicity towards web-scraping is really what makes me lose hope in the current web. People want their data to be public and all of the benefits that comes with public data but then they want to chose who gets to see it - it's a complete and utter paradox. This precedent doesn't really mean much but is definitely step in the right direction.

> People want their data to be public and all of the benefits that comes with public data but then they want to chose who gets to see it - it's a complete and utter paradox.

That's a complete misconception. Of course you can find manufacture inconsistent ideologies if you combine ideas from different people, but I think you'd have a difficult time finding one person who believes what you just described.

What I want is, put simply, organizational transparency, personal privacy. I believe humans have a right to privacy, but I don't believe organizations have rights, period, and I believe radical transparency within an organization prevents organizations from trampling the rights of individuals.

Organizations in this case include corporations, governments, and nonprofits.

Post reply on HN