Live data from Hacker News

Congrats! Web scraping is legal! (US precedent)

parsers.me

11–20 of 409 posts

Re: Congrats! Web scraping is legal! (US precedent)

#11
The toxicity towards web-scraping is really what makes me lose hope in the current web. People want their data to be public and all of the benefits that comes with public data but then they want to chose who gets to see it - it's a complete and utter paradox.

This precedent doesn't really mean much but is definitely step in the right direction.

Re: Congrats! Web scraping is legal! (US precedent)

#12
post #7

> Now many site owners are trying to put technical obstacles to competitors who completely copy their information that is not protected by copyright. For example, ticket prices, product lots, open user profiles, and so on. Some sites consider this information “their own”, and consider web scraping as “theft”. Legally, this is not the case, which is now officially enshrined in the US. Does this mean we can now scrape…

Probably not. Facts aren't copyrightable but creative works are. So prices on Amazon.com are facts. User reviews are creative so probably copyrighted. Similarly the videos on YouTube are copyrighted. However the number of views and the number of likes are probably scrapable.

> User reviews are creative so probably copyrighted.

I wonder if the number of stars are copyrighted. It's not creative, but a fact.

Re: Congrats! Web scraping is legal! (US precedent)

#13
post #6

Wonder if this means more native apps that are harder to scrap and less websites from companies that don’t want their data scrapped.

Native apps are generally a lot easier to scrape since they rely on an API which can't be changed willy nilly without breaking compatibility with older apps. Also you can't do captcha etc on API calls in the same way you can on websites. And of course the data is neatly formatted to be machine readable.

Re: Congrats! Web scraping is legal! (US precedent)

#14
post #7

> Now many site owners are trying to put technical obstacles to competitors who completely copy their information that is not protected by copyright. For example, ticket prices, product lots, open user profiles, and so on. Some sites consider this information “their own”, and consider web scraping as “theft”. Legally, this is not the case, which is now officially enshrined in the US. Does this mean we can now scrape…

There's some huge datasets of Amazon reviews available. Stanford has a big scrape out there, plus there's one from Amazon themselves in the AWS datasets.

Re: Congrats! Web scraping is legal! (US precedent)

#16
I always thought scraping was a fun idea, I just couldn't find the right use-case for it. I'm not a sports guy, and the big sites have pretty extensive API's. Something music related would be of interest to me, but I can already get updates on events like concerts (since the monopolies make a fortune selling tickets). I'm not sure what could be useful.

Re: Congrats! Web scraping is legal! (US precedent)

#17
post #7

> Now many site owners are trying to put technical obstacles to competitors who completely copy their information that is not protected by copyright. For example, ticket prices, product lots, open user profiles, and so on. Some sites consider this information “their own”, and consider web scraping as “theft”. Legally, this is not the case, which is now officially enshrined in the US. Does this mean we can now scrape…

Yes you can scrape them, no you cannot repubilsh them. Everything you listed is protected by copyright. You cannot infringe on copyrights because of this ruling. >hiQ argued that LinkedIn’s technical measures to block web scraping interfere with hiQ’s contracts with its own customers who rely on this data. In legal jargon, this is called” malicious interference with a contract”, which is prohibited by American law Do…

I think any ruling that says LinkedIn can't put in protectionary measures against automated requests is doomed to be overturned, as long as they're not doing it discriminately. Captcha, rate limiting, user agent testing, etc are all common tools to protect against malicious/unintentional denials of service. The question is what was LinkedIn doing, and did it specifically target hiQ while permitting others of the same class of traffic.

Re: Congrats! Web scraping is legal! (US precedent)

#18

Linkedin is taking this to the Supreme Court: https://www.law360.com/articles/1237505/linkedin-will-go-to-... No ultimate decision was ever made, and no, this doesn't make web scraping 100% legal. Wake me up when there's a new announcement because anyone interested in this already know this old news.

[deleted]

Re: Congrats! Web scraping is legal! (US precedent)

#19
post #11

The toxicity towards web-scraping is really what makes me lose hope in the current web. People want their data to be public and all of the benefits that comes with public data but then they want to chose who gets to see it - it's a complete and utter paradox. This precedent doesn't really mean much but is definitely step in the right direction.

I think you mean contradiction, not paradox.

Re: Congrats! Web scraping is legal! (US precedent)

#20
post #7

> Now many site owners are trying to put technical obstacles to competitors who completely copy their information that is not protected by copyright. For example, ticket prices, product lots, open user profiles, and so on. Some sites consider this information “their own”, and consider web scraping as “theft”. Legally, this is not the case, which is now officially enshrined in the US. Does this mean we can now scrape…

Yes you can scrape them, no you cannot repubilsh them. Everything you listed is protected by copyright. You cannot infringe on copyrights because of this ruling. >hiQ argued that LinkedIn’s technical measures to block web scraping interfere with hiQ’s contracts with its own customers who rely on this data. In legal jargon, this is called” malicious interference with a contract”, which is prohibited by American law Do…

Yeah what is the line here? Would it be against the rules to block known user agents, throttling of traffic?
Post reply on HN