Congrats! Web scraping is legal! (US precedent) • Hacker News Board

Re: Congrats! Web scraping is legal! (US precedent)

#51

Post by jermaustin1 » Wed, Jan 29, 2020, 2:41 PM UTC

fludlight wrote: post #25

This only affects the ninth circuit—which includes the tech hubs San Francisco, Seattle, LA, and Portland. It would only apply to the rest of the country if the Supreme Court affirmed it. Even then, a well-funded company or zealous prosecutor could say that it doesn’t apply in your case because of some technicality. In that case you would need hundreds of thousands or millions of dollars and a few years to litigate t…

Is that how really circuit court rulings get applied?

I always understood each ruling on the rungs up the ladder to the supreme court applied across the land until a final ruling was determined.

Re: Congrats! Web scraping is legal! (US precedent)

#52

Post by sillyquiet » Wed, Jan 29, 2020, 2:42 PM UTC

kabacha wrote: post #11

The toxicity towards web-scraping is really what makes me lose hope in the current web. People want their data to be public and all of the benefits that comes with public data but then they want to chose who gets to see it - it's a complete and utter paradox. This precedent doesn't really mean much but is definitely step in the right direction.

Yeah, I have nothing at all against scraping per se, it's more about the huge bot traffic the commercial scrapers generate, which would ALSO be fine, but 1. it can be hard to tell scrapers from malicious DDOS bots sometimes and 2.) the person being scraped literally pays for that scraping traffic.

Re: Congrats! Web scraping is legal! (US precedent)

#53

Post by city41 » Wed, Jan 29, 2020, 2:43 PM UTC

Pigo wrote: post #16

I always thought scraping was a fun idea, I just couldn't find the right use-case for it. I'm not a sports guy, and the big sites have pretty extensive API's. Something music related would be of interest to me, but I can already get updates on events like concerts (since the monopolies make a fortune selling tickets). I'm not sure what could be useful.

I once scraped job sites to get a mile high sense of the tech industries in cities I was considering moving to.

EDIT: and to add, I've always dreamed of a website that gathered job, rent, entertainment, etc data on cities you're interested in to help figure out where are good places to move to. A site like that would have to be pretty much entirely built from scraping.

Re: Congrats! Web scraping is legal! (US precedent)

#54

Post by freeone3000 » Wed, Jan 29, 2020, 2:44 PM UTC

Nitramp wrote: post #36

German copyright has the concept of a "Datenbankwerk" (since the 90s). E.g. the telephone book contains lots of boring facts that are each in themselves not copyrightable. However the collection in itself is copyrightable, as it required substantial effort to create. It seems odd that US copyright law wouldn't have a similar provision, or that it doesn't apply here?

It's an actual carve-out. Database work is explicitly NOT protected - Feist v Rural was actually about a telephone book deemed unprotectable!

Re: Congrats! Web scraping is legal! (US precedent)

#55

Post by dsr_ » Wed, Jan 29, 2020, 2:44 PM UTC

Nitramp wrote: post #36

German copyright has the concept of a "Datenbankwerk" (since the 90s). E.g. the telephone book contains lots of boring facts that are each in themselves not copyrightable. However the collection in itself is copyrightable, as it required substantial effort to create. It seems odd that US copyright law wouldn't have a similar provision, or that it doesn't apply here?

It does, and the result is that phone books and maps get fictional entries inserted in order to prove copying -- because it is perfectly legal to do your own work to amass the same data set.

https://en.wikipedia.org/wiki/Fictitious_entry

Re: Congrats! Web scraping is legal! (US precedent)

#56

Post by dillonmckay » Wed, Jan 29, 2020, 2:45 PM UTC

kabacha wrote: post #22

JaceLightning wrote: post #10

Earlier quoted context omitted.

Probably not. Facts aren't copyrightable but creative works are. So prices on Amazon.com are facts. User reviews are creative so probably copyrighted. Similarly the videos on YouTube are copyrighted. However the number of views and the number of likes are probably scrapable.

See that's where I have problem with this. Isn't data just _data_? Lets draw some pararells to real life. If I go to public space like town square - can't I take pictures, notes and records then go home and draw my analytics from it? What if I read something in a book I bought, can't I quote it? Same thing should be with web resources even if they are creative - as long as I don't publish them I should be able to scr…

Additionally, some public areas prohibit photography of architecture because of copyright.

https://www.diyphotography.net/10-famous-landmarks-youre-all...

Re: Congrats! Web scraping is legal! (US precedent)

#57

Post by nathanlied » Wed, Jan 29, 2020, 2:45 PM UTC

Wowfunhappy wrote: post #29

kabacha wrote: post #11

The toxicity towards web-scraping is really what makes me lose hope in the current web. People want their data to be public and all of the benefits that comes with public data but then they want to chose who gets to see it - it's a complete and utter paradox. This precedent doesn't really mean much but is definitely step in the right direction.

I want web scraping to be legal—but, is it really contradictory to say "I want this data to be accessible to real humans only"? Any person can post on Hacker News. However, if someone made a bot to post to Hacker News, I think most of us would be pretty upset.

Isn't that a different paradigm, though? A posting bot set loose on a forum/platform will (normally) visibly degrade service in a much more visible and impactful way than a scraping bot. And in either case, writing (and running!) a bot that posts on HN is not illegal behaviour in itself.

Re: Congrats! Web scraping is legal! (US precedent)

#58

Post by yuters » Wed, Jan 29, 2020, 2:45 PM UTC

tyingq wrote: post #40

kabacha wrote: post #11

The toxicity towards web-scraping is really what makes me lose hope in the current web. People want their data to be public and all of the benefits that comes with public data but then they want to chose who gets to see it - it's a complete and utter paradox. This precedent doesn't really mean much but is definitely step in the right direction.

There's plenty of grey here. For example, scrapers that try to check people in for flights to get better seats. Some that tried to charge for that. That creates problems, where some customers benefit at the expense of others, high load on a "locking type" piece of code, etc. Similar for ticket sales for concerts, and probably other spaces. There are also companies that provide added value by compiling and correlating…

Another problem with web scraping is a B2B website offering services for B2C companies to better reach consumers. It can be a tricky thing to do without basically giving your clients list to all your competitors.

Re: Congrats! Web scraping is legal! (US precedent)

#59

Post by mrosett » Wed, Jan 29, 2020, 2:45 PM UTC

I'm glad scraping isn't criminal. Applying the CFAA here is ridiculous. But saying LinkedIn can't put technical measures in place to prevent scraping seems like a huge stretch to me. Why should they have to pay server costs for persistent scraping, particularly from a company that is actively trying to harm them?

Re: Congrats! Web scraping is legal! (US precedent)

#60

Post by danShumway » Wed, Jan 29, 2020, 2:45 PM UTC

Wowfunhappy wrote: post #29

kabacha wrote: post #11

The toxicity towards web-scraping is really what makes me lose hope in the current web. People want their data to be public and all of the benefits that comes with public data but then they want to chose who gets to see it - it's a complete and utter paradox. This precedent doesn't really mean much but is definitely step in the right direction.

I want web scraping to be legal—but, is it really contradictory to say "I want this data to be accessible to real humans only"? Any person can post on Hacker News. However, if someone made a bot to post to Hacker News, I think most of us would be pretty upset.

I dunno.

I'm on Hackernews to see interesting articles and read interesting conversation. If a bot can post interesting articles and make interesting conversation, I'm not sure I care that it's a bot. And if a human can't do those things, I'm not sure I care whether or not they're 'real'.

https://xkcd.com/810/

That focus on "we don't care if you self-promote, we don't care why you're here, we just want you to be a good citizen" is part of why I like HN.

It's not completely black and white, but in general I believe that users online have the Right to Delegate[0]. That right should only be legally taken away if there's a really, unbelievably compelling social justification for doing so. I am pretty skeptical that banning web scraping has that kind of justification.

[0]: https://anewdigitalmanifesto.com/#right-to-delegate