Ask HN: Legality of Web Scraping User-Submitted Content? • Hacker News Board

Re: Ask HN: Legality of Web Scraping User-Submitted Content?

#41

Post by dhimes » Thu, Aug 07, 2008, 12:37 AM UTC

micks56 wrote: post #29

lacker wrote: post #22

Earlier quoted context omitted.

It is not true that commercial purpose precludes a fair use defense. From wikipedia: "While commercial copying for profit work may make it harder to qualify as fair use, it does not make it impossible." see: http://en.wikipedia.org/wiki/Fair_use

In this case it does. The use in the case you are referring to had several extra elements which prompted the court to not find copyright infringement. 2 Live Crew was fair use because their work was transformative. They took a beat and changed words to a Roy Orbison song. They did not blindy copy, as the question asker will be doing. They took the score, not the words. The poster is taking the story, not quotes. His…

And, they lost the Van Halen lawsuit.

Re: Ask HN: Legality of Web Scraping User-Submitted Content?

#42

Post by tstegart » Thu, Aug 07, 2008, 12:54 AM UTC

mattmaroon wrote: post #39

aneesh wrote: post #16

Earlier quoted context omitted.

Attorneys are expensive, Hacker News is free.

Attorneys know what they're talking about where law is concerned, hackers don't but often don't realize that. Asking a hacker for legal advice is just as bad as asking a lawyer for programming advice. Actually it's probably worse, since nobody ever ended up in jail for a coding error.

Oh, but we wish they did sometimes :)

Re: Ask HN: Legality of Web Scraping User-Submitted Content?

#43

Post by nickb » Thu, Aug 07, 2008, 1:36 AM UTC

drandall wrote: post #31

tstegart wrote: post #20

Earlier quoted context omitted.

Meh, if everyone listened to their lawyer we wouldn't have YouTube or BitTorrent :)

insert JP Morgan quote

For those who've never heard of this gem :)

Well, I don't know as I want a lawyer to tell me what I cannot do. I hire him to tell how to do what I want to do. ~ J. P. Morgan

Re: Ask HN: Legality of Web Scraping User-Submitted Content?

#44

Post by sh1mmer » Thu, Aug 07, 2008, 1:39 AM UTC

mattmaroon wrote: post #15

One of the most dangerous things you can do is rely on non-lawyers for legal advice. Ask an attorney.

That's true.

But some answers are so blazingly obvious or common knowledge that it's worth asking. It might at least speed up a conversation with a lawyer saving many hundreds of $s.

Re: Ask HN: Legality of Web Scraping User-Submitted Content?

#45

Post by sh1mmer » Thu, Aug 07, 2008, 1:46 AM UTC

My "not a lawyer" answer to this is:

It depends on the terms of service of the site.

#1 The TOS of the site may not let you use a robot on their site at all

#2 The TOS will define who owns the user generated content (UGC), either the user or the site

#3 Depending who owns the UGC you may or may not be able to scrape it, if it's the site it's against their TOS if it's the user you would need permission from the user.

#4 As other people have said fair use might come into play. If the site owns the material using a single user contribution might be fair use within the context of the whole site. If the users own the content you are likely to be using all their content, thereby not able to use fair use.

Again all of these are my observations. Hopefully it will give you something to think about. If you are starting a business based on this, you do need to consult a lawyer. Also starting a business based on page scraping is a pretty risky thing to do. If the scraped site turn you off you could be pretty screwed.

Re: Ask HN: Legality of Web Scraping User-Submitted Content?

#46

Post by mattmaroon » Thu, Aug 07, 2008, 2:00 AM UTC

tstegart wrote: post #20

mattmaroon wrote: post #15

One of the most dangerous things you can do is rely on non-lawyers for legal advice. Ask an attorney.

Meh, if everyone listened to their lawyer we wouldn't have YouTube or BitTorrent :)

I wouldn't suggest you listen to them blindly, but you're crazy if you think YouTube didn't ask actual attorneys for a risk assessment (their VCs would have required that). They certainly didn't ask a bunch of hackers.

Re: Ask HN: Legality of Web Scraping User-Submitted Content?

#47

Post by gcv » Thu, Aug 07, 2008, 2:34 AM UTC

mattmaroon wrote: post #39

aneesh wrote: post #16

Earlier quoted context omitted.

Attorneys are expensive, Hacker News is free.

Attorneys know what they're talking about where law is concerned, hackers don't but often don't realize that. Asking a hacker for legal advice is just as bad as asking a lawyer for programming advice. Actually it's probably worse, since nobody ever ended up in jail for a coding error.

How do you know some readers here don't read Supreme Court decisions because they're interested in law, even if they never went to law school? How do you know your lawyer doesn't hack on Arc on weekends? :)

Seriously, though: of course the person asking the question should retain a lawyer for definitive guidance. However, asking around to get a feel for the opinions of intelligent people makes sense, too. It might, at the very least, clarify the questions, and it will help detect if the lawyer's words make sense. While attorneys are valuable, they are also fallible, and their advice should be as critically considered as any other.

Re: Ask HN: Legality of Web Scraping User-Submitted Content?

#48

Post by mattmaroon » Thu, Aug 07, 2008, 2:08 PM UTC

gcv wrote: post #47

mattmaroon wrote: post #39

Earlier quoted context omitted.

Attorneys know what they're talking about where law is concerned, hackers don't but often don't realize that. Asking a hacker for legal advice is just as bad as asking a lawyer for programming advice. Actually it's probably worse, since nobody ever ended up in jail for a coding error.

How do you know some readers here don't read Supreme Court decisions because they're interested in law, even if they never went to law school? How do you know your lawyer doesn't hack on Arc on weekends? :) Seriously, though: of course the person asking the question should retain a lawyer for definitive guidance. However, asking around to get a feel for the opinions of intelligent people makes sense, too. It might, a…

Well, an attorney's opinion on the law is certainly not infallible, but it should be considered a lot less critically than a hacker's.

The danger is that misinformation is often worse than no information at all, and asking a bunch of non-attorneys for legal advice is sure to net you a ton of it.