Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Legality of Web Scraping User-Submitted Content?

news.ycombinator.com

Post reply on HN
Print view View original

11–20 of 48 posts

Re: Ask HN: Legality of Web Scraping User-Submitted Content?

#11

Post by tstegart »

brandon272 wrote: post #3
tstegart wrote: post #2

Well, there's the real world answer, and the legal answer. Not too sure about the legal answer, thats always murky. Some things are copyrighted, some aren't. You can't just take someone's story, or art, and use it on your own site without permission. Usually the creator has all rights until they are given away. Even if they have uploaded it to another site, that doesn't mean they have given you permission though. On…

I guess my followup with that would be this: What if the user is submitting content that they would very clearly want re-distributed? I guess my question with that is, if you take the end-user who is submitting the content out of the equation, does the site that is being scraped from have any kind of leg to stand on if they don't want you scraping their content (excluding technical means that they might implement), a…

The myspace suicide case really muddles this and makes it a problem. If the site you want to scrape from has TOS that say you can't re-use the info, (and it most likely would), then re-using the info would be a violation of the terms of service. Basically, in the MySpace case the feds are trying to make it a crime to violate a website's terms of service.

But like someone else said, if someone wants their content distributed, the user is not going to give you any trouble, and if they do, you're very protected if you take the offended material down right away, as someone else mentioned.

The website you're stealing from does have legal measures they can take, especially if you're directly competing with them. I think someone else mentioned that they could also just reconfigure their website to mess up your scrape, which is probably what they'll keep doing. They'll also likely publicize your bad business practices and you'll end up with a horrible reputation. Nobody likes a copier. Its like people who steal designs. They hardly ever outperform the website they're copying.

Re: Ask HN: Legality of Web Scraping User-Submitted Content?

#12

Post by pedalpete »

I'm not a lawyer, but had to do a bunch of legal research regarding this topic for some of my sites.

I think what most of the responses so far are missing is the importance of accrediting the content to the content owner (likely the site, not the contributing users), and providing a link to the source.

Check out this pdf http://www.law.berkeley.edu/journals/btlj/articles/vol16/sab...">Sableman's authorized linking and search on Google v. Perfect 10.

You haven't really given much to go on with respect to what you are scraping, and what you plan to do with it. But I think a bunch of common sense and ensuring that your site in no way harms the original source's site (defamation, etc), are the most important things to consider.

Hope this helps.

Re: Ask HN: Legality of Web Scraping User-Submitted Content?

#13

Post by jonmc12 »

timcederman wrote: post #6
jonmc12 wrote: post #4

A few years back I wanted to set up a menu service - putting a bunch of restaurant menus online so that others could search. My lawyer advised that because the restaurant made these materials open in the public domain, we could basically do whatever we wanted with them. As if they were public property. I'm not sure how this relates to other kind of content from a legal standpoint, but I've used it to ask myself 'did…

Woah, that is seriously not correct advice. Was your lawyer even a copyright lawyer?

We consulted a copyright lawyer, but I never moved forward with this, so did not really did too deep.

I've noticed there are plenty of menu services that take this same approach.. just do a google search and you will see there are many guides putting menus online without a license from the restaurant. What am I missing?

Re: Ask HN: Legality of Web Scraping User-Submitted Content?

#14

Post by micks56 »

It is most likely not legal.

The user owns copyright to the article or story that he wrote. This ownership of copyright gives the user the right to decide how it is distributed. For you to use the material legally, you must get permission of the copyright holder. The user may be the copyright holder, or the site may be if the user transfers the ownership. Either way, someone owns the copyright and it is not you. You need permission.

There are two defenses to copyright infringement: 1) fair use, and 2) parody.

Parody probably doesn't fit here. So that means you need to make a case for fair use.

Fair use has 4 elements:

1: The purpose of the use (commercial vs. non-commercial/educational) - if you are going to make money on this, fair use is out.

2. Nature of the copyrighted work - This doesn't really apply here, so I won't go into the lengthy explanation.

3. Amount of portion used in relation to the whole - Did you extract a quote? That is probably ok. Did you copy the entire article? Probably not ok.

4. Effect upon the market - if your site harms the market of the other site, no fair use.

Re: Ask HN: Legality of Web Scraping User-Submitted Content?

#17

Post by timcederman »

jonmc12 wrote: post #13
timcederman wrote: post #6

Earlier quoted context omitted.

Woah, that is seriously not correct advice. Was your lawyer even a copyright lawyer?

We consulted a copyright lawyer, but I never moved forward with this, so did not really did too deep. I've noticed there are plenty of menu services that take this same approach.. just do a google search and you will see there are many guides putting menus online without a license from the restaurant. What am I missing?

They are not 'open in the public domain' nor can they be treated as 'public property'. Somebody wrote those menus from scratch and as such they own the copyright. If you wanted to take an excerpt from the menu and comment on it, then that is fair use. When reproducing it in full, they technically can request you remove it, or any of the other avenues for restitution copyright law affords. Most won't because it benefits them to have their menu available and widely disseminated - which is why they give them away in the first place. But don't confuse them giving the content away as it then being your property you can then do what you please with.

Re: Ask HN: Legality of Web Scraping User-Submitted Content?

#18

Post by micks56 »

drewcrawford wrote: post #5

In the legal world, whenever text is set in fixed form, it is automatically copyrighted by its author, whether they claim it or not, unless they specifically say otherwise. "Specifically say otherwise" probably includes anything they might have agreed to in the TOS for a particular site, although the legal waters there are largely untested. The only exemption to this auto-copyright is statistics, phone numbers, or ot…

DMCA safe harbor applies when you are a service provider.

Examples are an ISP that merely provides access to the internet. The ISP cannot be sued for copyright infringement just because infringing bits passed through its servers.

Also, a message board that posts whatever a user writes would obtain DMCA safe harbor. They just provide a service and don't screen out for content. An example is Craigslist.

This person's site DOES NOT enjoy DMCA safe harbor. He scraped the other site and populated his own. He is not even acting as a service provider as the DMCA statute prescribes.

Re: Ask HN: Legality of Web Scraping User-Submitted Content?

#19

Post by tstegart »

timcederman wrote: post #6
jonmc12 wrote: post #4

A few years back I wanted to set up a menu service - putting a bunch of restaurant menus online so that others could search. My lawyer advised that because the restaurant made these materials open in the public domain, we could basically do whatever we wanted with them. As if they were public property. I'm not sure how this relates to other kind of content from a legal standpoint, but I've used it to ask myself 'did…

Woah, that is seriously not correct advice. Was your lawyer even a copyright lawyer?

Yeah, I'm not sure "chicken, $10" can be copyrighted. The image of a menu might be a problem, but the text? Not to mention his lawyer may have been giving practical advice (hopefully acknowledged as such): what restaurant would really argue with their menu being available?
Post reply on HN