Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Ethics and laws regarding scraping websites?

news.ycombinator.com

Post reply on HN
Print view View original

21–25 of 25 posts

Re: Ask HN: Ethics and laws regarding scraping websites?

#21

Post by webwright »

calvins wrote: post #20
webwright wrote: post #9

Search engines all scrape... Titles, meta data, and some or all of the content. I think a good rule of thumb is to consider whether the scraping target will benefit from being scraped. Most sites are delighted to be scraped by Google. Will your scraping drive sales/visitors to the target? Or will it cost sales/visitors? Will you link back (which helps them from an seo standpoint)?

Search engines do not all scrape. They scrape if the owners have indicated they wish their site to be scraped by not forbidding access via the robots.txt file.

Um, so yeah-- they all scrape-- like I said.

But no, they don't scrape all sites.

Re: Ask HN: Ethics and laws regarding scraping websites?

#22

Post by 0x44 »

dazzawazza wrote: post #3

I think this is a very grey area. If it's on the internet it's scrapable. Does that mean you can use it? You really need a lawyer. Take for example Amazon reviews. They are there to be scraped but some of them are not available via the web service API due to copyright restrictions. That's a clear sign to me that they don't wouldn't want this information scraped. I'm not sure how a judge would see it though. Another e…

Only the layout of the TV listing information is copyrighted. You can't copyright facts.

Re: Ask HN: Ethics and laws regarding scraping websites?

#23

Post by dazzawazza »

0x44 wrote: post #22
dazzawazza wrote: post #3

I think this is a very grey area. If it's on the internet it's scrapable. Does that mean you can use it? You really need a lawyer. Take for example Amazon reviews. They are there to be scraped but some of them are not available via the web service API due to copyright restrictions. That's a clear sign to me that they don't wouldn't want this information scraped. I'm not sure how a judge would see it though. Another e…

Only the layout of the TV listing information is copyrighted. You can't copyright facts.

This seems reasonable. However I seem to remember that the actual listings in the UK are 'owned' by some central authority (I forget who but it's got something to do with the Radio Times) and you license the right to publish them.

While it's a fact that a show is on it's also content owned by some authority. This is where lawyers make their money.

Re: Ask HN: Ethics and laws regarding scraping websites?

#24

Post by 0x44 »

dazzawazza wrote: post #23
0x44 wrote: post #22

Earlier quoted context omitted.

Only the layout of the TV listing information is copyrighted. You can't copyright facts.

This seems reasonable. However I seem to remember that the actual listings in the UK are 'owned' by some central authority (I forget who but it's got something to do with the Radio Times) and you license the right to publish them. While it's a fact that a show is on it's also content owned by some authority. This is where lawyers make their money.

My apologies, the copyright laws of the UK are quite different than those of the US. I don't know whether or not facts can be copyrighted there.
Post reply on HN