Live data from Hacker News

Ask HN: Dealing with a competitor who is scraping my content and ranking higher

news.ycombinator.com

81–90 of 130 posts

Re: Ask HN: Dealing with a competitor who is scraping my content and ranking higher

#81

This is pretty much what the DMCA and other copyright mechanisms were made for. Send your request to Google and have them delisted. https://support.google.com/legal/answer/3110420?hl=en

Does anything stop the scraper from using DCMA on the original site?

Re: Ask HN: Dealing with a competitor who is scraping my content and ranking higher

#82
post #9

Google ReCaptcha is a pretty low friction way for (most) users to get your content. You can do this while serving your real content to Google.

ReCaptcha is very high friction for many users, particularly those not logged into Google services

For example I have never reached the end of their 'identify the road signs' challenge and if I encounter a site that uses them I'll just close the tab. Even Google Search.

Much better to use rotating series of questions related to your domain of data.

Re: Ask HN: Dealing with a competitor who is scraping my content and ranking higher

#83

If you haven't already, try adding some "trap streets" to your data. Map makers occasionally include streets that don't exist, so if a competitors map includes it too, it's clear that the competitor copied it: https://en.wikipedia.org/wiki/Trap_street I did that with an online marketing dictionary I wrote years ago, some of the definitions included strange usage examples that contained the names of several of my frie…

This is the same as "poison pills" right? A directory website would include some fake entries so that if they pop up on other sites they could have only come from copying the original directory.

A scraper could get around that by x-referencing the data with another source, if possible.

Re: Ask HN: Dealing with a competitor who is scraping my content and ranking higher

#84
post #81

This is pretty much what the DMCA and other copyright mechanisms were made for. Send your request to Google and have them delisted. https://support.google.com/legal/answer/3110420?hl=en

Does anything stop the scraper from using DCMA on the original site?

They'd lose their source of data

Re: Ask HN: Dealing with a competitor who is scraping my content and ranking higher

#86

(Note:I am not a lawyer.) If you're a US citizen, any content that you've created yourself is -automatically- copyrighted. (That's also true in all countries where Berne Convention standards apply.) See this link for further basics: https://smallbusiness.findlaw.com/intellectual-property/what... If they're scraping content you created, they have already broken the copyright law. (They may not realize that. Make sure…

What if I created my content by slightly altering other content? Like writing a news article from a Reuters press release, how much difference does there need to be?

Seems like a pretty hard line to draw.

Re: Ask HN: Dealing with a competitor who is scraping my content and ranking higher

#87

(Note:I am not a lawyer.) If you're a US citizen, any content that you've created yourself is -automatically- copyrighted. (That's also true in all countries where Berne Convention standards apply.) See this link for further basics: https://smallbusiness.findlaw.com/intellectual-property/what... If they're scraping content you created, they have already broken the copyright law. (They may not realize that. Make sure…

Also not a lawyer buy If you're in europe I believe there's also something called database rights which gives some protection to people that gather data from different sources event if they are not content creator.

Correct, the database directive: https://en.wikipedia.org/wiki/Database_Directive

Aka Directive 96/9/EC of the European Parliament

Re: Ask HN: Dealing with a competitor who is scraping my content and ranking higher

#88
post #67

Add sensible rate-limits (modelled after an average user's expected usage) and present a captcha after that. Make it per-IP. Sure, the crawler can change IPs, but I doubt they have unlimited addresses and eventually they will run out. Another option (if you can reliably detect the scraper) is to poison their data by sending them bad (but valid-looking) data.

Proxies are actually shockingly cheap

Re: Ask HN: Dealing with a competitor who is scraping my content and ranking higher

#89
post #86

(Note:I am not a lawyer.) If you're a US citizen, any content that you've created yourself is -automatically- copyrighted. (That's also true in all countries where Berne Convention standards apply.) See this link for further basics: https://smallbusiness.findlaw.com/intellectual-property/what... If they're scraping content you created, they have already broken the copyright law. (They may not realize that. Make sure…

What if I created my content by slightly altering other content? Like writing a news article from a Reuters press release, how much difference does there need to be? Seems like a pretty hard line to draw.

You sound like you're asking this question as if it's a brand new question that nobody has ever thought of before, instead of something core to the entire idea of copyright: https://en.wikipedia.org/wiki/Derivative_work

Yes, it is a hard line to draw. It is advisable that if your business is dependent on being on the right side of copyright issues that you simply stay well away from the line.

Re: Ask HN: Dealing with a competitor who is scraping my content and ranking higher

#90
post #83

If you haven't already, try adding some "trap streets" to your data. Map makers occasionally include streets that don't exist, so if a competitors map includes it too, it's clear that the competitor copied it: https://en.wikipedia.org/wiki/Trap_street I did that with an online marketing dictionary I wrote years ago, some of the definitions included strange usage examples that contained the names of several of my frie…

This is the same as "poison pills" right? A directory website would include some fake entries so that if they pop up on other sites they could have only come from copying the original directory. A scraper could get around that by x-referencing the data with another source, if possible.

Though that creates quite a bit of extra overhead, and you only really have to catch them once.
Post reply on HN