Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Dealing with a competitor who is scraping my content and ranking higher

news.ycombinator.com

Post reply on HN
Print view View original

61–70 of 130 posts

Re: Ask HN: Dealing with a competitor who is scraping my content and ranking higher

#61

Post by amelius »

maxpupmax wrote: post #4

ideas: 1. Create content that can't be scraped. I'm not sure exactly what your "content" is in this case, but images can be watermarked, text can be given lots of references to your own brand and service, ect. 2. Submit legal requests to google to remove the content. Enough violations can get their domain blacklisted. I've done this successfully in the past for competitors using my trademark without permission to get…

You can also apply steganography to your data, i.e., embed a copyright message that can't be detected or easily removed.

https://en.wikipedia.org/wiki/Steganography

Re: Ask HN: Dealing with a competitor who is scraping my content and ranking higher

#62

Post by amelius »

Witesse wrote: post #37

When you say "my content", are you referring to content that you created, like a blog or online resource or are you referring to content your users created? Also is the information they're copying factual in nature (like a map or results of a calculation)? If it's factual in nature: It's not protected by copyright. If it's content your users created: You are not the copyright holder and cannot file a DMCA takedown no…

> If it's content your users created: You are not the copyright holder and cannot file a DMCA takedown notice.

But you can still have copyright on some of your data. If the other company simply copies everything, you can take legal action.

Re: Ask HN: Dealing with a competitor who is scraping my content and ranking higher

#64

Post by timrichard »

> If I change certain things, they counter it immediately and make it work.

If the content is markup based, are your countermeasures about changing the IDs, classes, or overall tag structure of the markup you serve? I was wondering if you could have several variations of the above, and serve your content via a random one each time that would be visually indistinguishable to a human viewer. The person maintaining the scraper would have to have seen and adapted to all of them to get all your new content reliably. Not an impossible hurdle, but they might try easier targets if too many barriers are in the way.

Re: Ask HN: Dealing with a competitor who is scraping my content and ranking higher

#66

Post by blantonl »

oliwarner wrote: post #57
8bitsrule wrote: post #49

(Note:I am not a lawyer.) If you're a US citizen, any content that you've created yourself is -automatically- copyrighted. (That's also true in all countries where Berne Convention standards apply.) See this link for further basics: https://smallbusiness.findlaw.com/intellectual-property/what... If they're scraping content you created, they have already broken the copyright law. (They may not realize that. Make sure…

Should be recognised that not all data is eligible for copyright. "Facts" is a broad category of exemption with strange case law. It may be relevant here if the data being scraped is available elsewhere.

Yes indeed. See:

https://en.wikipedia.org/wiki/Feist_Publications,_Inc.,_v._R....

Re: Ask HN: Dealing with a competitor who is scraping my content and ranking higher

#67

Post by Rjevski »

Add sensible rate-limits (modelled after an average user's expected usage) and present a captcha after that. Make it per-IP.

Sure, the crawler can change IPs, but I doubt they have unlimited addresses and eventually they will run out.

Another option (if you can reliably detect the scraper) is to poison their data by sending them bad (but valid-looking) data.

Re: Ask HN: Dealing with a competitor who is scraping my content and ranking higher

#68

Post by [deleted] »

badminton1 wrote: post #48
SyneRyder wrote: post #27

If you haven't already, try adding some "trap streets" to your data. Map makers occasionally include streets that don't exist, so if a competitors map includes it too, it's clear that the competitor copied it: https://en.wikipedia.org/wiki/Trap_street I did that with an online marketing dictionary I wrote years ago, some of the definitions included strange usage examples that contained the names of several of my frie…

Similar to trap streets are "phantom settlements", aka "paper towns", which are fake towns rather than streets. Now, this idea is not limited to maps: Google used trap search results to catch Microsoft using Internet Explorer to scrap Google search results: https://googleblog.blogspot.com/2011/02/microsofts-bing-uses...

[deleted]

Re: Ask HN: Dealing with a competitor who is scraping my content and ranking higher

#69

Post by ATsch »

oliwarner wrote: post #57
8bitsrule wrote: post #49

(Note:I am not a lawyer.) If you're a US citizen, any content that you've created yourself is -automatically- copyrighted. (That's also true in all countries where Berne Convention standards apply.) See this link for further basics: https://smallbusiness.findlaw.com/intellectual-property/what... If they're scraping content you created, they have already broken the copyright law. (They may not realize that. Make sure…

Should be recognised that not all data is eligible for copyright. "Facts" is a broad category of exemption with strange case law. It may be relevant here if the data being scraped is available elsewhere.

Note also that in some places, databases are explicity exempt from this "facts" exemption if you made considerable effort in compiling the database. (IANAL)

Re: Ask HN: Dealing with a competitor who is scraping my content and ranking higher

#70

Post by fimdomeio »

8bitsrule wrote: post #49

(Note:I am not a lawyer.) If you're a US citizen, any content that you've created yourself is -automatically- copyrighted. (That's also true in all countries where Berne Convention standards apply.) See this link for further basics: https://smallbusiness.findlaw.com/intellectual-property/what... If they're scraping content you created, they have already broken the copyright law. (They may not realize that. Make sure…

Also not a lawyer buy If you're in europe I believe there's also something called database rights which gives some protection to people that gather data from different sources event if they are not content creator.
Post reply on HN