This is pretty much what the DMCA and other copyright mechanisms were made for. Send your request to Google and have them delisted.
Ask HN: Dealing with a competitor who is scraping my content and ranking higher
11–20 of 130 posts
Re: Ask HN: Dealing with a competitor who is scraping my content and ranking higher
#12If you are able to detect them, serve them garbage data instead of blocking them. Mix in some false entries etc. If you completely block them they will work hard to work around the block, but if you give them some bad data randomly it will take them longer to notice.
Also, you can use the fake data as evidence they scraped you.
Re: Ask HN: Dealing with a competitor who is scraping my content and ranking higher
#13This is a good article I've followed in the past. idk how helpful this is, you might have already gone through it. https://github.com/JonasCz/How-To-Prevent-Scraping
That's a great resource (had to share it on HN LOL)
Re: Ask HN: Dealing with a competitor who is scraping my content and ranking higher
#14I thought Cloudflare's "under attack" mode can prevent scraping bots. Correct me if I'm wrong.
Re: Ask HN: Dealing with a competitor who is scraping my content and ranking higher
#15Have you tried filing a spam report with Google?
Re: Ask HN: Dealing with a competitor who is scraping my content and ranking higher
#16Can you implement a captcha somewhere? Or figure a way to obfuscate your html if that’s what he’s scraping. Something that can scramble the css class names each build.
It really depends on how complex/simple the page is. If its small site with little context you can usually traverse from parent body ignoring classes/ids. However, for more complex sites changing class/id values on each build can certainly help make this a lot more challenging.
Ideally changing the children counts to prevent things like $(body).children().children()[3].html().
Re: Ask HN: Dealing with a competitor who is scraping my content and ranking higher
#17[deleted]
Re: Ask HN: Dealing with a competitor who is scraping my content and ranking higher
#18If you are able to detect them, serve them garbage data instead of blocking them. Mix in some false entries etc. If you completely block them they will work hard to work around the block, but if you give them some bad data randomly it will take them longer to notice. Also, you can use the fake data as evidence they scraped you.
And also to try to obtain a pattern of what request headers are common... Try to create a special hash from the request headers. Then find a way to infiltrate the data with it so you can identify them better
Re: Ask HN: Dealing with a competitor who is scraping my content and ranking higher
#19Write a few things. Register them with the copyright office. Then post them and wait for them to copy it. Now you can sue them for the max and strong-arm them to take it all down or face obvious legal fees and punishments over 100k.
Re: Ask HN: Dealing with a competitor who is scraping my content and ranking higher
#20Find a way to contact the owner of the site and bait them with something like an acquisition offer, you might be able to use the email in the whois data if there is one. The point is to get the owner to click a link you provide, behind this link you want to log IP/Headers, maybe do some JS fingerprinting/etc... It is unlikely they will open this link over VPN/Tor. If you feel like it check your server logs to see if they have accessed your site from that IP and when. Then send a C&D containing all this information and tell them that you have legally contacted their ISP for their subscriber information and not only will they be sued for violating DMCA for possessing your data but that they will also get hit with the CFAA for circumventing protections against access and that they will enjoy a five year sentence in a federal pound me in the ass prison.