[1] https://unicode.org/cldr/utility/confusables.jsp?a=test&r=No...
Ask HN: Dealing with a competitor who is scraping my content and ranking higher
71–80 of 130 posts
Re: Ask HN: Dealing with a competitor who is scraping my content and ranking higher
#72> If I change certain things, they counter it immediately and make it work. If the content is markup based, are your countermeasures about changing the IDs, classes, or overall tag structure of the markup you serve? I was wondering if you could have several variations of the above, and serve your content via a random one each time that would be visually indistinguishable to a human viewer. The person maintaining the…
Re: Ask HN: Dealing with a competitor who is scraping my content and ranking higher
#73Add sensible rate-limits (modelled after an average user's expected usage) and present a captcha after that. Make it per-IP. Sure, the crawler can change IPs, but I doubt they have unlimited addresses and eventually they will run out. Another option (if you can reliably detect the scraper) is to poison their data by sending them bad (but valid-looking) data.
Re: Ask HN: Dealing with a competitor who is scraping my content and ranking higher
#74This is pretty much what the DMCA and other copyright mechanisms were made for. Send your request to Google and have them delisted. https://support.google.com/legal/answer/3110420?hl=en
Re: Ask HN: Dealing with a competitor who is scraping my content and ranking higher
#75Re: Ask HN: Dealing with a competitor who is scraping my content and ranking higher
#76Earlier quoted context omitted.
The spammers & scrapers seem to have found their way around ReCaptcha. I made a simple spam filter for my website comment form that logs every submission, and a lot of the spam being sent (maybe 20%?) has a valid Google Recaptcha. I'm thinking of removing Recaptcha because simple keyword based filters have been more effective than ReCaptcha has been.
ReCaptcha and the original two-word based one are just Google using your visitors as 'mechanical turks' - by solving ReCaptcha et al, they are actually helping Google apply identification tags to images in streetview, and previously the Google books collection. An afternoons coding would get you an independent self-hosted Captcha system that would be just as effective (if not more so, as it's proprietary) and probabl…
When your site is specifically targetted, captchas will not work
Re: Ask HN: Dealing with a competitor who is scraping my content and ranking higher
#77Earlier quoted context omitted.
They're probably using https://2captcha.com/ They pay 50c per 2 hours to the workers, and API access costs 50c-$1 per 2000 solved captchas. I think most of the people doing this are in India, where US 50c for a couple hours translates well enough. (Completely naive about the truth of the situation though.)
One of the regular spam messages mentions the software they're using ("You read this, so you know it works!"), so it's pretty clear that ReCaptcha has already been defeated. I'd mention the software, but I'd rather not give them publicity. Others have been able to beat reCaptcha too, though: https://www.youtube.com/watch?v=fsF7enQY8uI
Re: Ask HN: Dealing with a competitor who is scraping my content and ranking higher
#78> If I change certain things, they counter it immediately and make it work. If the content is markup based, are your countermeasures about changing the IDs, classes, or overall tag structure of the markup you serve? I was wondering if you could have several variations of the above, and serve your content via a random one each time that would be visually indistinguishable to a human viewer. The person maintaining the…
Re: Ask HN: Dealing with a competitor who is scraping my content and ranking higher
#79Find a way to contact the owner of the site and bait them with something like an acquisition offer, you might be able to use the email in the whois data if there is one. The point is to get the owner to click a link you provide, behind this link you want to log IP/Headers, maybe do some JS fingerprinting/etc... It is unlikely they will open this link over VPN/Tor. If you feel like it check your server logs to see if…
I don’t disagree with the content of your post and I know this is a US cultural trope at this point, but trivialising rape benefits precisely no-one.
If you already knew this, then I guess the applicable movie quote this time is "Lighten up, Francis" from Stripes.
Re: Ask HN: Dealing with a competitor who is scraping my content and ranking higher
#80This is pretty much what the DMCA and other copyright mechanisms were made for. Send your request to Google and have them delisted. https://support.google.com/legal/answer/3110420?hl=en
I reported it to google, and they removed it from search results in a few days. I also reported it to the site's host, who took it down completely.