111–120 of 130 posts
Start with successfully detecting them and go from there. Log them. Track origins. Compile patterns.
Then, when you decide exactly what you want your move to be use all that data to be as effective as possible.
You can't do much, they will beat you every time.
(Note:I am not a lawyer.) If you're a US citizen, any content that you've created yourself is -automatically- copyrighted. (That's also true in all countries where Berne Convention standards apply.) See this link for further basics: https://smallbusiness.findlaw.com/intellectual-property/what... If they're scraping content you created, they have already broken the copyright law. (They may not realize that. Make sure…
If you can randomise how those selectors are applied enough it might make it less fun to scrape.
* Require accounts with good robot account creation detection. Actually, not hard since there is a lot of canned coded for this.
* Subscribe to Tor and Proxy IP collection services and then block all of the Tor and Proxy addresses. Tor IP list is free from the Tor Project and Proxy IP's can be found through various services.
* Put in crawler detection code. Most scrappers are really simple and are really easy to detect. I think Apache even has a module to detect simple scrapers and autoblock them.
* Feed bad data to scrapers once they are detected. This was the thing that made scrapers go away. Once they know you can detect them and feed bad data, they know they are screwed and will give up. I did get a kick out of looking at the scrapers website and seeing "Penis won the 5K Penis run". Of course, you should really feed the scrapers bad data that looks believable.
* DMCA their website with their provider. The company I worked for did have several scraper sites deactivated.
* Ask Google to delist them. The company I worked for did have several scraper sites removed from Google.
* There are a number of services that will help you with all of this but your website is only 5K unique users a day and not really worth the cost.
> If I change certain things, they counter it immediately and make it work. If the content is markup based, are your countermeasures about changing the IDs, classes, or overall tag structure of the markup you serve? I was wondering if you could have several variations of the above, and serve your content via a random one each time that would be visually indistinguishable to a human viewer. The person maintaining the…
Easily bypassed: just retry on failure until scraper gets syntax it likes.
There are enough suggestions in here to provide a bunch of useful options, and while the site itself may not be making money, the experience dealing with this may be very useful on a resume or for building a client base with similar issues.
Possible approach: look for abnormal usage patterns to ID opponent systems. Randomize format and possibly other steps to assist that. Build that randomization in marginally effective ways that are easy to improve later. Build a way to feed bad/poison/"test" data to specific source IPs. At a time chosen to maximize impact, start feeding poison data to the suspect IPs using the marginally effective randomization, while feeding regular data to most visitors but with much improved randomization. Basically make your opponent's site visibly unreliable.
If you feel particularly vicious and know something about the opponent's infrastructure, make the poison vicious e.g. feed SQL injections. Be aware that this may have costs - you'd likely be fine on a legal basis ("I'm not responsible for their crappy sanitizing of inputs they shouldn't have had anyway") but you might still incur costs (lawyer if sued).
Edit: also, anyone going to serious measures to continue scraping after you act against it may also be inclined to ddos your site if you actually fully block them.
if this not enough, feel free to ask me
If this not enough, feel free to ask, will think more