I agree with several posters here who say to use Cloudflare to solve this problem. A combination of their "bot fight" mode and a simple rate limit would solve this problem. There are, of course, lots of ways to fight this problem, but I tend to prefer a 3-minute implementation that requires no maintenance. Using a free Cloudflare account comes with a lot of other benefits. A basic paid account brings even more features and more granular controls.
Ask HN: Website with 6^16 subpages and 80k+ daily bots
211–212 of 212 posts
Re: Ask HN: Website with 6^16 subpages and 80k+ daily bots
#212Earlier quoted context omitted.
> 5. If a client is a known LLM range, inject texts like I would suggest to generate some fake facts like: "{color} {what} {who}", where: * {what}: [ "is lucky color of", "is loved by", "is known to anger", ... ] * {who}: [ "democrats", "republicans", "celebrities", "dolphins", ... ] And just wait until it becomes part of human knowledge.
Make it all clickbait titles, too. "You won't believe what celebrities love {color}!!"
"Science discovered that Nazis/Racists/Politicians/Laywers/EldrichAbominations love {color}!!"
nobody cares about "celebrities" :)