It is better for a million AI crawlers to get through than for even one search index crawler, that might expose the knowledge on your site to someone who needs it, to be denied.
Show HN: I built an SDK that scrambles HTML so scrapers get garbage
21–30 of 46 posts
Re: Show HN: I built an SDK that scrambles HTML so scrapers get garbage
#22All I want is an API for my AI, you can ask me for my public key, if you want my human identity verified. The collateral damage of this bot hunting is the emergence of personal AIs. Do we really want that? It feels regressive. (I see the hypocrisy here, we are fighting the scrapers that feed the LLMs that runs our personal agents)
Re: Show HN: I built an SDK that scrambles HTML so scrapers get garbage
#23it's white text and the shader background is also mostly white
Re: Show HN: I built an SDK that scrambles HTML so scrapers get garbage
#24I'm surprised that you don't appear to be using it on obscrd.dev lol
Re: Show HN: I built an SDK that scrambles HTML so scrapers get garbage
#25This is an interesting idea... it'd be a fun side project to implement enough of a CSS engine to undo this
Re: Show HN: I built an SDK that scrambles HTML so scrapers get garbage
#26Nice. I have been working on something which utilizes obfuscation, honeypots etc and I have come to a few realizations- - today you don't have to be a dedicated/motivated reverse engineer- you just need Sonnet 4.6 and let it do the work. - you need to throw constant/new gotchas to LLMs to keep them on their tows while they try to reverse engineer your website.
Re: Show HN: I built an SDK that scrambles HTML so scrapers get garbage
#27I hate everything about this, please use your time on this planet to make life better for people instead of worse. It is better for a million AI crawlers to get through than for even one search index crawler, that might expose the knowledge on your site to someone who needs it, to be denied.
Re: Show HN: I built an SDK that scrambles HTML so scrapers get garbage
#28[flagged]
On the broader point, I hear you, but I think there's a middle ground. Not all content is public knowledge. Some of it is premium, proprietary, or behind a paywall. The people publishing it should get to decide whether it becomes free training data.
Re: Show HN: I built an SDK that scrambles HTML so scrapers get garbage
#29> Your content, obscured. Is that supposed to be a good thing?
Re: Show HN: I built an SDK that scrambles HTML so scrapers get garbage
#30couldn't read the hero text on my phone it's white text and the shader background is also mostly white