Live data from Hacker News

Show HN: I built an SDK that scrambles HTML so scrapers get garbage

obscrd.dev

21–30 of 46 posts

Re: Show HN: I built an SDK that scrambles HTML so scrapers get garbage

#21
I hate everything about this, please use your time on this planet to make life better for people instead of worse.

It is better for a million AI crawlers to get through than for even one search index crawler, that might expose the knowledge on your site to someone who needs it, to be denied.

Re: Show HN: I built an SDK that scrambles HTML so scrapers get garbage

#22

All I want is an API for my AI, you can ask me for my public key, if you want my human identity verified. The collateral damage of this bot hunting is the emergence of personal AIs. Do we really want that? It feels regressive. (I see the hypocrisy here, we are fighting the scrapers that feed the LLMs that runs our personal agents)

You are not wrong. But the use case I keep seeing is companies with proprietary content they spent real money creating, who don't want it showing up in someone else's training data for free. It's less about bot hunting and more about content owners having a choice.

Re: Show HN: I built an SDK that scrambles HTML so scrapers get garbage

#25
post #12

This is an interesting idea... it'd be a fun side project to implement enough of a CSS engine to undo this

You are more than welcome to do so. Please keep in mind the realistic goal is raising the cost of scraping. Most bots use simple HTTP requests, and we make that useless.

Re: Show HN: I built an SDK that scrambles HTML so scrapers get garbage

#26
post #6

Nice. I have been working on something which utilizes obfuscation, honeypots etc and I have come to a few realizations- - today you don't have to be a dedicated/motivated reverse engineer- you just need Sonnet 4.6 and let it do the work. - you need to throw constant/new gotchas to LLMs to keep them on their tows while they try to reverse engineer your website.

The bar for reverse engineering dropped to "paste the HTML into Claude and ask it to decode." That's partly why the v2 roadmap moves toward techniques where the readable text never exists in the DOM at all. Static obfuscation patterns need to keep evolving or they become a one-prompt solve.

Re: Show HN: I built an SDK that scrambles HTML so scrapers get garbage

#27

I hate everything about this, please use your time on this planet to make life better for people instead of worse. It is better for a million AI crawlers to get through than for even one search index crawler, that might expose the knowledge on your site to someone who needs it, to be denied.

For public knowledge sites this would be the wrong tool entirely. The use case is more like paywalled articles, proprietary product data, or premium content that companies paid to create and don't want scraped into a competitor's training set. obscrd is opt-in per component, not a whole-site lockdown.

Re: Show HN: I built an SDK that scrambles HTML so scrapers get garbage

#28

[flagged]

The TalkBack issue is useful feedback, thank you. I tested with NVDA and VoiceOver but not TalkBack on Android. If light mode is reading individual words instead of a continuous block that's a real bug I want to fix.

On the broader point, I hear you, but I think there's a middle ground. Not all content is public knowledge. Some of it is premium, proprietary, or behind a paywall. The people publishing it should get to decide whether it becomes free training data.

Post reply on HN