I want to make my own search engine, one day, with my own crawler. There is an SEO-proof way to determine what the ranking of a site should be - penalise it for each advertisement, penalise further for delivering different content to the crawler[1], allow logged-in users to down-rank a site, etc. Basically, a site starts off with a perfect score, then gets penalised for each violation, for each dark-pattern, for each…
But that web doesn’t exist anymore. The dominance of Google means all the creators and authors went elsewhere. Filter out the crap and you’ll realise that there’s very little substance left.
And even if it doesn't (which I think it does), disincentiving bad behavior has to start somewhere