Earlier quoted context omitted.
> or a search engine I would guess the crawling to be the easiest part.
Maybe "easiest" in the "I can explain it to my mother" way, but as with many good things, the devil's in the details about that. That goes double now that Cloudflare have basically taken over some non-trivial portion of the Internet and they really do hate crawlers I have a theory that the big search engines have agreements in place for "side-channel" access to CF and Akamai POPs but I would guess none of the parties…
> but I would guess none of the parties involved have any incentive to discuss such a thing in a place that I could read it
This reminds me how all the tech sites banned people like Alex Jones simultaneously. It's collusion.