Earlier quoted context omitted.
Cloudflare has not used CAPTCHAs since 2023: https://blog.cloudflare.com/turnstile-ga/
To save a visit: they use turnstile, a captcha replacement. The checkbox with verify you are a human. I would call that a captcha, but it is debatable if a non-puzzle check is.
Cloudflare to introduce pay-per-crawl for AI bots
41–50 of 317 posts
Re: Cloudflare to introduce pay-per-crawl for AI bots
#421. Encourage fencing off everything by default to maximize need for bypass
2. Offer bypass through payment
3. Profit!
You wouldn't believe the number of public administrations with public information that have (mostly unwittingly) had some lazy contractor put Cloudflare in front of their entire site, blocking even their RSS feeds from M2M. Yes, you can send them mails and call and sometimes, if they even understand the problem, they will fix it after a few months just before the next cheapest contractor is hired and we start all over again.
Not saying Cloudflare is just an extortion racket, but it's getting closer by the day.
Re: Cloudflare to introduce pay-per-crawl for AI bots
#43Nice to see someone addressing this annoying problem, I'm seeing first hand bot traffic go up as they are just gobbling up data. However instead of relying on Cloudflare, it would be better to have a open source protocol that handles permission and payment for crawlers/scraper.
Re: Cloudflare to introduce pay-per-crawl for AI bots
#44Re: Cloudflare to introduce pay-per-crawl for AI bots
#45"AI" is certainly creating problems that can be monetized. So humans get the Cloudflare captchas in order to access their own content at Stackoverflow and Mathoverflow, and "AI" crawlers get the data highway for a fee. And all of this does not stop the incumbents who have already stolen everything.
Re: Cloudflare to introduce pay-per-crawl for AI bots
#46This is basically just how we want to do micro payments. I think coinbase recently introduced a library for the same using cryptocurrency and the 402 status code. In fact yea it's called x402. https://github.com/coinbase/x402
Brave's BAT is also a good attempt at fixing this, but x402 seems like a more generic solution. It's a shame that neither has any chance of gaining traction, partly because of the cryptocurrency stigma, and partly because of adtech's tight grip on the current web.
Re: Cloudflare to introduce pay-per-crawl for AI bots
#47So: 1. Encourage fencing off everything by default to maximize need for bypass 2. Offer bypass through payment 3. Profit! You wouldn't believe the number of public administrations with public information that have (mostly unwittingly) had some lazy contractor put Cloudflare in front of their entire site, blocking even their RSS feeds from M2M. Yes, you can send them mails and call and sometimes, if they even understa…
Re: Cloudflare to introduce pay-per-crawl for AI bots
#48I would totally do this. "Read my blog for free, or pay $25/page for your AI to read it for you." This is praxis. Enshittify the enshittification machine. We should also throw ads in there, via a deliberate prompt injection that the AI companies expose through an API. I totally won't misuse it ;)
I mean, I wouldn't care who referred the paying user. I would optimize my blog to serve free and paying users to the best of my ability. I would hope that I could do that but I don't know what you are trying to do with your blog. Most blogs could probably be self-hosted with some caching and static layout where possible to perhaps avoid needing to use Cloudflare. I guess you already have to be using CF to have access…
I think the more pertinent question is: Can an AI company determine the value of that content automatically without seeing it? Because if they can’t, why would they pay for it?
Re: Cloudflare to introduce pay-per-crawl for AI bots
#49Someone should use this to create a new browser. A human user drops $100 into the browser, and each website offers a per-page-view rate, gradually deducted from the $100. In exchange the user doesn't have to suffer through advertisements.
Re: Cloudflare to introduce pay-per-crawl for AI bots
#50Someone should use this to create a new browser. A human user drops $100 into the browser, and each website offers a per-page-view rate, gradually deducted from the $100. In exchange the user doesn't have to suffer through advertisements.
And we are back to: how much would you pay for a website before you access said website for the first time?