Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cloudflare to introduce pay-per-crawl for AI bots

blog.cloudflare.com

Post reply on HN
Print view View original

21–30 of 317 posts

Re: Cloudflare to introduce pay-per-crawl for AI bots

#21

Post by aspenmayer »

some_furry wrote: post #17
aspenmayer wrote: post #15

Earlier quoted context omitted.

I mean, I wouldn't care who referred the paying user. I would optimize my blog to serve free and paying users to the best of my ability. I would hope that I could do that but I don't know what you are trying to do with your blog. Most blogs could probably be self-hosted with some caching and static layout where possible to perhaps avoid needing to use Cloudflare. I guess you already have to be using CF to have access…

I pay $300/year for the privilege of being able to write what I want without the pressure to monetize or surveil my readers. Some of my blog posts are linked by programming language docs for cryptography. Others have helped queer folks transition to a higher-paying tech career. It's difficult to quantify either of those things in a dollar value. I've opted to not ever do so. But if I can make the AI slop machines pay…

Since I've got you here, are you going to DEF CON?

Re: Cloudflare to introduce pay-per-crawl for AI bots

#22

Post by some_furry »

aspenmayer wrote: post #21
some_furry wrote: post #17

Earlier quoted context omitted.

I pay $300/year for the privilege of being able to write what I want without the pressure to monetize or surveil my readers. Some of my blog posts are linked by programming language docs for cryptography. Others have helped queer folks transition to a higher-paying tech career. It's difficult to quantify either of those things in a dollar value. I've opted to not ever do so. But if I can make the AI slop machines pay…

Since I've got you here, are you going to DEF CON?

Yup. I'm planning to debut my fursuit there too.

Re: Cloudflare to introduce pay-per-crawl for AI bots

#24

Post by odyssey7 »

crgwbr wrote: post #10

All this is going to do is drive AI companies to mask their user agent to appear as a standard browser, resulting in a worse end state than we’re in now. It’s an exercise in futility.

In theory, this presents a form of competition that should drive the tolls down to an equilibrium level. Though, theory doesn’t always play out perfectly in practice.

Re: Cloudflare to introduce pay-per-crawl for AI bots

#25

Post by raesene9 »

crgwbr wrote: post #10

All this is going to do is drive AI companies to mask their user agent to appear as a standard browser, resulting in a worse end state than we’re in now. It’s an exercise in futility.

Potentially for smaller players but I'd guess that the larger players (OpenAI, Anthropic, etc) won't go down that line as it'd be pretty easy to spot at the volume they're crawling and a bad look for them when they inevitably get discovered.

Also, Cloudflare is in the position of being able to see a lot of traffic making it easier for them to spot that kind of masking activity.

Re: Cloudflare to introduce pay-per-crawl for AI bots

#26

Post by kassner »

crgwbr wrote: post #10

All this is going to do is drive AI companies to mask their user agent to appear as a standard browser, resulting in a worse end state than we’re in now. It’s an exercise in futility.

Weren’t they already doing that for years (plus using residential proxies)?

Re: Cloudflare to introduce pay-per-crawl for AI bots

#27

Post by GaggiX »

some_furry wrote: post #6

I would totally do this. "Read my blog for free, or pay $25/page for your AI to read it for you." This is praxis. Enshittify the enshittification machine. We should also throw ads in there, via a deliberate prompt injection that the AI companies expose through an API. I totally won't misuse it ;)

>pay $25/page for your AI

A residential proxy is way cheaper for scraping.

Re: Cloudflare to introduce pay-per-crawl for AI bots

#29

Post by some_furry »

GaggiX wrote: post #27
some_furry wrote: post #6

I would totally do this. "Read my blog for free, or pay $25/page for your AI to read it for you." This is praxis. Enshittify the enshittification machine. We should also throw ads in there, via a deliberate prompt injection that the AI companies expose through an API. I totally won't misuse it ;)

>pay $25/page for your AI A residential proxy is way cheaper for scraping.

Sure, and suing the AI companies that use residential proxies to get out of paying is expensive, but so are my other hobbies. Hilarity could ensue.
Post reply on HN