Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cloudflare's new AI traffic options for customers

blog.cloudflare.com

Post reply on HN
Print view View original

101–110 of 169 posts

Re: Cloudflare's new AI traffic options for customers

#101

Post by m00dy »

bornfreddy wrote: post #84
m00dy wrote: post #59

Earlier quoted context omitted.

Not sure why Anubis is getting so much hype on HN, but honestly, it is not the solution. A real solution would use behavioral modeling. Most browser fingerprinting issues are already largely solved anyway.

As someone who is regularly blocked by CF and G - no, they are not. Even worse, they should not be, because that would mean the complete end of privacy online (not that we are far from that). Anubis works because at scale it wastes bots' resources (time mainly).

I think you’re confusing bots that crawl a website with bots used to launch DDoS attacks.

Re: Cloudflare's new AI traffic options for customers

#102

Post by AnthonyMouse »

dannyw wrote: post #43
simonw wrote: post #4

The big news here is that Googlebot will be blocked from September 15th onwards by one the "block training" policies, because Google use the same crawler infrastructure for their search index AND for training Gemini: > Another change that will apply on September 15 is that multi-purpose crawlers (specifically those that combine Search with Training) will be allowed/blocked according to all of their behaviors, in line…

Good. Google's approach here is manifestly predator, unfair, and IMO illegal. They deserve to be in court for this behaviour, and mandating owners give consent for AI training or drop out of Google; which is just a non-starter because they're a search monopoly. That's exactly what antitrust laws are supposed to do, and I hope at least EU regulators take action. Every single Googlebot crawl in your access logs is a tr…

The irony here is that the people blocking all other crawlers are the ones shoring up their monopoly. If you can't block Googlebot because you need the search traffic but you block everybody else so that nobody other than Google can index your site, how do you expect to ever get any search traffic that isn't from Google?

Re: Cloudflare's new AI traffic options for customers

#103

Post by motbus3 »

simonw wrote: post #4

The big news here is that Googlebot will be blocked from September 15th onwards by one the "block training" policies, because Google use the same crawler infrastructure for their search index AND for training Gemini: > Another change that will apply on September 15 is that multi-purpose crawlers (specifically those that combine Search with Training) will be allowed/blocked according to all of their behaviors, in line…

Don't that feel like a threat to businesses who dare to avoid their content being stolen?

Re: Cloudflare's new AI traffic options for customers

#104

Post by siva7 »

inigyou wrote: post #90
usef- wrote: post #32

Earlier quoted context omitted.

How are they playing both sides? I thought their scraping products were also about having it behave and not take down systems

Their scraping products are mostly about paying them money to not be blocked.

So Google has to pay Cloudflare 10B$ to get Googlebot moved to their default Allowlist otherwise Google would loose access to a significant portion of the internet? Genius move.

Re: Cloudflare's new AI traffic options for customers

#105

Post by siva7 »

holografix wrote: post #26

What’s the end goal for Cloudflare and the web here? I don’t think ADOG (anthropic, deepmind, openai, google) is going to pay to crawl. What would force their hand? It’s more likely they’ll strike undisclosed agreements with major sources of discussion like reddit etc. That’s not to say getting new information as a way of context-providing is not going to happen but that’s not scraping.

"I don’t think ADOG (anthropic, deepmind, openai, google) is going to pay to crawl."

You seem to misunderstand. You pay or you die. There's nothing in between. Cloudflare will happily collect the tax. As does Apple (collecting 20B$ yearly from Google for the "tax"). Cloudflare's users also won't mind about how the company handles ADOG as long as they get a chunk of the cake by getting freebies and cheap services.

Re: Cloudflare's new AI traffic options for customers

#107

Post by jwr »

I find it unsettling that we are willingly outsourcing the decision on who can access our sites to an increasingly dominant corporate entity.

The reasoning behind this is also flawed: blocking "bots" and "AI" means that our AI agents working for us are unable to do their work for us, because of knee-jerk bot-blocks.

Re: Cloudflare's new AI traffic options for customers

#109

Post by timpera »

fc417fc802 wrote: post #19

Please consider installing one of the many PoW schemes such as anubis rather than use these cloudflare "features". I increasingly encounter outright blocks rather than any sort of captcha when visiting cloudflare "protected" sites. Each individual site isn't particularly important to me but it's depressing to watch the process unfold like this. You really are choosing to erode the core basis of the internet if you go…

Please don't use Anubis, it makes visiting websites very difficult (often multiple minutes wait times) on old hardware and low-end smartphones.
Post reply on HN