Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The web does not need gatekeepers: Cloudflare’s new “signed agents” pitch

positiveblue.substack.com

Post reply on HN
Print view View original

41–50 of 520 posts

Re: The web does not need gatekeepers: Cloudflare’s new “signed agents” pitch

#41

Post by jsheard »

rikafurude21 wrote: post #37
matt-p wrote: post #5

I have zero issue with Ai Agents, if there's a real user behind there somewhere. I DO have a major issue with my sites being crawled extremely aggressively by offenders including Meta, Perplexity and OpenAI - it's really annoying realising that we're tying up several cpu cores on AI crawling. Less than on real users and google et al.

Cloudflare is trying to gatekeep which user-initated agents are allowed to read website content, which is of course very different from scraping website for training data. Meta, Perplexity and OpenAI all have some kind of web-search functionality where they sent requests based on user prompts. These are not requests that get saved to train the next LLM. Cloudflare intentionally blurs the line between both types of bo…

> Cloudflare is trying to gatekeep which user-initated agents are allowed to read website content, which is of course very different from scraping website for training data.

That distinction requires you to take companies which benefit from amassing as much training data as possible at their word when they pinky swear that a particular request is totally not for training, promise.

Re: The web does not need gatekeepers: Cloudflare’s new “signed agents” pitch

#42

Post by positiveblue »

whimsicalism wrote: post #19

[flagged]

Narrator: but he did put effort...

Anyway, main take aways for you:

- We REALLY need a way to tie identity to agent/requests - The idea of registering with cloudflare to be able to access a website is bad - Sites should be able to block whoever they want or make anything they desire a requirement (there are people that has login only with google after all)

We have the right primitives to build something that works for any provider (from cloudflare, to Akamai, to self hosting nginx servers). Let's take that route

Re: The web does not need gatekeepers: Cloudflare’s new “signed agents” pitch

#43

Post by seanvelasco »

as a Cloudflare customer, I am happy with their proposition. I personally do not want companies like Perplexity that fake their user-agent and ignore my robots.txt to trespass.

and isn't this why people sign up with Cloudflare in the first place? for bot protection? to me, this is just the same, but with agents.

i love the idea of an open internet, but this requires all party to be honest. a company like Perplexity that fakes their user-agent to get around blocks disrespects that idea.

my attitude towards agents is positive. if a user used an LLM to access my websites and web apps, i'm all for it. but the LLM providers must disclose who they are - that they are OpenAI, Google, Meta, or the snake oil company Perplexity

Re: The web does not need gatekeepers: Cloudflare’s new “signed agents” pitch

#44

Post by asats »

matt-p wrote: post #5

I have zero issue with Ai Agents, if there's a real user behind there somewhere. I DO have a major issue with my sites being crawled extremely aggressively by offenders including Meta, Perplexity and OpenAI - it's really annoying realising that we're tying up several cpu cores on AI crawling. Less than on real users and google et al.

I've some personal apps online and I had to turn the cloudflare ai bot protection on because one of them got 1.6TB of data accessed by the bots in the last month, 1.3 million requests per day, just non stop hammering it with no limits.

Re: The web does not need gatekeepers: Cloudflare’s new “signed agents” pitch

#45

Post by nromiun »

didibus wrote: post #23
nromiun wrote: post #17

Earlier quoted context omitted.

Bankruptcy as a surprise gift is not an alternative. Even those that use big cloud providers like AWS and GCP use CDNs like Cloudflare to protect themselves. And there is no free CDN like Cloudflare.

> And there is no free CDN like Cloudflare. Their pricing page says: No-nonsense Free Tier As part of the AWS free Usage Tier you can get started with Amazon CloudFront for free. Included in Always Free Tier 1 TB of data transfer out to the internet per month 10,000,000 HTTP or HTTPS Requests per month 2,000,000 CloudFront Function invocations per month 2,000,000 CloudFront KeyValueStore reads per month 10 Distributi…

1 TB per month of data is literally nothing. A kid could rent a VPS for an hour and drain all that. What do you do after that? AWS is not going to stop your bill going up is it?

I don't care about any of those fancy serverless services. I am just talking about the cheapest CDN.

Re: The web does not need gatekeepers: Cloudflare’s new “signed agents” pitch

#46

Post by doublerabbit »

giantrobot wrote: post #30
fooqux wrote: post #11

Earlier quoted context omitted.

Ok, I'll bite. Why is turning the Internet into a walled garden necessary now?

Multi-Tbps DDoS attacks, pervasive scanning of sites for exploits, comically expensive egress bandwidth on services like AWS, and ISPs disallowing hosting services on residential accounts.

Start forcing tighter security on the devices causing the Multi-Tbps DDoS attacks would be a better option, no? Cheap unsecured IoT devices are a problem.

It's not just computers anymore. Web enabled CCTV, doorbell cameras are all culprits.

Re: The web does not need gatekeepers: Cloudflare’s new “signed agents” pitch

#47

Post by [deleted] »

Gud wrote: post #27
TIPSIO wrote: post #20

Everyone loves the dream of a free for all and open web. But the reality is how can someone small protect their blog or content from AI training bots? E.g.: They just blindly trust someone is sending Agent vs Training bots and super duper respecting robots.txt? Get real... Or, fine what if they do respect robots.txt, but they buy the data that may or may not have been shielded through liability layers via "licensed d…

By developing Free Software combating these hostile softwares. Corporations develop hostile AI agents, Capable hackers develop anti-AI-agents. This defeatist atittude "we have no power".

[deleted]

Re: The web does not need gatekeepers: Cloudflare’s new “signed agents” pitch

#48

Post by supriyo-biswas »

TIPSIO wrote: post #32
Gud wrote: post #27

Earlier quoted context omitted.

By developing Free Software combating these hostile softwares. Corporations develop hostile AI agents, Capable hackers develop anti-AI-agents. This defeatist atittude "we have no power".

Yes, I obviously agree with you. My comment's point is missed a little I think by you. CF is making these tools and giving access to it to millions of people.

Well there's open source stuff like https://github.com/TecharoHQ/anubis; one doesn't need a top-down mandated solution coming from a corporation.

In general Cloudflare has been pushing DRMization of the web for quite some time, and while I understand why they want to do it, I wish they didn't always show off as taking the moral high ground.

Re: The web does not need gatekeepers: Cloudflare’s new “signed agents” pitch

#49

Post by inetknght »

clvx wrote: post #24
TIPSIO wrote: post #20

Everyone loves the dream of a free for all and open web. But the reality is how can someone small protect their blog or content from AI training bots? E.g.: They just blindly trust someone is sending Agent vs Training bots and super duper respecting robots.txt? Get real... Or, fine what if they do respect robots.txt, but they buy the data that may or may not have been shielded through liability layers via "licensed d…

You can lock it up with a user account and payment system. The fact the site is up on the internet doesn’t mean you can or cannot profit from it. It’s up to you. What I would like it’s a way to notify my isp and say, block this traffic to my site.

> What I would like it’s a way to notify my isp and say, block this traffic to my site.

I would love that, and make it automated.

A single message from your IP to your router: block this traffic. That router sends it upstream, and it also blocks it. Repeat ad nauseum until source changes ASN or (if the originator is on the same ASN) reaches the router from the originator, routing table space notwithstanding. Maybe it expires after some auto-expiry -- a day or month or however long your IP lease exists. Plus, of course, a way to query what blocks I've requested and a way to unblock.

Re: The web does not need gatekeepers: Cloudflare’s new “signed agents” pitch

#50

Post by Analemma_ »

Gud wrote: post #27
TIPSIO wrote: post #20

Everyone loves the dream of a free for all and open web. But the reality is how can someone small protect their blog or content from AI training bots? E.g.: They just blindly trust someone is sending Agent vs Training bots and super duper respecting robots.txt? Get real... Or, fine what if they do respect robots.txt, but they buy the data that may or may not have been shielded through liability layers via "licensed d…

By developing Free Software combating these hostile softwares. Corporations develop hostile AI agents, Capable hackers develop anti-AI-agents. This defeatist atittude "we have no power".

How does an agent help my website not get crushed by traffic load, and how is this proposal any different from the gatekeeping problem to the open web, except even less transparent and accountable because now access is gated by logic inside an impenetrable web of NN weights?

This seems like slogan-based planning with no actual thought put into it.

Post reply on HN