Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The web does not need gatekeepers: Cloudflare’s new “signed agents” pitch

positiveblue.substack.com

Post reply on HN
Print view View original

31–40 of 520 posts

Re: The web does not need gatekeepers: Cloudflare’s new “signed agents” pitch

#31

Post by timshell »

I think about this as a startup founder building a 'proof-of-human' layer on the Internet.

One of the hard parts in this space is what level of transparency should you have. We're advancing the thesis that behavioral biometrics offers robust continuous authentication that helps with bot/human and good/bad, but people are obviously skeptical to trust black-box models for accuracy and/or privacy reasons.

We've defaulted to a lot of transparency in terms of publishing research online (and hopefully in scientific journals), but we've seen the downside: competitors fake claims about their own best in-house behavioral tools that is behind their company walls in addition to investors constantly worried about an arms race.

As someone genuinely interested (and incentivized!) to build a great solution in this space, what are good protocols/examples to follow?

Re: The web does not need gatekeepers: Cloudflare’s new “signed agents” pitch

#32

Post by TIPSIO »

Gud wrote: post #27
TIPSIO wrote: post #20

Everyone loves the dream of a free for all and open web. But the reality is how can someone small protect their blog or content from AI training bots? E.g.: They just blindly trust someone is sending Agent vs Training bots and super duper respecting robots.txt? Get real... Or, fine what if they do respect robots.txt, but they buy the data that may or may not have been shielded through liability layers via "licensed d…

By developing Free Software combating these hostile softwares. Corporations develop hostile AI agents, Capable hackers develop anti-AI-agents. This defeatist atittude "we have no power".

Yes, I obviously agree with you. My comment's point is missed a little I think by you. CF is making these tools and giving access to it to millions of people.

Re: The web does not need gatekeepers: Cloudflare’s new “signed agents” pitch

#33

Post by jurking_hoff »

fooqux wrote: post #11
cyberlurker wrote: post #2

I would love that vision to become reality but what Cloudflare is doing is unfortunately necessary atm.

Ok, I'll bite. Why is turning the Internet into a walled garden necessary now?

[dead]

Re: The web does not need gatekeepers: Cloudflare’s new “signed agents” pitch

#34

Post by banku_brougham »

Gud wrote: post #27
TIPSIO wrote: post #20

Everyone loves the dream of a free for all and open web. But the reality is how can someone small protect their blog or content from AI training bots? E.g.: They just blindly trust someone is sending Agent vs Training bots and super duper respecting robots.txt? Get real... Or, fine what if they do respect robots.txt, but they buy the data that may or may not have been shielded through liability layers via "licensed d…

By developing Free Software combating these hostile softwares. Corporations develop hostile AI agents, Capable hackers develop anti-AI-agents. This defeatist atittude "we have no power".

This is the attitude I like to see. As they say, actually I hate this because of past connotations but "freedom isn't free"

Re: The web does not need gatekeepers: Cloudflare’s new “signed agents” pitch

#35

Post by palmfacehn »

matt-p wrote: post #28
palmfacehn wrote: post #22

Earlier quoted context omitted.

You'd think they would have an interest in developing reasonable crawling infrastructure, like Google, Bing or Yandex. Instead they go all in on hosts with no metering. All of the search majors reduce their crawl rate as request times increase. On one hand these companies announce themselves as sophisticated, futuristic and highly-valued, on the other hand we see rampant incompetence, to the point that webmasters eve…

Honestly it's just tragedy of the commons. Why put the effort in when you don't have to identify yourself, just crawl and if you get blocked move the job to another server.

At this point I'm blocking several ASNs. Most are cloud provider related, but there are also some repurposed consumer ASNs coming out of the PRC. Long term, this devalues the offerings of those cloud providers, as prospective customers will not be able to use them for crawling.

Re: The web does not need gatekeepers: Cloudflare’s new “signed agents” pitch

#37

Post by rikafurude21 »

matt-p wrote: post #5

I have zero issue with Ai Agents, if there's a real user behind there somewhere. I DO have a major issue with my sites being crawled extremely aggressively by offenders including Meta, Perplexity and OpenAI - it's really annoying realising that we're tying up several cpu cores on AI crawling. Less than on real users and google et al.

Cloudflare is trying to gatekeep which user-initated agents are allowed to read website content, which is of course very different from scraping website for training data. Meta, Perplexity and OpenAI all have some kind of web-search functionality where they sent requests based on user prompts. These are not requests that get saved to train the next LLM. Cloudflare intentionally blurs the line between both types of bots, and in that sense it is a bait-and-switch where they claim to 'protect content creators' by being the man in the middle and collecting tolls from LLM providers to pay creators (and of course take a cut for themselves). Its not something they do because it would be fair, theres financial motivation.

Re: The web does not need gatekeepers: Cloudflare’s new “signed agents” pitch

#38

Post by ryoshu »

didibus wrote: post #25
miohtama wrote: post #9

Earlier quoted context omitted.

AWS needs a dedicated AWS engineer while any technical person and some non-technical people have skill to set up Cloudflare. Esp. Without surprise bills.

I always hear this, but honestly I'm not sure it's true. It's hard to assess the validity of this versus Cloudflare having a really good marketing department. I've used neither, so I can't say, but I've also never seen anyone truly explain why/why-not.

Why not use both and find out? Cloudflare is much less technical than AWS, but still a bit technical.
Post reply on HN