Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Web Bot Auth

developers.cloudflare.com

Post reply on HN
Print view View original

61–70 of 77 posts

Re: Web Bot Auth

#61

Post by nembal »

Web Bot Auth solves a real problem with a real standard. Per-request signatures make automated traffic accountable, and that is the right long-term primitive. In Cloudflare’s hands, the current implementation is built first for bots, not agents. “Signed agents” read like a label added to a bot-centric system, not a first-class agent identity fabric. The design also centers Cloudflare as the arbiter and on-ramp, which is great for reliability inside their network and great for their business moat, but not great for an open, decentralized agentic web.

While it builds on standards as the top poster notes, cloudflare's version is a business moat driven central registry service and nothing what the decentralized internet would/should look.

i wrote a but more about this on my blog if someone care to read https://blog.agentcommunity.org/2025-08-23-web_auth_box_not_...

Re: Web Bot Auth

#62

Post by jrochkind1 »

binarymax wrote: post #33
jrochkind1 wrote: post #32

Earlier quoted context omitted.

What's now at the top has links to IETF drafts in the first paragraph. What am I missing? A way to authenticate identity for crawlers so I can allow-list ones I want to get in, exempt them from turnstile/captcha, etc -- is something I need. I'm not following what makes this controversial. Cryptographic verification of identity for web requests, sounds right.

I think about failure modes. What happens if cloudflare decides you are a bot and you’re not. What recourse do you have? What are the formal mechanisms to ensure a person is not blocked from the majority of the web because cloudflare is a middleman and you are a false positive?

I am not following what any of that has to do with the Web Bot Auth protocol?

it seems like complaints about Cloudflare's anti-DOS protection services and how they have a monopoly on such, I get that.

I'm not seeing the connection to a protocol for bots/crawlers voluntarily cryptographically signing their http requests, so sites (anyone implementing the protocol not just cloudflare) can use it to authenticate known actors?

I am interested in using it to exempt bots/crawlers I trust/support/have an agreement with from the anti-bot measures I, like many, am being forced to implement to keep our sites up under an enormously increased wave of what is apparently AI-training-motivated repeat crawling. Right now these measures are keeping out bots I don't want to keep out too. I would like to be able to securely identify them to let them in.

Re: Web Bot Auth

#63

Post by estearum »

account42 wrote: post #59
bobbiechen wrote: post #4

I disagree with the other top-level comments at the moment: I believe Web Bot Auth is a useful and non-centralized emerging standard for self-identifying bots and agents. This press release today is a better statement of _why_ this feature exists (as opposed to the submission link, which is nuts-and-bolts of implementing): https://blog.cloudflare.com/signed-agents/ Web Bot Auth is a way for bots to self-identify cryp…

> This is a good thing! We want bots to be able to self-identify in a way that can't be impersonated. Who is we? I absolutely don't want that.

Earnest question: why not? I would think "option to prove who you are and guarantee not to be impersonated" is a pretty broadly appealing capability except to people trying to do the impersonating.

Re: Web Bot Auth

#64

Post by jimmydoe »

ActivityPub had sth similar? Maybe just reuse that to identify the source identity, then determine if you want to trust that domain or person/bot?

Re: Web Bot Auth

#65

Post by immibis »

acdha wrote: post #40
immibis wrote: post #39

Earlier quoted context omitted.

They don't have to, but they're tricked into doing so. Via marketing.

I miss the 90s, too, but these days anyone who wants to deal with current levels of bot traffic is probably going to look at a service like Cloudflare as much cheaper than the amount of ops time they’d otherwise spend keeping things up and secure.

You could just, like, not make a website that takes several seconds to handle each request.

I let bots hit Gitea 2-3 times per second on a $10/month VPS, and the only actual problem was that it doesn't seem to ever delete zip snapshots, filling up the disk when enough snapshot links are clicked. So I disabled that feature by setting the snapshots folder read-only. There were no other problems. I mention Gitea because people complain about having to protect Gitea a lot, for some reason.

Re: Web Bot Auth

#66

Post by skeezyboy »

estearum wrote: post #63
account42 wrote: post #59

Earlier quoted context omitted.

> This is a good thing! We want bots to be able to self-identify in a way that can't be impersonated. Who is we? I absolutely don't want that.

Earnest question: why not? I would think "option to prove who you are and guarantee not to be impersonated" is a pretty broadly appealing capability except to people trying to do the impersonating.

>"option to prove who you are and guarantee not to be impersonated"

guaranteed as long as no attacker gets hold of the private key, which cannot be guaranteed

Re: Web Bot Auth

#67

Post by estearum »

skeezyboy wrote: post #66
estearum wrote: post #63

Earlier quoted context omitted.

Earnest question: why not? I would think "option to prove who you are and guarantee not to be impersonated" is a pretty broadly appealing capability except to people trying to do the impersonating.

>"option to prove who you are and guarantee not to be impersonated" guaranteed as long as no attacker gets hold of the private key, which cannot be guaranteed

Yeah, I don't find this to be a compelling argument at all.

That's an argument against all authentication anywhere.

Re: Web Bot Auth

#68

Post by skeezyboy »

estearum wrote: post #67
skeezyboy wrote: post #66

Earlier quoted context omitted.

>"option to prove who you are and guarantee not to be impersonated" guaranteed as long as no attacker gets hold of the private key, which cannot be guaranteed

Yeah, I don't find this to be a compelling argument at all. That's an argument against all authentication anywhere.

> That's an argument against all authentication anywhere.

its a problem isnt it

Re: Web Bot Auth

#69

Post by palmfacehn »

>3. Register your bot and key directory

Register with CF is the specific part I object to. Of all of the numerous hazards here centralizing the registration with CF is most clearly problematic. This part of the spec could have easily been an additional header linking to key data.

Re: Web Bot Auth

#70

Post by acdha »

immibis wrote: post #65
acdha wrote: post #40

Earlier quoted context omitted.

I miss the 90s, too, but these days anyone who wants to deal with current levels of bot traffic is probably going to look at a service like Cloudflare as much cheaper than the amount of ops time they’d otherwise spend keeping things up and secure.

You could just, like, not make a website that takes several seconds to handle each request. I let bots hit Gitea 2-3 times per second on a $10/month VPS, and the only actual problem was that it doesn't seem to ever delete zip snapshots, filling up the disk when enough snapshot links are clicked. So I disabled that feature by setting the snapshots folder read-only. There were no other problems. I mention Gitea because…

Sure, I’ve been doing that since the 90s. I still pay for hardware and egress, and it turns out that everything has limits for the amount of traffic it can handle which bots can easily saturate. I’ve had sites which were mostly Varnish serving cached content at wire speed go down because they saturated the upstream.
Post reply on HN