Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Web Bot Auth

developers.cloudflare.com

Post reply on HN
Print view View original

1–10 of 77 posts

Re: Web Bot Auth

#2

Post by mips_avatar »

Cloudflare's verified bots program is a terrible idea. They want to be the central chokepoint for agents, and they're doing it in shady ways like auto enrolling customers into blocking agents.

Re: Web Bot Auth

#4

Post by bobbiechen »

I disagree with the other top-level comments at the moment: I believe Web Bot Auth is a useful and non-centralized emerging standard for self-identifying bots and agents.

This press release today is a better statement of _why_ this feature exists (as opposed to the submission link, which is nuts-and-bolts of implementing): https://blog.cloudflare.com/signed-agents/

Web Bot Auth is a way for bots to self-identify cryptographically. Unlike the user agent header (which is trivially spoofed) or known IPs (painful to manage), Web Bot Auth uses HTTP Message Signatures using the bot's key, which should be published at some well-known location.

This is a good thing! We want bots to be able to self-identify in a way that can't be impersonated. This gives website operators the power to allow or deny well-behaved bots with precision. It doesn't change anything about bots who try to hide their identity, who are not going to self-identify anyways.

It's worth reading the proposal on the details: https://datatracker.ietf.org/doc/html/draft-meunier-web-bot-... . Nothing about this is limited to Cloudflare.

I'm also working on support for Web Bot Auth for our Agent Identification project at Stytch https://www.isagent.dev . Well-behaved bots benefit from this self-identification because it enables a better Agent Experience: https://stytch.com/blog/introducing-is-agent/

Re: Web Bot Auth

#5

Post by binarymax »

bobbiechen wrote: post #4

I disagree with the other top-level comments at the moment: I believe Web Bot Auth is a useful and non-centralized emerging standard for self-identifying bots and agents. This press release today is a better statement of _why_ this feature exists (as opposed to the submission link, which is nuts-and-bolts of implementing): https://blog.cloudflare.com/signed-agents/ Web Bot Auth is a way for bots to self-identify cryp…

I agree in principle, but I disagree that it should be designed and mandated by a private gatekeeper

Re: Web Bot Auth

#6

Post by kylehotchkiss »

zb3 wrote: post #3

Seems like Cloudflare wants to regulate the internet.. they should not have that power.

Disagree. Not everybody wants their sites scraped and their content used to train a model that they'll never see a penny from. Cloudflare is the only party who wants to build a system where both the models and individual sites have their interests respected.

Re: Web Bot Auth

#7

Post by ATechGuy »

kylehotchkiss wrote: post #6
zb3 wrote: post #3

Seems like Cloudflare wants to regulate the internet.. they should not have that power.

Disagree. Not everybody wants their sites scraped and their content used to train a model that they'll never see a penny from. Cloudflare is the only party who wants to build a system where both the models and individual sites have their interests respected.

Are you sure that CF can stop AI bots?

Re: Web Bot Auth

#8

Post by realityfactchex »

No offense, but screw CloudFlare, screw their captchas for humans, and screw their wedging themselves between web operators and web users.

They can offer what they want for bots. But stop ruining the experience for humans first.

Re: Web Bot Auth

#9

Post by hsbauauvhabzb »

ATechGuy wrote: post #7
kylehotchkiss wrote: post #6

Earlier quoted context omitted.

Disagree. Not everybody wants their sites scraped and their content used to train a model that they'll never see a penny from. Cloudflare is the only party who wants to build a system where both the models and individual sites have their interests respected.

Are you sure that CF can stop AI bots?

Do you have a better alternative?

Re: Web Bot Auth

#10

Post by nerdsniper »

Why use a "web bot" instead of an API? Either can be driven by an AI "agent"...but this just seems like an "API key for a visual api interface", and rather wasteful in cost and resources. If a company could afford to pay a partner for an API key they wouldn't need this. If they can't afford to pay the partner for access -- they'd still be blocked with or without "Web Bot Auth". I don't understand what this is for.

I suspect I'm missing something, what am I missing?

Post reply on HN