Live data from Hacker News

Web Bot Auth

developers.cloudflare.com

31–40 of 77 posts

Re: Web Bot Auth

#31

Earlier quoted context omitted.

Have you looked into open-source alternatives? I'm assuming that it's a pressing problem for you, and you have already explored alternatives.

I have, sadly they are basically worthless and often worse then worthless as they negatively impact the site.

Interesting. Care to list them here so that we all can learn.

Re: Web Bot Auth

#32

I disagree with the other top-level comments at the moment: I believe Web Bot Auth is a useful and non-centralized emerging standard for self-identifying bots and agents. This press release today is a better statement of _why_ this feature exists (as opposed to the submission link, which is nuts-and-bolts of implementing): https://blog.cloudflare.com/signed-agents/ Web Bot Auth is a way for bots to self-identify cryp…

I agree in principle, but I disagree that it should be designed and mandated by a private gatekeeper

What's now at the top has links to IETF drafts in the first paragraph. What am I missing?

A way to authenticate identity for crawlers so I can allow-list ones I want to get in, exempt them from turnstile/captcha, etc -- is something I need.

I'm not following what makes this controversial. Cryptographic verification of identity for web requests, sounds right.

Re: Web Bot Auth

#33

Earlier quoted context omitted.

I agree in principle, but I disagree that it should be designed and mandated by a private gatekeeper

What's now at the top has links to IETF drafts in the first paragraph. What am I missing? A way to authenticate identity for crawlers so I can allow-list ones I want to get in, exempt them from turnstile/captcha, etc -- is something I need. I'm not following what makes this controversial. Cryptographic verification of identity for web requests, sounds right.

I think about failure modes. What happens if cloudflare decides you are a bot and you’re not. What recourse do you have? What are the formal mechanisms to ensure a person is not blocked from the majority of the web because cloudflare is a middleman and you are a false positive?

Re: Web Bot Auth

#34

No offense, but screw CloudFlare, screw their captchas for humans, and screw their wedging themselves between web operators and web users. They can offer what they want for bots. But stop ruining the experience for humans first.

> screw their wedging themselves between web operators and web users Web operators choose to use them; hell they even pay Cloudflare to be between them. Seriously I just think you don't understand how bad it is to run a site without someone in-front of it.

Couldn’t agree more — Much like running my own DNS or email server, I don’t think I’ll ever go back to running my own website directly on the internet. It’s just not worth the hassle. For stuff only I use, it sits behind my VPN. For anything that _must_ be public, it’s going behind a WAF someone else can run.

Re: Web Bot Auth

#36

Earlier quoted context omitted.

Do you have a better alternative?

https://anubis.techaro.lol/ ?

Your browser is configured to disable cookies. Anubis requires cookies for the legitimate interest of making sure you are a valid client. Please enable cookies for this domain.

Thing is, my browser isn’t configured that way. So works well, I guess.

Re: Web Bot Auth

#37

Why use a "web bot" instead of an API? Either can be driven by an AI "agent"...but this just seems like an "API key for a visual api interface", and rather wasteful in cost and resources. If a company could afford to pay a partner for an API key they wouldn't need this. If they can't afford to pay the partner for access -- they'd still be blocked with or without "Web Bot Auth". I don't understand what this is for. I…

The website the human sees is the new API.

That's needed because many APIs are either nonexistent or extremely marginal in design and content coverage.

Re: Web Bot Auth

#38

Earlier quoted context omitted.

https://anubis.techaro.lol/ ?

Your browser is configured to disable cookies. Anubis requires cookies for the legitimate interest of making sure you are a valid client. Please enable cookies for this domain. Thing is, my browser isn’t configured that way. So works well, I guess.

The target was better than cloudflare, which also demands cookies but with more tracking. This is still better.

Re: Web Bot Auth

#39

No offense, but screw CloudFlare, screw their captchas for humans, and screw their wedging themselves between web operators and web users. They can offer what they want for bots. But stop ruining the experience for humans first.

> screw their wedging themselves between web operators and web users Web operators choose to use them; hell they even pay Cloudflare to be between them. Seriously I just think you don't understand how bad it is to run a site without someone in-front of it.

They don't have to, but they're tricked into doing so. Via marketing.

Re: Web Bot Auth

#40
post #39

Earlier quoted context omitted.

> screw their wedging themselves between web operators and web users Web operators choose to use them; hell they even pay Cloudflare to be between them. Seriously I just think you don't understand how bad it is to run a site without someone in-front of it.

They don't have to, but they're tricked into doing so. Via marketing.

I miss the 90s, too, but these days anyone who wants to deal with current levels of bot traffic is probably going to look at a service like Cloudflare as much cheaper than the amount of ops time they’d otherwise spend keeping things up and secure.
Post reply on HN