Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The web does not need gatekeepers: Cloudflare’s new “signed agents” pitch

positiveblue.substack.com

Post reply on HN
Print view View original

211–220 of 520 posts

Re: The web does not need gatekeepers: Cloudflare’s new “signed agents” pitch

#211

Post by xdennis »

thanatos_dem wrote: post #88
IshKebab wrote: post #83

[flagged]

Allowlist is arguably fitting for a list of things which are allowed.

There are so many terms in software which are nonsensical (starting with "computer science") which could be fixed.

The problem with changing whitelist to "allowlist" is that it implies that people who use whitelist are racists. You're not just virtue signaling (and confusing my spellchecker) but causing discord.

It would be perfectly fine if people switched to "allowlist" because they think it's a better term, but that's not the reason. They do it because they want to virtue signal or they're afraid of their peers (because they'll be called racists).

Using "allowlist" is actually bad because it gives agitators power and they keep changing more words to get more power.

Re: The web does not need gatekeepers: Cloudflare’s new “signed agents” pitch

#212

Post by nc0 »

rsync wrote: post #182
TIPSIO wrote: post #20

Everyone loves the dream of a free for all and open web. But the reality is how can someone small protect their blog or content from AI training bots? E.g.: They just blindly trust someone is sending Agent vs Training bots and super duper respecting robots.txt? Get real... Or, fine what if they do respect robots.txt, but they buy the data that may or may not have been shielded through liability layers via "licensed d…

“But the reality is how can someone small protect their blog or content from AI training bots?” Why would you need to? If your inability to assemble basic HTML forces you to adopt enormous, bloated frameworks that require two full cores of a cpu to render your post… … or if you think your online missives are a step in the road to content creator riches … … then I suppose I see the problem. Otherwise there’s no proble…

It's not a question of languages or frameworks, but hardware. I cannot finance servers large enough to keep up with AI bots constantly scrapping my host, bypassing cache indications, or changing IP to avoid bans.

Re: The web does not need gatekeepers: Cloudflare’s new “signed agents” pitch

#213

Post by bobbiechen »

avtar wrote: post #36

> An allowlist run by ONE company? An allowlist run by one company that site owners chose to engage with. But the irony of taking an ideological stance about fairness while using AI generated comics for blog posts…

Cloudflare is implementing the (still-emerging) Web Bot Auth standard. We're working on the same at Stytch for https://IsAgent.dev .

The discourse around this is a little wild and I'm glad you said this. The allowlist is a Cloudflare feature and their customers are free to use it. The core functionality involving HTTP Message Signatures is decentralized and open, so anyone can adopt it and benefit.

Re: The web does not need gatekeepers: Cloudflare’s new “signed agents” pitch

#214

Post by lucb1e »

victorbjorklund wrote: post #137
Gud wrote: post #134

Earlier quoted context omitted.

What’s so painful about self hosting? I’ve been self hosting since before I hit puberty. If 12 year old me can run a httpd, anyone can. And if you don’t want to self host, at least try to use services from organisations that aren’t hostile to the open web

I self-host lots of stuff. But yes it is more pain to host a WAF that can handle billions of request per minute. Even harder to do it for free like Cloudflare. And in the end the end result for the user is exactly the same if you use a self-hosted WAF or let someone else host it for you.

If you're handling billions of requests per second, you're not a self hoster. That's a commercial service with a dedicated team to handle traffic around the clock. Most ISPs probably don't even operate lines that big

To put that in perspective, even if they're sending empty TCP packets, "several billion" pps is 200 to 1800 gigabits of traffic, depending on what you mean by that. Add a cookieless HTTP payload and you're at many terabits per second. The average self hoster is more likely to get struck by lightning than encounter and need protection from this (even without considering the, probably modest, consequences of being offline a few hours if it does happen)

Edit: off by a factor of 60, whoops. Thanks to u/Gud for pointing that out. I stand by the conclusion though: less likely to occur than getting struck by lightning (or maybe it's around equally likely now? But somewhere in that ballpark) and the consequences of being down for a few hours are generally not catastrophic anyway. You can always still put big brother in front if this event does happen to you and your ISP can't quickly drop the abusive traffic

Re: The web does not need gatekeepers: Cloudflare’s new “signed agents” pitch

#215

Post by [deleted] »

wilson090 wrote: post #176
BrenBarn wrote: post #175

Earlier quoted context omitted.

I think that was the point. Everyone loves the dream, but the reality is different.

How so? If you don't want AI bots reading information on the web, you don't actually want a free and open web. The reality of an open web is that such information is free and available for anyone.

[deleted]

Re: The web does not need gatekeepers: Cloudflare’s new “signed agents” pitch

#216

Post by BoredPositron »

TIPSIO wrote: post #20

Everyone loves the dream of a free for all and open web. But the reality is how can someone small protect their blog or content from AI training bots? E.g.: They just blindly trust someone is sending Agent vs Training bots and super duper respecting robots.txt? Get real... Or, fine what if they do respect robots.txt, but they buy the data that may or may not have been shielded through liability layers via "licensed d…

We have thousands of engineers of these companies right here on hackernews and they cry and scream about privacy and data governance on every topic but their own work. If you guys need a mirror to do some self reflection I am offering to buy.

Re: The web does not need gatekeepers: Cloudflare’s new “signed agents” pitch

#217

Post by ctoth »

The web doesn't need attestation. It doesn't need signed agents. It doesn't need Cloudflare deciding who's a "real" user agent. It needs people to remember that "public" means PUBLIC and implement basic damn rate limiting if they can't handle the traffic.

The web doesn't need to know if you're a human, a bot, or a dog. It just needs to serve bytes to whoever asks, within reasonable resource constraints. That's it. That's the open web. You'll miss it when it's gone.

Re: The web does not need gatekeepers: Cloudflare’s new “signed agents” pitch

#218

Post by Loughla »

pton_xd wrote: post #197
wilson090 wrote: post #176

Earlier quoted context omitted.

How so? If you don't want AI bots reading information on the web, you don't actually want a free and open web. The reality of an open web is that such information is free and available for anyone.

> If you don't want AI bots reading information on the web, you don't actually want a free and open web. This is such a bad faith argument. We want a town center for the whole community to enjoy! What, you don't like those people shooting up drugs over there? But they're enjoying it too, this is what you wanted right? They're not harming you by doing their drugs. Everyone is enjoying it!

Set aside that there's a pretty big difference between AI scraping and illegal drug usage.

If the person using illegal drugs is on no way harming anyone but themselves and not being a nuisance, then yeah, I can get behind that. Put whatever you want in your body, just don't let it negatively impact anyone around you. Seems reasonable?

Re: The web does not need gatekeepers: Cloudflare’s new “signed agents” pitch

#219

Post by davepeck »

TIPSIO wrote: post #20

Everyone loves the dream of a free for all and open web. But the reality is how can someone small protect their blog or content from AI training bots? E.g.: They just blindly trust someone is sending Agent vs Training bots and super duper respecting robots.txt? Get real... Or, fine what if they do respect robots.txt, but they buy the data that may or may not have been shielded through liability layers via "licensed d…

> Everyone loves the dream of a free for all and open web. But the reality is how can someone small protect their blog or content from AI training bots?

I'm old enough to remember when people asked the same questions of Hotbot, Lycos, Altavista, Ask Jeeves, and -- eventually -- Google.

Then, as now, it never felt like the right way to frame the question. If you want your content freely available, make it freely available... including to the bots. If you want your content restricted, make it restricted... including to the humans.

It's also not clear to me that AI materially changes the equation, since Google has for many years tried to cut out links to the small sites anyway in favor of instant answers.

(FWIW, the big companies typically do honor robots.txt. It's everyone else that does what they please.)

Re: The web does not need gatekeepers: Cloudflare’s new “signed agents” pitch

#220

Post by unethical_ban »

wvenable wrote: post #172
TIPSIO wrote: post #20

Everyone loves the dream of a free for all and open web. But the reality is how can someone small protect their blog or content from AI training bots? E.g.: They just blindly trust someone is sending Agent vs Training bots and super duper respecting robots.txt? Get real... Or, fine what if they do respect robots.txt, but they buy the data that may or may not have been shielded through liability layers via "licensed d…

> Everyone loves the dream of a free for all and open web... But the reality is how can someone small protect their blog or content from AI training bots? Aren't these statements entirely in conflict? You either have a free for all open web or you don't. Blocking AI training bots is not free and open for all.

That's a very "BSD is freedom and GPL isn't" kind of philosophy.

Nothing is truly free unless you give equal respect to fellow hobbyists and megacorps using your labor for their profit.

Post reply on HN