Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The web does not need gatekeepers: Cloudflare’s new “signed agents” pitch

positiveblue.substack.com

Post reply on HN
Print view View original

261–270 of 520 posts

Re: The web does not need gatekeepers: Cloudflare’s new “signed agents” pitch

#261

Post by chongli »

PaulHoule wrote: post #255
SamBam wrote: post #254

Earlier quoted context omitted.

But many people feel that the very act of incorporating your copyrighted words into their for-profit training set is itself the bad behavior. It's not about rate-limiting scrapers, it's letting them in the door in the first place.

Why was it OK for Google to incorporate their words into a for-profit search index which has increasingly sucked all the profit out of the system? My Ithaca friends on Facebook complain incessantly about the very existence of AI to the extent that I would not want to say I ask Copilot how to use Windows Narrator or Junie where the CSS that makes this text bold or sometimes have Photoshop draw an extra row of bricks i…

They were okay with it when Google was sending them traffic. Now they often don’t. They’ve broken the social contract of the web. So why should the sites whose work is being scraped be expected to continue upholding their end?

Re: The web does not need gatekeepers: Cloudflare’s new “signed agents” pitch

#262

Post by bayindirh »

wvenable wrote: post #233
bayindirh wrote: post #187

Earlier quoted context omitted.

The problem is not AI bot scraping, per se, but "AI bot scraping while disregarding all licenses and ethical considerations". Freedom, the word, while implies no boundaries, is always bound by ethics, mutual respect and "do no harm" principle. The moment you trip either one of these wires and break them, the mechanisms to counter it becomes active. Then we cry "but, freedom?!". Freedom also contains the consequences…

> The problem is not AI bot scraping, per se, but "AI bot scraping while disregarding all licenses and ethical considerations". What licenses? Free and open web. Go crazy. What ethical considerations? Do I police how users use the information on my site? No. If they make a pipe bomb using an 6502 CPU using code taken from my website -- am I supposed to do something about that?

Creative Commons, GFDL, Unlicense, GPL/AGPL, MIT, WTFPL. Go crazy. I have the freedom to police how users use the information on my site. Yes.

Real examples: My blog is BY-NC-SA and digital garden is GFDL. You can't take them, mangle and sell them. Especially, the blog.

AI companies take these posts, and sell derivatives, without any references, consent or compensation. BY-NC-SA is complete opposite of what they do.

This is why I'm not uploading any photos I take publicly anymore.

Re: The web does not need gatekeepers: Cloudflare’s new “signed agents” pitch

#263

Post by [deleted] »

hombre_fatal wrote: post #244
ctoth wrote: post #238

Earlier quoted context omitted.

You make unauthenticated requests cheap enough that you don't care about volume. Reserve rate limiting for authenticated users where you have real identity. The open web survives by being genuinely free to serve, not by trying to guess who's "real." A basic Varnish setup should get you most of the way there, no agent signing required!

Your response to unauthenticated requests could be Hello world served from memory and your server/link will still fail under a volumetric attack, and you still get the pleasure of paying for the bandwidth. So no, this advice has been outdated for decades. Also you're doing some sort of victim blaming where everyone on earth has to engineer their service to withstand DoS instead of outsourcing that to someone else. Ab…

[deleted]

Re: The web does not need gatekeepers: Cloudflare’s new “signed agents” pitch

#264

Post by BrenBarn »

PaulHoule wrote: post #255
SamBam wrote: post #254

Earlier quoted context omitted.

But many people feel that the very act of incorporating your copyrighted words into their for-profit training set is itself the bad behavior. It's not about rate-limiting scrapers, it's letting them in the door in the first place.

Why was it OK for Google to incorporate their words into a for-profit search index which has increasingly sucked all the profit out of the system? My Ithaca friends on Facebook complain incessantly about the very existence of AI to the extent that I would not want to say I ask Copilot how to use Windows Narrator or Junie where the CSS that makes this text bold or sometimes have Photoshop draw an extra row of bricks i…

> Why was it OK for Google to incorporate their words into a for-profit search index which has increasingly sucked all the profit out of the system?

It wasn't okay, it's just that the reasons it wasn't okay didn't become apparent until later.

> The same people seem to have no problem with Facebook using their words for all things Facebook uses them for, however.

Many of those people will likely have a problem with it later, for reasons that are happening now but that they won't become fully aware of until later.

Re: The web does not need gatekeepers: Cloudflare’s new “signed agents” pitch

#265

Post by [deleted] »

ctoth wrote: post #217

The web doesn't need attestation. It doesn't need signed agents. It doesn't need Cloudflare deciding who's a "real" user agent. It needs people to remember that "public" means PUBLIC and implement basic damn rate limiting if they can't handle the traffic. The web doesn't need to know if you're a human, a bot, or a dog. It just needs to serve bytes to whoever asks, within reasonable resource constraints. That's it. Th…

[deleted]

Re: The web does not need gatekeepers: Cloudflare’s new “signed agents” pitch

#266

Post by ctoth »

hombre_fatal wrote: post #244
ctoth wrote: post #238

Earlier quoted context omitted.

You make unauthenticated requests cheap enough that you don't care about volume. Reserve rate limiting for authenticated users where you have real identity. The open web survives by being genuinely free to serve, not by trying to guess who's "real." A basic Varnish setup should get you most of the way there, no agent signing required!

Your response to unauthenticated requests could be Hello world served from memory and your server/link will still fail under a volumetric attack, and you still get the pleasure of paying for the bandwidth. So no, this advice has been outdated for decades. Also you're doing some sort of victim blaming where everyone on earth has to engineer their service to withstand DoS instead of outsourcing that to someone else. Ab…

We started with "AI crawlers are too aggressive" and you've escalated to volumetric DDoS. These aren't the same problem. OpenAI hitting your API too hard is solved by caching, not by Cloudflare deciding who gets an "agent passport."

"Victim blaming"? Can we please leave these therapy-speak terms back in the 2010s where they belong and out of technical discussions? If expecting basic caching is victim blaming, then so is expecting HTTPS, password hashing, or any technical competence whatsoever.

Your decentralization point actually proves mine: yes, attackers distribute while defenders centralize. That's why we shouldn't make centralization mandatory! Right now you can choose Cloudflare. With attestation, they become the web's border control.

The fine article makes it clear what this is really about - Cloudflare wants to be the gatekeeper for agent traffic. Agent attestation doesn't solve volumetric attacks (those need the DDoS protection they already sell, no new proposal required!) They're creating an allowlist where they decide who's "legitimate."

But sure, let's restructure the entire web's trust model because some sites can't configure a cache. That seems proportional.

Re: The web does not need gatekeepers: Cloudflare’s new “signed agents” pitch

#267

Post by bayindirh »

tliltocatl wrote: post #204
bayindirh wrote: post #187

Earlier quoted context omitted.

The problem is not AI bot scraping, per se, but "AI bot scraping while disregarding all licenses and ethical considerations". Freedom, the word, while implies no boundaries, is always bound by ethics, mutual respect and "do no harm" principle. The moment you trip either one of these wires and break them, the mechanisms to counter it becomes active. Then we cry "but, freedom?!". Freedom also contains the consequences…

The problem isn't "AI bot scraping while disregarding all licenses and ethical considerations". The problem is "AI bot scraping while ignoring every good practice to reduce bandwidth usage".

If you ask me "every good practice to reduce bandwidth usage" falls under ethics pretty squarely, too.

While this is certainly a problem, it's not the only problem.

Re: The web does not need gatekeepers: Cloudflare’s new “signed agents” pitch

#268

Post by novemp »

PaulHoule wrote: post #255
SamBam wrote: post #254

Earlier quoted context omitted.

But many people feel that the very act of incorporating your copyrighted words into their for-profit training set is itself the bad behavior. It's not about rate-limiting scrapers, it's letting them in the door in the first place.

Why was it OK for Google to incorporate their words into a for-profit search index which has increasingly sucked all the profit out of the system? My Ithaca friends on Facebook complain incessantly about the very existence of AI to the extent that I would not want to say I ask Copilot how to use Windows Narrator or Junie where the CSS that makes this text bold or sometimes have Photoshop draw an extra row of bricks i…

[flagged]

Re: The web does not need gatekeepers: Cloudflare’s new “signed agents” pitch

#269

Post by Gud »

PaulHoule wrote: post #258
lucb1e wrote: post #214

Earlier quoted context omitted.

If you're handling billions of requests per second, you're not a self hoster. That's a commercial service with a dedicated team to handle traffic around the clock. Most ISPs probably don't even operate lines that big To put that in perspective, even if they're sending empty TCP packets, "several billion" pps is 200 to 1800 gigabits of traffic, depending on what you mean by that. Add a cookieless HTTP payload and you'…

If somebody decides they hate you, your site that could handle, say, 100,000 legitimate requests per day could suddenly get billions of illegitimate requests.

Not everybody wants to manage some commercial grade packet filter that can handle some DDoSing script kiddie, it’s a strong argument.

But another argument against using the easiest choice, the near monopoly, is that we need a diverse, thriving ecosystem.

We don’t want to end up in a situation where suddenly Cloudflare gets to dictate what is allowed on the web.

We have already lost email to the tech giants, try running your own mail sometime. The technical aspect is easy, the problem is you will end up in so many spam folders it’s disgusting.

What we need are better decentralized protocols.

Re: The web does not need gatekeepers: Cloudflare’s new “signed agents” pitch

#270

Post by krapp »

IshKebab wrote: post #259
krapp wrote: post #184

Earlier quoted context omitted.

Both whitelist and allowlist are equally normal and good. It's weird that people will claim that "politics" have no place in software while insisting that there is one and only one term "normal" people should use because the politics of the people who object to it are bad and wrong.

> Both whitelist and allowlist are equally normal and good. Good is debatable, but normal ?? No, obviously not. One is a word that has been around for over a hundred years and is understood by everyone that speaks English; the other is like 4 years old and only used by some software nerds. 90% of normal people would not know what it means.

90% of normal people have never heard of a whitelist either, but any English speaker could intuit what "allowlist" means more easily that "whitelist" without context. And both are technical terms of art in the context of this conversation, so what 90% of normal people would or wouldn't understand isn't even relevant.

And "enshittification" is even newer than "allowlist" and it's practically mainstream.

Post reply on HN