Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The web does not need gatekeepers: Cloudflare’s new “signed agents” pitch

positiveblue.substack.com

Post reply on HN
Print view View original

351–360 of 520 posts

Re: The web does not need gatekeepers: Cloudflare’s new “signed agents” pitch

#351

Post by giardini »

Twirrim wrote: post #328
arjie wrote: post #276

Earlier quoted context omitted.

The dream is real, man. If you want open content on the Internet, it's never been a better time. My blog is open to all - machine or man. And it's hosted on my home server next to me. I don't see why anyone would bother trying to distinguish humans from AI. A human hitting your website too much is no different from an AI hitting your website too much. I have a robots.txt that tries to help bots not get stuck in loops…

The only bot that bugs the crap out of me is Anthropic's one. They're the reason I set up a labyrinth using iocaine ( https://iocaine.madhouse-project.org/ ). Their bot was absurdly aggressive, particularly with retries. It's probably trivial in the whole scheme of things, but I love that anthropic spent months making about 10rps against my stupid blog, getting markov chain responses generated from the text of Moby D…

No wonder Anthropic isn't working well! The "Moby Dicked" explanation of the state of AI!

But seriously, Why must someone search even a significant part of the public Internet to develop an AI? Is it believed that missing some text will cripple the AI?

Isn't there some sort of "law of diminishing returns" where, once some percentage of coverage is reached, further scraping is not cost-effective?

Re: The web does not need gatekeepers: Cloudflare’s new “signed agents” pitch

#352

Post by krapp »

1gn15 wrote: post #347
BobaFloutist wrote: post #243

Earlier quoted context omitted.

What if I want my content freely available to humans, and not to bots? Why is that such an insane, unworkable ask? All I want is a copyleft protection that specifically allows humans to access my work to their heart's content, but disallows AI use of it in any form. Is that truly so unreasonable?

Yes, it is an unreasonable and absurd ask. You cannot want freedom while restricting it. You forget that it is people that use AI agents, essentially, being cyborgs. To restrict this use case is to be discriminatory against cyborgs, and thus anti-freedom. We are lucky that there is no way to detect it.

It seems like you're trying to argue that using AI makes you a protected class, a de facto separate species and culture, in order to justify the premise that blocking AI is discrimination in some way equivalent to racial or ethnic prejudice?

If so, no. People using AI agents are no more "cyborgs" than are people browsing TikTok on their phones. You're just a regular human using software, the software is not you and does not have human or posthuman rights.

Re: The web does not need gatekeepers: Cloudflare’s new “signed agents” pitch

#353

Post by fny »

PaulHoule wrote: post #255
SamBam wrote: post #254

Earlier quoted context omitted.

But many people feel that the very act of incorporating your copyrighted words into their for-profit training set is itself the bad behavior. It's not about rate-limiting scrapers, it's letting them in the door in the first place.

Why was it OK for Google to incorporate their words into a for-profit search index which has increasingly sucked all the profit out of the system? My Ithaca friends on Facebook complain incessantly about the very existence of AI to the extent that I would not want to say I ask Copilot how to use Windows Narrator or Junie where the CSS that makes this text bold or sometimes have Photoshop draw an extra row of bricks i…

There's something more obviously nefarious and existential about AI. It takes the idea of "you are the product" to a whole new level.

Re: The web does not need gatekeepers: Cloudflare’s new “signed agents” pitch

#354

Post by culi »

esseph wrote: post #77
viktorcode wrote: post #68

Earlier quoted context omitted.

AI poisoning is a better protection. Cloudflare is capable of serving stashes of bad data to AI bots as protective barrier to their clients.

AI poisoning is going to get a lot of people killed, be cause the AI won't stop being used.

The current state of the art in AI poisoning is Nightshade from the University of Chicago. It's meant to eventually be an addon to their WebGlaze[1] which is an invite-only tool meant for artists to protect their art from AI mimicry

Nobody is dying because artists are protecting their art

[0] https://nightshade.cs.uchicago.edu/whatis.html

[1] https://glaze.cs.uchicago.edu/webglaze.html

Re: The web does not need gatekeepers: Cloudflare’s new “signed agents” pitch

#355

Post by didibus »

lucb1e wrote: post #327
didibus wrote: post #315

Earlier quoted context omitted.

Shouldn't this be seen as success? You weren't a normal user, you were trying to penetrate the site, and you got a bunch of friction?

On the first fricking pageload I got blocked and couldn't open it at all, no captcha shown. That's a success only insofar as you want to exclude random people who don't have a second person whose cookie state to copy Also mind that not every request we make is malicious. A lot of it is also seeing what's even there, doing baseline requests, normal things. I didn't get the impression that I got blocked more on malicio…

I hear you, but I find it suspicious. I mean CloudFront is used by over 10% of all CDN content online, and is used by Amazon itself.

It wouldn't just randomly block something.

It must be based on something no?

Re: The web does not need gatekeepers: Cloudflare’s new “signed agents” pitch

#356

Post by culi »

verdverm wrote: post #147
viktorcode wrote: post #68

Earlier quoted context omitted.

AI poisoning is a better protection. Cloudflare is capable of serving stashes of bad data to AI bots as protective barrier to their clients.

You don't think that the AI companies will take efforts to detect and filter bad data for training? Do you suppose they are already doing this, knowing that data quality has an impact on model capabilities?

The current state of the art in AI poisoning is Nightshade from the University of Chicago. It's meant to eventually be an addon to their WebGlaze[1] which is an invite-only tool meant for artists to protect their art from AI mimicry

If these companies are adding extra code to bypass artists trying to protect their intellectual property from mimicry then that is an obvious and egregious copyright violation

More likely it will push these companies to actually pay content creators for the content they work on to be included in their models.

[0] https://nightshade.cs.uchicago.edu/whatis.html

[1] https://glaze.cs.uchicago.edu/webglaze.html

Re: The web does not need gatekeepers: Cloudflare’s new “signed agents” pitch

#357

Post by paool »

TIPSIO wrote: post #20

Everyone loves the dream of a free for all and open web. But the reality is how can someone small protect their blog or content from AI training bots? E.g.: They just blindly trust someone is sending Agent vs Training bots and super duper respecting robots.txt? Get real... Or, fine what if they do respect robots.txt, but they buy the data that may or may not have been shielded through liability layers via "licensed d…

You can't trust everyone will be polite or follow "standards".

However, you can incentivize good behavior. Let's say there's a scraping agent, you could make a x402 compatible endpoint and offer them a discount or something.

Kinda like piracy; if you offer a good, simple, cheap service people will pay for it versus go through the hassle of pirating.

Re: The web does not need gatekeepers: Cloudflare’s new “signed agents” pitch

#358

Post by didibus »

nromiun wrote: post #45
didibus wrote: post #23

Earlier quoted context omitted.

> And there is no free CDN like Cloudflare. Their pricing page says: No-nonsense Free Tier As part of the AWS free Usage Tier you can get started with Amazon CloudFront for free. Included in Always Free Tier 1 TB of data transfer out to the internet per month 10,000,000 HTTP or HTTPS Requests per month 2,000,000 CloudFront Function invocations per month 2,000,000 CloudFront KeyValueStore reads per month 10 Distributi…

1 TB per month of data is literally nothing. A kid could rent a VPS for an hour and drain all that. What do you do after that? AWS is not going to stop your bill going up is it? I don't care about any of those fancy serverless services. I am just talking about the cheapest CDN.

Ah, for cheapest CDN, maybe you're right. I think BlazingCDN can also be cheap, but CLoudFlare might be the best deal. OP didn't really say there wasn't any cheaper alternative, just said "no real good alternatives".

Re: The web does not need gatekeepers: Cloudflare’s new “signed agents” pitch

#359

Post by didibus »

rustc wrote: post #116
didibus wrote: post #23

Earlier quoted context omitted.

> And there is no free CDN like Cloudflare. Their pricing page says: No-nonsense Free Tier As part of the AWS free Usage Tier you can get started with Amazon CloudFront for free. Included in Always Free Tier 1 TB of data transfer out to the internet per month 10,000,000 HTTP or HTTPS Requests per month 2,000,000 CloudFront Function invocations per month 2,000,000 CloudFront KeyValueStore reads per month 10 Distributi…

> Included in Always Free Tier > 1 TB of data Someone can rent a 1Gbps server for cheap (under $50 on OVH) and pull 330TB in a month from your site. That's about $30k of egress on AWS if you don't do anything to stop it.

True, CloudFlare DDoS protection is unmatched, they just eat the cost for free.

Re: The web does not need gatekeepers: Cloudflare’s new “signed agents” pitch

#360

Post by esperent »

palmfacehn wrote: post #22
Operyl wrote: post #8

Earlier quoted context omitted.

They're getting to the point of 200-300RPS for some of my smaller marketing sites, hallucinating URLs like crazy. It's fucking insane.

You'd think they would have an interest in developing reasonable crawling infrastructure, like Google, Bing or Yandex. Instead they go all in on hosts with no metering. All of the search majors reduce their crawl rate as request times increase. On one hand these companies announce themselves as sophisticated, futuristic and highly-valued, on the other hand we see rampant incompetence, to the point that webmasters eve…

I suspect it's because they're dealing with such unbelievable levels of bandwidth and compute for training and inference that the amount required to blast the entire web like this barely registers to them.
Post reply on HN