account wall :-(
Cloudflare Introduces Default Blocking of A.I. Data Scrapers
51–60 of 342 posts
Re: Cloudflare Introduces Default Blocking of A.I. Data Scrapers
#52No one else can really do this except Cloudflare.
Re: Cloudflare Introduces Default Blocking of A.I. Data Scrapers
#53I’ve been using this for a while on my mastodon server and after a few tweaks to make sure it wasn’t blocking legit traffic it’s been really working great. Between Microsoft and Meta, they were hitting my services more than any other traffic combined which says a lot of you know how noisy mastodon can be. Server load went down dramatically. It also completely put a stop to perplexity as far as I can tell. And the rob…
This does nothing dude. Literally nothing. OpenAI or whoever are just going to hire people like me who dont get caught. Stop ruining the experience of users and allowing cf to fill the internet with more bloated javascript challenge pages and privacy invading fingerprinting. Stop making cf the police of the internet. We're literally handing the internet to this company on a silver platter to do MITM attacks on our privacy and god knows what else. Fucking wild.
Re: Cloudflare Introduces Default Blocking of A.I. Data Scrapers
#54I’ve been using this for a while on my mastodon server and after a few tweaks to make sure it wasn’t blocking legit traffic it’s been really working great. Between Microsoft and Meta, they were hitting my services more than any other traffic combined which says a lot of you know how noisy mastodon can be. Server load went down dramatically. It also completely put a stop to perplexity as far as I can tell. And the rob…
Have you considered Anubis? I know it's harder to install, but personally, I think the point of Mastodon is trying to avoid centralization where possible, and CloudFlare is one of the corporations that are keeping the internet centralized.
Re: Cloudflare Introduces Default Blocking of A.I. Data Scrapers
#55> If A.I. companies freely use data from various websites without permission or payment, people will be discouraged from creating new digital content I don't see a way out of this happening. AI fundamentally discourages other forms of digital interaction as it grows. Its mechanism of growing is killing other kinds of digital content. It will eventually kill the web, which is, ironically, its main source of food.
Yes what everyone wants to do with AI: generate entertainment and interactions with humans, including economical ones, will need to happen or AI will starve.
That's what is going to make it starve. Belly full, but of its own shit being tossed around humans seeking cheap copouts of doing actual work.
Re: Cloudflare Introduces Default Blocking of A.I. Data Scrapers
#56> When you enable this feature via a pre-configured managed rule, Cloudflare can detect and block verified AI bots that comply with robots.txt and respect crawl rates, and do not hide their behavior from your website. The rule has also been expanded to include more signatures of AI bots that do not follow the rules. We already know companies like Perplexity are masking their traffic. I'm sure there's more than meets…
Next line: > The rule has also been expanded to include more signatures of AI bots that do not follow the rules. The Block AI Bots rule on the Super Bot Fight Mode page does filter out most bot traffic. I was getting 10x the traffic from bots than I was from users. It definitely doesn't rely on robots.txt or user agent. I had to write a page rule bypass just to let my own tooling work on my website after enabling it.
How many of those "bots" you are filtering are actually bots and how many are regular users buttflare has misidentified as bots?
Re: Cloudflare Introduces Default Blocking of A.I. Data Scrapers
#57I assume they will "protect original content online" by blocking LLM clients from ingesting data as context?
I'm not optimistic that you can effectively block your original content from ending up in training sets by simply blocking the bots. For now I just assume that anything I put online will end up being used to train some LLM
Re: Cloudflare Introduces Default Blocking of A.I. Data Scrapers
#58This is a bit silly. Slowing down, yes, but blocking? People who *really* want that content will find a way and this will hit everyone else instead that will have to do silly riddles before following every link or run crypto mining for them before being shown the content . I recently went to a big local auction site on which I buy frequently and I got one of these "we detected unusual traffic from your network" messa…
I was chatting with my sister last weekend. As a hobby, she creates and sells wedding invitation and other designs at an online marketplace site called Zazzle. She was telling me all about how that site implemented some automatic bot detection and it sounded like it was a total disaster. Real content creators were getting wrongly flagged as bots and then getting blocked from using the site just for using the most fundamental site functionality, and to make it worse, then it was apparently impossible for them to get past the captcha challenge or whatever it showed next. She forwarded me a link to some support forum discussion about it and it was mindboggling the troubles that some of the content creators there had to go through:
https://community.zazzle.com/t5/technical-issues/bot-test-wo...
My sister said that her sales figures are way down compared to what they used to be and she didn't know if this bot flagger was disrupting real paying customers too. She said it had flagged her a couple of times, although she was luckily able to get past the bot challenge. She has pretty much given up on making and uploading new designs because of what was happening to other content creators there. She's now scared to use the site because she doesn't want to get wrongly locked out of her account.
Re: Cloudflare Introduces Default Blocking of A.I. Data Scrapers
#59> If A.I. companies freely use data from various websites without permission or payment, people will be discouraged from creating new digital content I don't see a way out of this happening. AI fundamentally discourages other forms of digital interaction as it grows. Its mechanism of growing is killing other kinds of digital content. It will eventually kill the web, which is, ironically, its main source of food.
Additionally, ad blocker usage is apparently at 30%. So it's a redundant or more nuanced argument, really.
Ad blockers only discourage commercialized content creation, not all of it. IMO that actually improves the quality of the content created.
Re: Cloudflare Introduces Default Blocking of A.I. Data Scrapers
#60I've heard lots of people on HN complaining about bot traffic bogging down their websites, and as a website operator myself I'm honestly puzzled. If you're already using Cloudflare, some basic cache configuration should guarantee that most bot traffic hits the cache and doesn't bog down your servers. And even if you don't want to do that, bandwidth and CPU are so cheap these days that it shouldn't make a difference.…
The presumption I’m already using cloudfare is a start. Is this a requirement for maintaining a simple website now?